A Measure of Quality

Improving outcomes is a key source of IT value; try to prove it

Measuring the impact of specific information technologies on individual processes or quality indicators can be difficult enough. Measuring the systemic impact of broad IT investments on quality generally is a monumental task.

For the third consecutive year, Hospitals & Health Networks worked with outside analysts to determine if the 100 Most Wired have demonstrably different mortality rates than other hospitals. Solucient LLC, Evanston, Ill., a performance measurement company, conducted both the 2005 and the 2006 analyses.

The 2005 results were statistically strongest, showing that the nation’s 100 Most Wired have, on average, risk-adjusted mortality rates that are 7.2 percent lower than other hospitals. The conclusion, valid at the 99 percent confidence level, controlled for the size of hospital and teaching status.

The 2006 results showed that the 100 Most Wired have, on average, 1.6 percent lower risk-adjusted mortality rates valid at the 90 percent confidence level. The Most Wired listing consists of both hospitals and systems. In 2006, the total number of hospitals represented in the sample fell, making it statistically more difficult to achieve a higher confidence interval.

Results in 2004 were valid at the 95 percent confidence level, but the absolute difference was less than 1 percent. CareScience conducted the 2004 analytics.

“I don’t think these differences in statistical outcomes change the overall conclusion. The likelihood that you can explain these results as a random occurrence is still pretty small,” says Kaveh Safevi, M.D., Solucient’s chief medical officer. “The problem is that you still can’t sort out cause and effect with any confidence.”

The analysis shows an association between outcomes and IT, but doesn’t establish that the outcomes were caused by the technology. The question is whether technology resulted in the improved outcomes or if implementing technology is a practice that’s found alongside other activities that drive quality.

“You can’t achieve the kinds of improvements in quality that the industry is striving for without greater investments in information technology,” says Lewis Redd, managing partner, Accenture Health Provider Practice, Atlanta. Those investments will drive both quality and operational improvements, he says.

“It would be a mistake, just because you can’t prove causality, to dismiss the role of IT in improving quality. That would be dismissing everything we know about the use of technology to hard wire process improvements,” Safevi says. “First you need a plausible explanation. Then you have to ask, ‘What is the likelihood that these two things occur randomly?’ ”

The answer depends on both infrastructure and adoption, says Steve Mayfield, the American Hospital Association’s senior vice president of quality and performance improvement. Disparate systems must connect, he says, or the power of IT to standardize processes and provide clinical intelligence is thwarted.

“IT must connect across the value stream. Otherwise, all you get are pockets of information,” says Mayfield, director of the AHA Quality Center. “Until the infrastructure is in place, it’s hard to determine that the technology makes much of a difference.”

Yet, by itself, all the infrastructure that money can buy won’t drive quality. “If physicians aren’t using it, we’re back to a limiting factor that constrains value,” Mayfield says.

Billie Waldo, R.N., vice president, McKesson Provider Technologies, Alpharetta, Ga., says that physician adoption is one aspect of the broader challenge of clinician adoption.

“The physicians are important, but don’t forget about the nurses, the pharmacists and the other clinical staff,” Waldo says. “The nurses and ancillary staff are doing most of the clinical work. They’re with the patients. You’re not going to drive value in any way without clinician adoption.”

The goal is to standardize processes and deliver information to caregivers. With both infrastructure and adoption, IT is vital to improving processes and driving quality, Mayfield says.

Accenture’s Redd agrees. “As hospitals get more focused on measuring quality and outcomes, they will find that it requires a great deal more clinical information technology.”

Safevi suggests that demonstrating systemic improvements in quality driven by IT may be moot. “To try to disentangle it would be to suggest that you don’t need clinical IT to improve results. That flies in the face of everything we know about the role of technology in improving quality.”

This article 1st appeared in the July 2006 issue of HHN Magazine.

To respond to this article, please click here.