Patient Safety

Monitoring Patient Safety and Quality: A Simple Framework

By Christine Goeschel, R.N.

Results from two national hospital leadership surveys conducted in 2006—one by the American Hospital Association’s Health Research & Educational Trust and the other from the San Diego-based Governance Institute—gave mixed reviews to existing tools that help boards track their hospital’s progress on patient safety and quality-of-care improvement activities. Both surveys interviewed CEOs, senior administrators and trustees.

The surveys affirmed that trustees and hospital leaders accept their responsibility to help solve quality and safety problems in health care. In so doing, they understand they will also need to address the myriad pressures that health care policy-makers, regulators, employers, consumers, payers and accrediting agencies are exerting on them. Results also indicated that trustees and senior administrators are willing to own this challenge, acknowledging that solutions will require the support of broad coalitions within and outside the health care industry.

But surveyed trustees and administrators also noted that current measures used to evaluate progress in patient safety are neither broad nor deep enough to provide an adequate evaluation and many are of dubious validity. Without rigorous and standardized measurement, stakeholders are therefore left wondering whether care is really any safer.

Unfortunately, a clear and standardized national framework to measure and report quality and safety performance does not yet exist. Although the Centers for Medicare & Medicaid Services (CMS) publicly report a few standardized hospital quality indicators, these reflect an extremely small portion of health care services.

A compendium of safety scorecards made available in early 2007 by the Agency for Healthcare Research and Quality (AHRQ) shows wide variation in the types of safety and quality measures used in hospitals. Measures usually target specific populations, such as heart attack patients, and thus have limited utility as indicators of overall quality.

Moreover, data are often reported by hospitals as rates (e.g., medication errors) when, in fact, they do not satisfy scientific parameters for rate-based measurement. Rate-based measures require a clearly defined numerator (event) and denominator (those put at risk for the event), as well as a surveillance system for identifying both. Without these defined characteristics, rate-based measures can actually misinform board members and administrators.

In response, we have developed a simple framework (see table) that may be used alone as a tool for boards and quality and safety committees, or as part of a hospital’s balanced scorecard to evaluate performance, refine goals and allocate resources for improvement.

Within the framework, we address the issue of using rates appropriately by stratifying measurements into two categories: valid rate-based measures using evidence-based standards of care; and indicators of care that are essential to patient safety but not measurable as valid rates, such as reports of patient falls.

Relative to the first category, the board should ask the CEO and other quality and safety leaders two key questions—typically reported on a quarterly basis—that address outcome and process measures: “How often do we harm patients?” and “How often do clinical staff provide evidence-based care?” Examples might include hospital rates of infection as a measure of harm and rates of compliance with established cardiac care standards as a measure of evidence-based care.

Relative to the second category, which tracks patient safety in less concrete ways, two key questions that boards might ask the CEO and quality/safety leaders are: “How do we know we have learned from our mistakes?” and “How well have we created a culture of safety?” This category of measurement focuses on structure and context—i.e., the care setting—and can be used regardless of what clinical measures are part of an organization’s scorecard.

These measures track hospital learning processes—such as whether an adverse event (i.e., a foreign object left in the patient during surgery) has been corrected or has recurred—and caregiver perceptions regarding their care environment. For example, are caregivers comfortable speaking up if they see a safety issue? Would they be a patient in their hospital?

Mechanisms already in place to help boards prioritize their focus may be added to this framework. For example, balanced scorecards typically include a broad array of hospital performance measures—e.g., financial, operational, regulatory, quality and safety—and frequently use color-coding to depict performance. Typically, red is used for “needs immediate attention or great improvement,” yellow for “needs close monitoring or continued improvement” and green for “needs monitoring for maintenance of good performance.”

Using this color-coded framework to organize quality and safety metrics within the balanced scorecard can provide complementary value. At a glance, leaders will see not only where additional focus is needed, but also whether patient safety and quality efforts are balanced across the critical dimensions of measuring harm, providing evidence-based care, learning from mistakes, and creating a culture of safety.

CMS pay for performance and public reporting on a select set of hospital quality indicators have expedited the demand for consistent and reliable quality and safety measurement. Yet hospital leaders still need a broader and more robust set of measures to fulfill their quality oversight responsibilities.

Until there are comprehensive standards for accounting and reporting quality and safety performance, however, hospital boards will continue to grapple with quality and safety data that may be confusing and difficult to use. A simple framework to organize efforts and track hospital progress may help boards answer with confidence the questions: “Are we safer?” and “How do we know?”

Christine Goeschel, R.N., is director of patient safety and quality initiatives, Quality and Safety Research Group, Johns Hopkins University School of Medicine, Baltimore. She can be reached at cgoesch1@jhmi.edu.

Score Card Framework for Patient Safety and Effectiveness

Domain	Measure Definition	Performance Measure
How often do we harm patients? (rate-based measures)	Standardized definitions and measures of health care-associated infections from the Centers for Disease Control and Prevention.	Central line-associated bloodstream infection (CLABSI) rates.
How often do we provide needed patient interventions? (rate-based measures)	Centers for Medicare & Medicaid Services (CMS) and Joint Commission standards for use of beta blockers and aspirin in patients with acute myocardial infarction (AMI).	The proportion of patients that receive evidence-based interventions, as reported on the CMS Web site.
How do we know we have learned from defects (i.e, something that should not happen again)? (not rate-based)	Review of defects including development of a mitigation plan with follow-up at each level from board to front-line staff (e.g., liability claims, sentinel events, medica-tion errors, etc.). Frequency is standardized locally by month, quarter, etc.	Proportion of time in which at least one defect is fully addressed at each level of the organization (e.g., a sentinel event is reviewed and a policy is created/revised, and/or staff awareness or use of that policy is measured).
How well have we created a culture of safety? (not rate-based)	Annual assessment of the safety culture at the unit level using a valid survey instrument, such as Joint Commission and National Quality Forum recommendations.	Percentage of patient care areas in which 80 percent of staff report a positive safety and teamwork climat.e

This article 1st appeared in the March 2008 issue of HHN Magazine.

To respond to this article, please click here.