As consumers take on greater financial responsibility for their health care, they increasingly want guidance to help them decide which hospital to use.

That's a role a number of organizations hope to fill, including US News & World Report, Consumer Reports and HealthGrades, which offer hospital ratings of different sorts.

But many hospital executives say the report cards fail to give an accurate picture of hospital rankings and together offer conflicting information that may only confuse prospective patients. "When you try to compare them across the continuum, it doesn't make sense," says David Evelyn, M.D., vice president for medical affairs at Cayuga Medical Center, Ithaca, N.Y.

To demonstrate how the major ratings of hospitals vary, executives for a New York hospital association decided to evaluate them and create a report card of report cards.

"A lot of our members were expressing their frustration with the report cards," says Kathy Ciccone, executive director of HANYS Quality Institute, part of the Healthcare Association of New York State, which produced the report card grading system.

Most of the national report cards graded by HANYS received a half- or one-star rating out of a total of a possible four. Inadequate data quality is a common denominator in the lower-rated report cards, but the ratings came as a result of multiple gauges, says Robert Panzer, M.D., associate vice president of patient care quality and safety at the University of Rochester (N.Y.) Medical Center.

On the upside, the Joint Commission's quality rating system and most other government-produced grades in the report were given three-star ratings.

Leah Binder, president and CEO of the Leapfrog Group, which got one star for its hospital safety score, says the HANYS results were unsurprising given that it's a hospital-driven effort. "I'm sure our school-age children would love to be given the opportunity to grade their teachers; call me cynical, but I don't think the toughest teachers would earn straight A's from the school bus crowd," Binder says via email. "Similarly, a hospital association purporting to grade its graders deserves to be taken with a rather large grain of salt," she says.

HANYS officials say they would like to see the report card spur change in the industry. "We're hoping to have a national dialogue on the problems these reports cards cause," Ciccone says.