This Article Full Text Full Text (PDF) All Versions of this Article: 116/25/2969    most recent CIRCULATIONAHA.107.703553v1 Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by O’Brien, S. M. Articles by Peterson, E. D. Search for Related Content PubMed PubMed Citation Articles by O’Brien, S. M. Articles by Peterson, E. D. Related Collections Health policy and outcome research

(Circulation. 2007;116:2969-2975.)
© 2007 American Heart Association, Inc.

Health Services and Outcomes Research

Exploring the Behavior of Hospital Composite Performance Measures

An Example From Coronary Artery Bypass Surgery

Sean M. O’Brien, PhD; Elizabeth R. DeLong, PhD; Rachel S. Dokholyan, MPH; Fred H. Edwards, MD; Eric D. Peterson, MD, MPH

From Duke Clinical Research Institute (S.M.O., E.R.D., R.S.D., E.D.P.), Durham, NC, and University of Florida (F.H.E.), Jacksonville, Fla.

Correspondence to Sean M. O’Brien, PhD, Box 17969, Duke Clinical Research Institute, Durham, NC 27715. E-mail obrie027{at}mc.duke.edu

Received March 17, 2007; accepted October 9, 2007.

Background— Composite scores that combine several performance measures into a single ranking are becoming the accepted metric for assessing hospital performance. In particular, the Centers for Medicare & Medicaid Services Hospital Quality Incentive Demonstration (HQID) project bases financial rewards and penalties on these scores. Although the HQID composite calculation is straightforward and easily understood, its method of combining process and outcome measures has not been validated.

Methods and Results— Using data on 530 hospitals from the Society of Thoracic Surgeons National Cardiac Database, we replicated the HQID methodology with 6 nationally endorsed performance measures (5 process measures plus survival) for coronary artery bypass surgery. Composite scores were essentially determined by process measure performance alone; the survival component explained only 4% of the composite score’s total variance. This result persisted even when the survival component was allowed a 5-fold greater weighting in the composite summary. The popular "all-or-none" measurement approach was also dominated by the process component. Substantial disagreement was found among hospital rankings when several alternative methods were used; up to 60% of hospitals eligible for the top financial reward under HQID would change designation depending on the composite methodology used. The application of a simple statistical adjustment (standardization) to each method would provide more consistent results and a more balanced assessment of performance based on both process and outcomes.

Conclusions— Existing methods used to create composite performance measures have remarkably different weighting of process versus outcomes metrics and lead to highly divergent provider rankings. Simple alternative methods can create more balanced process-outcome performance assessments.

---

 

CLINICAL PERSPECTIVE

This article has been cited by other articles:

				M. A. Hlatky and P. A. Heidenreich The Year in Epidemiology, Health Services Research, and Outcomes Research J. Am. Coll. Cardiol., April 21, 2009; 53(16): 1459 - 1466. [Full Text] [PDF]

				R. M. Califf and E. D. Peterson Public reporting of quality measures what are we trying to accomplish? J. Am. Coll. Cardiol., March 10, 2009; 53(10): 831 - 833. [Full Text] [PDF]

				B. D. Richman, K. Udayakumar, W. Mitchell, and K. A. Schulman Lessons From India In Organizational Innovation: A Tale Of Two Heart Hospitals Health Aff., September 1, 2008; 27(5): 1260 - 1270. [Abstract] [Full Text] [PDF]

				E. E. Drye and J. Chen Evaluating Quality in Small-Volume Hospitals Arch Intern Med, June 23, 2008; 168(12): 1249 - 1251. [Full Text] [PDF]

				J. V. Tu and P. C. Austin Cardiac Report Cards: How Can They Be Made Better? Circulation, December 18, 2007; 116(25): 2897 - 2899. [Full Text] [PDF]