doi: 10.1161/CIRCULATIONAHA.108.767897
(Circulation. 2008;118:e9.)
© 2008 American Heart Association, Inc.
Correspondence |
Letter by Gale et al Regarding Article, "Which Hospitals Have Significantly Better or Worse Than Expected Mortality Rates for Acute Myocardial Infarction Patients? Improved Risk Adjustment With Present-at-Admission Diagnoses"
Division of Cardiovascular and Diabetes Research, Leeds Institute of Genetics Health and Therapeutics, University of Leeds, Leeds, UK
National Audit of Myocardial Infarction Project, National Institute for Clinical Outcomes Research, The Heart Hospital, London, UK
Biostatistics Unit, Centre for Epidemiology and Biostatistics, University of Leeds, Leeds, UK
We read with great interest the article by Stukenborg et al.1 The article describes the use of a logistic risk adjustment model using patient-at-admission diagnoses in the identification of hospitals that have significantly better or worse than expected mortality rates for acute myocardial infarction patients. The resulting hospital mortality rates were compared with results obtained for the same hospitals with patient-level mortality risk adjustment based on California A model covariates. Three statistical methods were used to identify hospitals as outcome outliers: indirect standardization, adjusted odds ratios, and hierarchical models.
There are various statistical methods for hospital profiling.2,3 In using the adjusted odds ratios and hierarchical models, the authors used patient-level data in the adjustment risk model, which included hospitals as either indicator variables or random effects. By using patient-level data, rather than hospital level data, the statistical power of their analyses was greatly increased, giving results that are more reliable.
Although the statistical analyses are sound and valid, however, whether these methods (and results) are reproducible in remote administrative or clinical cohorts is yet to be seen. That is, the current model used 241 variables, and it is unlikely that other databases collect this many variables. Indeed, although the Myocardial Infarction National Audit Project is an extensive database of acute coronary syndrome admissions to all acute hospitals in England and Wales, at present it only collects 108 patient-level variables.4 Is it really practical to collect more variables? Although it is well known that the more variables entered into a risk adjustment model, the better the c statistic, secondary abstraction of difficult-to-obtain key clinical findings adds little to the predictive power of risk-adjustment scores.5 Statistically, one should aim for parsimony. Therefore, perhaps a conservative model (eg, age, systolic blood pressure, and heart rate or the California Model A covariates, with a c statistic approximating 0.77 would suffice.
A further point is that any increase in the number of outcome-related variables entered into a model will decrease the variation seen in hospital performance. Thus, fewer hospitals will be seen to have better or worse than expected mortality rates. The identification but fair representation of outlying hospitals is important, and therefore the method of analysis requires careful consideration.
Finally, although Stukenborg et al1 plotted paired comparison statistics and showed the correlation coefficient to be in agreement (Figures 1A, 1C, 2A, and 2C in the article), 2 points are worth noting. First, a high correlation does not mean that the 2 methods agree. Second, as can be seen from the plots in the paper, often the data points are clustered along the line of unity, which makes the investigation of between-method differences difficult. The result is that such plots are less informative. Lack of agreement could easily be observed from plots of the differences against the mean.
 |
Acknowledgments |
|---|
Disclosures
None.
|
References |
|---|
 Top References |
|---|
1. Stukenborg GJ, Wagner DP, Harrell FE Jr, Oliver MN, Heim SW, Price AL, Han CK, Wolf AMD, Connors AF Jr. Which hospitals have significantly better or worse than expected mortality rates for acute myocardial infarction patients? Improved risk adjustment with present-at-admission diagnoses. Circulation. 2007; 116: 2960–2968.
2. Marshall EC, Spiegelhalter DJ. Institutional performance. In: Leyland AH, Goldstein H, eds. Multilevel Modelling of Health Statistics. Chichester, UK: Wiley; 2001: 127–142.
3. Austin C. A comparison of Bayesian methods for profiling hospital performance. Medical Decision Making. 2002: 163–172.
4. Birkhead JS, Walker L, Pearson M, Weston C, Cunningham AD, and Rickards AF. Improving care for patients with acute coronary syndromes: initial results from the National Audit of Myocardial Infarction Project (MINAP). Heart. 2004; 90: 1004–1009.
5. Pine M, Jordan HS, Elixhauser A, Fry DE, Hoaglin DC, Jones B, Meimban R, Warner D, and Gonzales J. Enhancement of claims data to improve risk adjustment of hospital mortality. JAMA. 2007; 297: 71–76.
This article has been cited by other articles:
 |
|
 |
|
  C P Gale, S O M Manda, B A Cattle, C F Weston, J S Birkhead, P D Batin, A S Hall, and R M West The authors' reply: Heart, June 15, 2009; 95(12): 1019 - 1020. [Full Text] [PDF]
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||