Response to Letter Regarding Article, “Which Hospitals Have Significantly Better or Worse Than Expected Mortality Rates for Acute Myocardial Infarction Patients? Improved Risk Adjustment With Present-at-Admission Diagnoses”
We thank Drs Gale, Manda, and Birkhead for their comments and questions regarding our study.1 The variables in our model were derived from California hospital data similar to most US hospital administrative data, except for one difference: California hospitals report which secondary diagnoses were present at admission. The California data included up to 20 ICD-9-CM diagnosis codes for each patient. We grouped the codes into 233 mutually exclusive categories of comorbid disease and included these as dichotomous variables in the model.2
Parsimonious models can be advantageous when original data collection is required and resources are limited. However, our study used data routinely collected on discharge from US hospitals. Our aims were to use available hospital data distinguishing which diagnoses are present at admission to develop a valid model that maximally explained and accurately predicted death. We found that substantial additional predictive value is obtained by including adjustments for the large number of conditions that are present at admission, including conditions with small effects on mortality and conditions that occur infrequently. The model that we achieved improved statistical performance and demonstrated external predictive validity in an independent California hospital discharge data set. Most importantly, we found that the increase in statistical performance matters, because models with better statistical performance produce substantially different results about which hospitals have better or worse than expected death rates.
The figures do not directly address agreement. Rather, they illustrate the dispersion and correlation between fixed effects model standardized mortality rates, odds ratios, and random effects model standard mortality rates. To address model agreement, we presented tables that compare hospitals identified as significantly better (or worse) than expected and calculated the proportion of observed to expected agreement using the κ statistic.3 The models we compared demonstrated moderate agreement overall about which hospitals were better (or worse) than expected for each of the 3 indices.
Douglas Paul Wagner, Professor of Biostatistics and Epidemiology in the Department of Public Health Sciences at the University of Virginia, passed away on Sunday, March 23, 2008. Professor Wagner is remembered by his students and colleagues as a man of great intellect, humility, compassion, and unfailing generosity.
Stukenborg GJ, Wagner DP, Harrell FE Jr, Oliver MN, Heim SW, Price AL, Han CK, Wolf AMD, Connors AF Jr. Which hospitals have significantly better or worse than expected mortality rates for acute myocardial infarction patients? Improved risk adjustment with present-at-admission diagnoses. Circulation. 2007; 116: 2960–2968.
Fleiss J. Statistical Methods for Rates and Proportions. Second ed. New York, NY: Wiley; 1981.