Simple Bedside Additive Tool for Prediction of In-Hospital Mortality After Percutaneous Coronary Interventions
Background Risk-adjustment models for percutaneous coronary intervention (PCI) mortality have been recently reported, but application in bedside prediction of prognosis for individual patients remains untested.
Methods and Results Between July 1, 1997 and September 30, 1999, 10 796 consecutive procedures were performed in a consortium of 8 hospitals. Predictors of in-hospital mortality were identified by use of multivariate logistic regression analysis. The final model was validated by use of the bootstrap technique. Additional validation was performed on an independent data set of 5863 consecutive procedures performed between October 1, 1999, and August 30, 2000. An additive risk-prediction score was developed by rounding coefficients of the logistic regression model to the closest half-integer, and a visual bedside tool for the prediction of individual patient prognosis was developed. In this patient population, the in-hospital mortality rate was 1.6%. Multivariate regression analysis identified acute myocardial infarction, cardiogenic shock, history of cardiac arrest, renal insufficiency, low ejection fraction, peripheral vascular disease, lesion characteristics, female sex, and advanced age as independent predictors of death. The model had excellent discrimination (area under the receiver operating characteristic curve, 0.90) and was accurate for prediction of mortality among different subgroups. Near-perfect correlation existed between calculated scores and observed mortality, with higher scores associated with higher mortality.
Conclusions Accurate predictions of individual patient risk of mortality associated with PCI can be achieved with a simple bedside tool. These predictions could be used during discussions of prognosis before and after PCI.
Received February 13, 2001; revision received April 18, 2001; accepted April 28, 2001.
Prediction of individual patient outcomes can be of particular value during discussions with patients and families of prognosis before and after percutaneous coronary interventions (PCI), and in clinical decision making. Recently, risk-adjustment models for PCI mortality were developed and used to compare specific, aggregate outcomes of operators.1–7 However, application of these models to prediction of individual patient outcomes remains untested. Although these models could be used to calculate individual probabilities of death, bedside application requires solving an exponential equation and, thus, is impractical. The objective of the present study was to develop a simple bedside visual tool to predict in-hospital mortality after PCI.
Study samples consisted of 10 729 consecutive PCI performed between July 1, 1997, and September 30, 1999 (training set), and 5863 consecutive PCI performed between October 1, 1999, and August 30, 2000 (validation set), in a consortium of 8 hospitals. The consortium included 3 academic centers, 4 tertiary referral centers, and 1 community hospital. Clinical, procedure, and outcome data were collected prospectively by use of a standardized data collection form agreed on by the participating centers. Coronary artery stenoses were classified according to the modified American College of Cardiology/American Heart Association classification.8 Additional lesion characteristics, including presence of visible thrombus or filling defect suggestive of thrombus, coronary calcification, and location of lesion treated at the ostium of the coronary artery segment, were collected. The registry is part of the quality assessment and quality improvement program of the hospitals participating in the consortium, and it was approved by the Institutional Review Board of the University of Michigan and by local institutional review boards.
After researchers had several meetings to discuss data elements and definitions, a dictionary with standard definitions was prepared and distributed to participating hospitals. Each hospital agreed to allocate a dedicated staff member to coordination and quality assurance of data collection. Data forms were processed by the coordinating center and individually evaluated for face validity and completeness. Incomplete forms were recoded by participating hospitals and resubmitted to the coordinating center. To ensure that consecutive patients were enrolled, site visits were performed by the coordinating center, whereas the coordinating center was visited by 1 of the other participating hospitals. During site visits, cardiac catheterization logs were compared with database logs. Medical records of any patient who died or who underwent CABG were reviewed and compared with the form submitted. In addition, 2% of the remaining cases were randomly selected for audit.
Baseline demographics (including age and sex), comorbidities, and procedure and outcome data were recorded in every case. Among the other data elements, baseline creatinine levels and ejection fraction percentages were missing in 8.5% and 25% of cases, respectively. Missing values for creatinine levels were coded as ≤1.5 mg/dL, whereas missing values for ejection fraction were entered by use of a linear regression model that included age, left ventricular end-diastolic pressure, cardiogenic shock, history of prior CABG, history of prior myocardial infarction, sex, and history of congestive heart failure.9,10
Data are expressed as mean±SD or as percentages. Statistical analysis was performed with Stata statistical analysis software (Stata Corp). Univariate predictors of in-hospital death were identified by use of logistic regression analysis. Independent predictors of death were determined with stepwise multivariate logistic regression analysis. The model was developed on the training set (10 729 procedures) and validated by use of the technique of “bootstrap” resampling.11 The technique allows nearly unbiased estimates of predictive accuracy and is more efficient than data splitting or cross-validation.12 A total of 100 samples of 80% of the initial data set were drawn at random with replacement. Model discrimination was assessed by use of area under the receiver operating characteristic (ROC) curve.13 Area under the ROC curve was calculated for each bootstrap sample and for different subgroups. For each of these determinations, average area and 95% bias-corrected confidence interval (CI) were calculated. The final model was tested for goodness of fit by use of the Hosmer-Lemeshow statistic.14 Model overfitting was assessed by the technique of shrinkage.12
External validation was performed on an independent data set of 5863 consecutive procedures performed between October 1, 1999, and August 30, 2000. Predicted probabilities of in-hospital death for individual patients and for the whole population were calculated with the model developed on the training set. Model discrimination was assessed by ROC curve analysis, and goodness of fit was tested with the Hosmer-Lemeshow statistic. ROC curves were developed by use of the program ROCKIT 0.9B (IBM PC version 0.9 Beta, 1998, C.E. Metz, Department of Radiology, University of Chicago, Chicago, Ill). Additional validation was performed with the bootstrap technique.
Development of Visual Additive Scoring System Tool
An additive scoring system was developed by use of coefficients of the regression model. For simplicity of use, scores were approximated by rounding coefficients to the nearest half-integer. A bedside card was developed by plotting scores, corresponding predicted mortality, and observed mortality. The scoring system and bedside card were validated on the validation set by ROC curve analysis and by construction of calibration plots.
Table 1 lists baseline demographic and clinical characteristics and device use in training and validation sets. High percentages of patients presented within <24 hours of myocardial infarction (13.4%) and with cardiogenic shock (2.4%). Also, 17% of patients had history of prior CABG, 31% had history of prior PTCA, and 34% had history of prior myocardial infarction.
In this population, overall in-hospital mortality rate was 1.6%. Highest mortality rates were observed in patients who presented with acute myocardial infarction (7%), cardiogenic shock (32.9%), or baseline renal insufficiency (8%). A more-detailed analysis of renal insufficiency revealed a mortality rate of 1.04% for patients with normal creatinine levels, of 7.6% for patients with creatinine levels >1.5 and <2 mg/dL, and of 9.7% for patients with creatinine levels ≥2 mg/dL.
Predictive Model for In-Hospital Mortality
Multivariate analysis identified age, female sex, cardiogenic shock, acute myocardial infarction, number of diseased vessels, visible thrombus, creatinine >1.5 mg/dL, history of cardiac arrest, peripheral vascular disease, and decreased ejection fraction as independent predictors of in-hospital mortality (Table 2). The average area under the ROC curve was 0.90 (95% bias-corrected CI, 0.87 to 0.93), which indicates excellent model discrimination, and the Hosmer-Lemeshow statistic was not significant, which indicates little departure from a perfect fit (χ2=6.8; P=0.5; Figure 1). The model still performed well when applied to different subsets, with areas under the ROC curve between 0.77 and 0.90 (Table 3).
Development of Scoring Tool
Final risk variables and corresponding scores are listed in the Appendix. Highest scores were assigned to acute myocardial infarction (score 1), cardiogenic shock (score 2.5), renal insufficiency (score 1.5), history of cardiac arrest (score 1.5), and advanced age (score 1.0). Figure 2 shows near-perfect correlation between total scores and observed mortality rates. Correlation also was excellent when the scoring system was applied to different risk subcategories (Table 3). In addition, in the low-risk group (score <1.5), only 2 deaths occurred per 3746 procedures (0.05%), whereas in the high-risk group (score >5), 86 deaths occurred in 251 procedures (34.3%). The final bedside card, definitions, scores, and instructions for scoring tool use are included in the Appendix.
External Validation of Logistic Regression Model and of Simplified Scoring Model
In the validation data set, the area under the ROC curve was 0.92 (95% bias-corrected CI, 0.88 to 0.94) for the logistic regression model and 0.91 (95% bias-corrected CI, 0.88 to 0.94) for the simplified scoring model, consistent with excellent model discrimination (Figure 3). The Hosmer-Lemeshow statistic was not significant (χ2=10.7; P=0.22; 8 df; logistic regression model), which indicates little departure from a perfect fit.
Figure 4 shows application of the risk-prediction tool to the validation data set. Once again, near-perfect correlation was seen between total scores and observed mortality rates. In the low-risk group (score <1.5), only 1 death occurred per 1820 procedures (0.05%), whereas in the high-risk group (score >5), 49 deaths occurred per 136 procedures (36.03%).
As for every invasive or surgical procedure, the potential benefit of PCI must be weighed against its potential risks. Recent mechanical and pharmacological advancements have resulted in a substantial reduction in risk of death or of major complications for patients undergoing PCI.15 However, presence of certain comorbidities, such as acute myocardial infarction or cardiogenic shock, is still associated with the increased risk that is inherent to the natural history of the disease itself. Although risk of adverse outcome might vary substantially from patient to patient, physicians performing PCI usually have been able to provide only a general estimate of that risk, without mathematical precision based on proper weighting of key comorbidities. Thus, precise prediction of individual patient outcome has particular importance during discussions with patients and families of prognosis before and after PCI or for clinical decision making.
Predictors of In-Hospital Mortality
In this analysis, we identified independent predictors of in-hospital mortality after PCI and developed a bedside risk-prediction tool. The mortality rate observed in this population is relatively high compared with mortality rates previously reported in angioplasty registries,16–18 which reflects the high percentage of patients with cardiogenic shock and acute myocardial infarction.
In agreement with findings from other studies, higher mortality rates were observed in patients who were elderly, who presented with acute myocardial infarction and cardiogenic shock, who were women, who had multivessel disease, and who had renal insufficiency.1–5 In particular, variables identified as independent predictors of in-hospital mortality are similar to variables identified by previously validated models, which supports overall consistency in variable selection across different populations. In addition, these variables are identical to variables included in a list recently proposed as a tool for efficient data collection to risk-adjust outcomes of coronary interventions.19
Preprocedural Morbidity and PCI Mortality
We underscore that the high mortality rate observed in the high-risk subgroups, including acute myocardial infarction and cardiogenic shock, is probably a reflection of the natural history of the disease rather than a consequence of the procedure itself. The issue of preprocedural morbidity is important when trying to differentiate between procedure- and disease-related deaths. Death after PCI is now a rare event, and most such deaths are related to preprocedural morbidity rather than to the procedure itself. Few deaths will be related to interaction between preprocedural morbidity and the procedure, and even less of them will be secondary to the procedure alone. In support of this statement, Malenka et al,20 in an analysis of 121 deaths after 12 232 percutaneous coronary revascularization procedures, showed that ≥50% of those deaths were unrelated to the procedure and represented the natural history of the disease. Thus, the value of risk adjustment resides in its ability to identify and “correct” for those comorbidities that affect the relationship between observed outcomes and the procedure itself.
Implications of This Analysis
The important finding of the present study is that despite the rare occurrence of death after PCI, gross estimates of risk of death still are possible in individual cases with the use of simple bedside tools. The high discriminatory power of the model and the high value of the area under the ROC curve suggest that risk of death after PCI is relatively predictable. In addition, accuracy of the scoring system in adjusting for comorbidities in high-risk subgroups adds supportive mathematical evidence to the initial qualitative impression that some patients are at higher risk of death. We underscore that many of these high-risk patients, including patients with cardiogenic shock or acute myocardial infarction, might be those who benefit the most from percutaneous revascularization. Thus, this tool should not be used to deny care to high-risk patients, but to help patients to make more informed decisions, to reassure low-risk patients that their low-risk estimate is based on robust mathematical models, and to allow family members to have realistic expectations when successful revascularization is achieved in association with major comorbidities likely to result, nonetheless, in an adverse outcome.
Comparison With Other Scoring Systems
Similar examples of scoring systems in cardiovascular medicine include prediction of fatal and nonfatal complications after coronary artery bypass surgery (Northern New England Cardiovascular Study Group),21 and the prediction of fatal and nonfatal cardiac complications after noncardiac surgery.22 Variables such as advanced age, peripheral vascular disease, and renal insufficiency consistently have been reported to be independent predictors of death, which suggests that these variables identify a subset of patients at high risk of death after any major surgery.6–8,21,22 This observation could be used in clinical decision making to optimize therapy and to decrease risks.
One perceived limitation of the present analysis is that despite our large sample size, relatively few patients had scores >5. Although excellent correlation existed between observed and predicted mortality rates and the model validated well on the independent data set, additional validation in the high-risk subgroup with larger samples might be needed. Second, the independent validation data set was obtained from the same consortium from which the initial model was developed. Although our consortium includes a diverse group of institutions and this diversity should lead to stronger external validity, predictive accuracy is expected to decrease with external validation in other multicenter registries. Therefore, we should be cautious when generalizing these findings until this presumption has been tested.
A third important limitation is that we did not analyze the potential relationship between hospital or operator procedure volume and in-hospital mortality. Although the relationship between low institution or operator procedure volume and outcomes such as procedure success and unplanned CABG is still present, the relationship between procedure volume and PCI mortality now appears to be less significant for operator volume but still important for institution volume.23 Given that all hospitals participating in our consortium are high-volume hospitals, until further validation has been performed, our findings might not be applicable to low-volume hospitals. In addition, the potential effect of low operator volume remains to be determined. Fourth and finally, a potential concern is that the high discriminatory power of the model might be due to a tendency to report or overreport comorbidities in high-risk patients and to underreport comorbidities in low-risk patients. However, none of these patterns were observed during site audits or cross-sectional analysis of the database.
Results of the present study support the observation that in stable and low-risk patients, risk of death after PCI is low. However, presence of specific comorbidities identifies patients who are at much higher risk. Accurate predictions of individual patient risk of mortality can be achieved by use of a simple bedside tool that takes into account these comorbidities. These predictions can be helpful for enhancement of informed decision making and accurate expectations during discussions with patients and families of prognosis before and after PCI.
Risk-Prediction Tool for In-Hospital Mortality After PCI
Acute MI indicates acute myocardial infarction with balloon dilatation of the infarct vessel within 24 h of onset of symptoms. Shock is systolic pressure <90 mm Hg for ≥30 min or pump failure after correction of contributing extra myocardial factors (hypovolemia, arrhythmias, pain, and vasovagal reactions) as manifested by either cardiac index <2.2 and PCWP >18 mm Hg or signs of hypoperfusion (peripheral vasoconstriction, urine output <30 cm3/h, or altered sensorium). Indicate whether patient experienced cardiogenic shock within 24 h of cardiac catheterization. For a history of cardiac arrest, determine whether cardiac arrest was the primary indication for current coronary intervention. No. of diseased vessels refers to those with ≥70% stenosis and includes left anterior descending coronary artery, right coronary artery, left circumflex artery, and grafts but does not include branches off native vessels. For example, if a patient has a >70% lesion in the obtuse marginal artery and proximal left circumflex artery, mark as 1 diseased vessel. EF <50% indicates that current ejection fraction is <50%; thrombus, filling defect suggestive of thrombus is in coronary artery segment treated. PVD or peripheral vascular disease includes that from claudication; amputation; vascular reconstruction, peripheral bypass surgery, or angioplasty; aortic aneurysm; and history of stroke, transient ischemic attack and carotid stenosis. See Table 4 for individual risk scores.
To estimate risk, calculate total score by adding individual scores if comorbidity is present. For No. of diseased vessels, add 0.5 for each major epicardial vessel with >70% stenosis. Identify total score on horizontal axis of the plot and corresponding probability on vertical axis (Figure 5). Scores ≤2.5 are associated with risk of death <0.8%, whereas scores >7 are associated with risk of death >40%.
The present work was supported by a grant from the Blue Cross Blue Shield of Michigan Foundation, Detroit, Mich.
Hannan El, Arani DT, Johnson LW, et al. Percutaneous transluminal coronary angioplasty in New York State: risk factors and outcomes. JAMA. 1993; 268: 3092–3097.
Ellis SG, Omoigui N, Bittl JA, et al. Analysis and comparison of operator specific outcomes in interventional cardiology: from a multicenter database of 4860 quality-controlled procedures. Circulation. 1996; 93: 431–439.
Ellis SG, Vandormael MG, Cowley MJ, et al. Coronary morphologic and clinical determinants of procedural outcome with angioplasty for multivessel disease: implications for patient selection. Circulation. 1990; 82: 1193–1202.
Little RJA, Rubin DB. Statistical Analysis With Missing Data. New York, NY: John Wiley & Sons; 1987.
Greenland S, Finkle WD. A critical look at methods for handling missing covariates in epidemiologic regression analyses. Am J Epidemiol. 1995; 142: 1255–1264.
Efron B, Tibshirami RJ. An Introduction to the Bootstrap. New York, NY: Chapman & Hall; 1993.
Egan JP. Signal Detection Theory and ROC Analysis. New York, NY: Academic Press; 1975.
Hosmer DW, Lemeshow S. Applied Logistic Regression. New York, NY: John Wiley & Sons; 1989.
Holmes DR Jr, Holubkov R, Vliestra RE, et al. Comparison of complications during percutaneous transluminal coronary angioplasty from 1977 to 1981 and from 1985 to 1986; the National Heart, Lung, and Blood Institute Percutaneous Transluminal Coronary Angioplasty Registry. J Am Coll Cardiol. 1988; 12: 1149–1155.
Eagle KA, Guyton RA, Davidoff R0, et al. ACC/AHA guidelines for coronary artery bypass graft surgery: executive summary and recom-mendations: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee to Revise the 1991 Guidelines for Coronary Artery Bypass Graft Surgery). Circulation. 1999; 100: 1464–1480.
Lee TH, Macantonio ER, Mangione CM, et al. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation. 1999; 100: 1043–1049.