| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Circulation. 2001;104:263.)
© 2001 American Heart Association, Inc.
Clinical Investigation and Reports |
From the Division of Cardiology (M.M., E.K.-R., K.A.E.), University of Michigan, Blue Cross Blue Shield of Michigan Cardiovascular Consortium Coordinating Center, Ann Arbor; Blue Cross Blue Shield Center for Health Care Quality (D.S.), Detroit; St. Joseph Mercy Hospital (M.O.), Ann Arbor; Spectrum Health (A.M.-E.), Grand Rapids; Northern Michigan Hospital (W.L.M.), Petoskey; Henry Ford Hospital (P.K.), Detroit; McLaren Regional Medical Center (A.C.D, J.L.C.), Flint; St. Joseph Hospital (K.P.), Pontiac; and Harper Hospital-Wayne State University (J.G.M.), Detroit, Mich.
Correspondence to Mauro Moscucci, MD, University of Michigan Medical Center, Taubman 3910, 1500 E Medical Ctr Dr, Ann Arbor, MI 48109-0022. E-mail moscucci{at}umich.edu
| Abstract |
|---|
|
|
|---|
Methods and Results Between July 1, 1997 and September 30, 1999, 10 796 consecutive procedures were performed in a consortium of 8 hospitals. Predictors of in-hospital mortality were identified by use of multivariate logistic regression analysis. The final model was validated by use of the bootstrap technique. Additional validation was performed on an independent data set of 5863 consecutive procedures performed between October 1, 1999, and August 30, 2000. An additive risk-prediction score was developed by rounding coefficients of the logistic regression model to the closest half-integer, and a visual bedside tool for the prediction of individual patient prognosis was developed. In this patient population, the in-hospital mortality rate was 1.6%. Multivariate regression analysis identified acute myocardial infarction, cardiogenic shock, history of cardiac arrest, renal insufficiency, low ejection fraction, peripheral vascular disease, lesion characteristics, female sex, and advanced age as independent predictors of death. The model had excellent discrimination (area under the receiver operating characteristic curve, 0.90) and was accurate for prediction of mortality among different subgroups. Near-perfect correlation existed between calculated scores and observed mortality, with higher scores associated with higher mortality.
Conclusions Accurate predictions of individual patient risk of mortality associated with PCI can be achieved with a simple bedside tool. These predictions could be used during discussions of prognosis before and after PCI.
Key Words: angioplasty risk factors coronary disease
| Introduction |
|---|
|
|
|---|
| Methods |
|---|
|
|
|---|
Data Validity
After researchers had several meetings to discuss data elements and definitions, a dictionary with standard definitions was prepared and distributed to participating hospitals. Each hospital agreed to allocate a dedicated staff member to coordination and quality assurance of data collection. Data forms were processed by the coordinating center and individually evaluated for face validity and completeness. Incomplete forms were recoded by participating hospitals and resubmitted to the coordinating center. To ensure that consecutive patients were enrolled, site visits were performed by the coordinating center, whereas the coordinating center was visited by 1 of the other participating hospitals. During site visits, cardiac catheterization logs were compared with database logs. Medical records of any patient who died or who underwent CABG were reviewed and compared with the form submitted. In addition, 2% of the remaining cases were randomly selected for audit.
Missing Data
Baseline demographics (including age and sex), comorbidities, and procedure and outcome data were recorded in every case. Among the other data elements, baseline creatinine levels and ejection fraction percentages were missing in 8.5% and 25% of cases, respectively. Missing values for creatinine levels were coded as
1.5 mg/dL, whereas missing values for ejection fraction were entered by use of a linear regression model that included age, left ventricular end-diastolic pressure, cardiogenic shock, history of prior CABG, history of prior myocardial infarction, sex, and history of congestive heart failure.9,10
Statistical Analysis
Data are expressed as mean±SD or as percentages. Statistical analysis was performed with Stata statistical analysis software (Stata Corp). Univariate predictors of in-hospital death were identified by use of logistic regression analysis. Independent predictors of death were determined with stepwise multivariate logistic regression analysis. The model was developed on the training set (10 729 procedures) and validated by use of the technique of "bootstrap" resampling.11 The technique allows nearly unbiased estimates of predictive accuracy and is more efficient than data splitting or cross-validation.12 A total of 100 samples of 80% of the initial data set were drawn at random with replacement. Model discrimination was assessed by use of area under the receiver operating characteristic (ROC) curve.13 Area under the ROC curve was calculated for each bootstrap sample and for different subgroups. For each of these determinations, average area and 95% bias-corrected confidence interval (CI) were calculated. The final model was tested for goodness of fit by use of the Hosmer-Lemeshow statistic.14 Model overfitting was assessed by the technique of shrinkage.12
External Validation
External validation was performed on an independent data set of 5863 consecutive procedures performed between October 1, 1999, and August 30, 2000. Predicted probabilities of in-hospital death for individual patients and for the whole population were calculated with the model developed on the training set. Model discrimination was assessed by ROC curve analysis, and goodness of fit was tested with the Hosmer-Lemeshow statistic. ROC curves were developed by use of the program ROCKIT 0.9B (IBM PC version 0.9 Beta, 1998, C.E. Metz, Department of Radiology, University of Chicago, Chicago, Ill). Additional validation was performed with the bootstrap technique.
Development of Visual Additive Scoring System Tool
An additive scoring system was developed by use of coefficients of the regression model. For simplicity of use, scores were approximated by rounding coefficients to the nearest half-integer. A bedside card was developed by plotting scores, corresponding predicted mortality, and observed mortality. The scoring system and bedside card were validated on the validation set by ROC curve analysis and by construction of calibration plots.
| Results |
|---|
|
|
|---|
|
In this population, overall in-hospital mortality rate was 1.6%. Highest mortality rates were observed in patients who presented with acute myocardial infarction (7%), cardiogenic shock (32.9%), or baseline renal insufficiency (8%). A more-detailed analysis of renal insufficiency revealed a mortality rate of 1.04% for patients with normal creatinine levels, of 7.6% for patients with creatinine levels >1.5 and <2 mg/dL, and of 9.7% for patients with creatinine levels
2 mg/dL.
Predictive Model for In-Hospital Mortality
Multivariate analysis identified age, female sex, cardiogenic shock, acute myocardial infarction, number of diseased vessels, visible thrombus, creatinine >1.5 mg/dL, history of cardiac arrest, peripheral vascular disease, and decreased ejection fraction as independent predictors of in-hospital mortality (Table 2). The average area under the ROC curve was 0.90 (95% bias-corrected CI, 0.87 to 0.93), which indicates excellent model discrimination, and the Hosmer-Lemeshow statistic was not significant, which indicates little departure from a perfect fit (
2=6.8; P=0.5; Figure 1). The model still performed well when applied to different subsets, with areas under the ROC curve between 0.77 and 0.90 (Table 3).
|
|
|
Development of Scoring Tool
Final risk variables and corresponding scores are listed in the Appendix. Highest scores were assigned to acute myocardial infarction (score 1), cardiogenic shock (score 2.5), renal insufficiency (score 1.5), history of cardiac arrest (score 1.5), and advanced age (score 1.0). Figure 2 shows near-perfect correlation between total scores and observed mortality rates. Correlation also was excellent when the scoring system was applied to different risk subcategories (Table 3). In addition, in the low-risk group (score <1.5), only 2 deaths occurred per 3746 procedures (0.05%), whereas in the high-risk group (score >5), 86 deaths occurred in 251 procedures (34.3%). The final bedside card, definitions, scores, and instructions for scoring tool use are included in the Appendix.
|
External Validation of Logistic Regression Model and of Simplified Scoring Model
In the validation data set, the area under the ROC curve was 0.92 (95% bias-corrected CI, 0.88 to 0.94) for the logistic regression model and 0.91 (95% bias-corrected CI, 0.88 to 0.94) for the simplified scoring model, consistent with excellent model discrimination (Figure 3). The Hosmer-Lemeshow statistic was not significant (
2=10.7; P=0.22; 8 df; logistic regression model), which indicates little departure from a perfect fit.
|
Figure 4 shows application of the risk-prediction tool to the validation data set. Once again, near-perfect correlation was seen between total scores and observed mortality rates. In the low-risk group (score <1.5), only 1 death occurred per 1820 procedures (0.05%), whereas in the high-risk group (score >5), 49 deaths occurred per 136 procedures (36.03%).
|
| Discussion |
|---|
|
|
|---|
Predictors of In-Hospital Mortality
In this analysis, we identified independent predictors of in-hospital mortality after PCI and developed a bedside risk-prediction tool. The mortality rate observed in this population is relatively high compared with mortality rates previously reported in angioplasty registries,1618 which reflects the high percentage of patients with cardiogenic shock and acute myocardial infarction.
In agreement with findings from other studies, higher mortality rates were observed in patients who were elderly, who presented with acute myocardial infarction and cardiogenic shock, who were women, who had multivessel disease, and who had renal insufficiency.15 In particular, variables identified as independent predictors of in-hospital mortality are similar to variables identified by previously validated models, which supports overall consistency in variable selection across different populations. In addition, these variables are identical to variables included in a list recently proposed as a tool for efficient data collection to risk-adjust outcomes of coronary interventions.19
Preprocedural Morbidity and PCI Mortality
We underscore that the high mortality rate observed in the high-risk subgroups, including acute myocardial infarction and cardiogenic shock, is probably a reflection of the natural history of the disease rather than a consequence of the procedure itself. The issue of preprocedural morbidity is important when trying to differentiate between procedure- and disease-related deaths. Death after PCI is now a rare event, and most such deaths are related to preprocedural morbidity rather than to the procedure itself. Few deaths will be related to interaction between preprocedural morbidity and the procedure, and even less of them will be secondary to the procedure alone. In support of this statement, Malenka et al,20 in an analysis of 121 deaths after 12 232 percutaneous coronary revascularization procedures, showed that
50% of those deaths were unrelated to the procedure and represented the natural history of the disease. Thus, the value of risk adjustment resides in its ability to identify and "correct" for those comorbidities that affect the relationship between observed outcomes and the procedure itself.
Implications of This Analysis
The important finding of the present study is that despite the rare occurrence of death after PCI, gross estimates of risk of death still are possible in individual cases with the use of simple bedside tools. The high discriminatory power of the model and the high value of the area under the ROC curve suggest that risk of death after PCI is relatively predictable. In addition, accuracy of the scoring system in adjusting for comorbidities in high-risk subgroups adds supportive mathematical evidence to the initial qualitative impression that some patients are at higher risk of death. We underscore that many of these high-risk patients, including patients with cardiogenic shock or acute myocardial infarction, might be those who benefit the most from percutaneous revascularization. Thus, this tool should not be used to deny care to high-risk patients, but to help patients to make more informed decisions, to reassure low-risk patients that their low-risk estimate is based on robust mathematical models, and to allow family members to have realistic expectations when successful revascularization is achieved in association with major comorbidities likely to result, nonetheless, in an adverse outcome.
Comparison With Other Scoring Systems
Similar examples of scoring systems in cardiovascular medicine include prediction of fatal and nonfatal complications after coronary artery bypass surgery (Northern New England Cardiovascular Study Group),21 and the prediction of fatal and nonfatal cardiac complications after noncardiac surgery.22 Variables such as advanced age, peripheral vascular disease, and renal insufficiency consistently have been reported to be independent predictors of death, which suggests that these variables identify a subset of patients at high risk of death after any major surgery.68,21,22 This observation could be used in clinical decision making to optimize therapy and to decrease risks.
Limitations
One perceived limitation of the present analysis is that despite our large sample size, relatively few patients had scores >5. Although excellent correlation existed between observed and predicted mortality rates and the model validated well on the independent data set, additional validation in the high-risk subgroup with larger samples might be needed. Second, the independent validation data set was obtained from the same consortium from which the initial model was developed. Although our consortium includes a diverse group of institutions and this diversity should lead to stronger external validity, predictive accuracy is expected to decrease with external validation in other multicenter registries. Therefore, we should be cautious when generalizing these findings until this presumption has been tested.
A third important limitation is that we did not analyze the potential relationship between hospital or operator procedure volume and in-hospital mortality. Although the relationship between low institution or operator procedure volume and outcomes such as procedure success and unplanned CABG is still present, the relationship between procedure volume and PCI mortality now appears to be less significant for operator volume but still important for institution volume.23 Given that all hospitals participating in our consortium are high-volume hospitals, until further validation has been performed, our findings might not be applicable to low-volume hospitals. In addition, the potential effect of low operator volume remains to be determined. Fourth and finally, a potential concern is that the high discriminatory power of the model might be due to a tendency to report or overreport comorbidities in high-risk patients and to underreport comorbidities in low-risk patients. However, none of these patterns were observed during site audits or cross-sectional analysis of the database.
Conclusions
Results of the present study support the observation that in stable and low-risk patients, risk of death after PCI is low. However, presence of specific comorbidities identifies patients who are at much higher risk. Accurate predictions of individual patient risk of mortality can be achieved by use of a simple bedside tool that takes into account these comorbidities. These predictions can be helpful for enhancement of informed decision making and accurate expectations during discussions with patients and families of prognosis before and after PCI.
| Appendix |
|---|
|
|
|---|
30 min or pump failure after correction of contributing extra myocardial factors (hypovolemia, arrhythmias, pain, and vasovagal reactions) as manifested by either cardiac index <2.2 and PCWP >18 mm Hg or signs of hypoperfusion (peripheral vasoconstriction, urine output <30 cm3/h, or altered sensorium). Indicate whether patient experienced cardiogenic shock within 24 h of cardiac catheterization. For a history of cardiac arrest, determine whether cardiac arrest was the primary indication for current coronary intervention. No. of diseased vessels refers to those with
70% stenosis and includes left anterior descending coronary artery, right coronary artery, left circumflex artery, and grafts but does not include branches off native vessels. For example, if a patient has a >70% lesion in the obtuse marginal artery and proximal left circumflex artery, mark as 1 diseased vessel. EF <50% indicates that current ejection fraction is <50%; thrombus, filling defect suggestive of thrombus is in coronary artery segment treated. PVD or peripheral vascular disease includes that from claudication; amputation; vascular reconstruction, peripheral bypass surgery, or angioplasty; aortic aneurysm; and history of stroke, transient ischemic attack and carotid stenosis. See Table 4 for individual risk scores.
|
To estimate risk, calculate total score by adding individual scores if comorbidity is present. For No. of diseased vessels, add 0.5 for each major epicardial vessel with >70% stenosis. Identify total score on horizontal axis of the plot and corresponding probability on vertical axis (Figure 5). Scores
2.5 are associated with risk of death <0.8%, whereas scores >7 are associated with risk of death >40%.
|
| Acknowledgments |
|---|
Received February 13, 2001; revision received April 18, 2001; accepted April 28, 2001.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
B Kunadian, J Dunning, R Das, A P Roberts, R Morley, A J Turley, D Twomey, J A Hall, R A Wright, A G C Sutton, et al. External validation of established risk adjustment models for procedural complications after percutaneous coronary intervention Heart, August 1, 2008; 94(8): 1012 - 1018. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. S. Gurm, D. E. Smith, J. S. Collins, D. Share, A. Riba, A. J. Carter, T. LaLonde, E. Kline-Rogers, M. O'Donnell, H. Changezi, et al. The relative safety and efficacy of abciximab and eptifibatide in patients undergoing primary percutaneous coronary intervention insights from a large regional registry of contemporary percutaneous coronary intervention. J. Am. Coll. Cardiol., February 5, 2008; 51(5): 529 - 535. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Singh, B. J. Gersh, S. Li, J. S. Rumsfeld, J. A. Spertus, S. M. O'Brien, R. M. Suri, and E. D. Peterson Mayo Clinic Risk Score for Percutaneous Coronary Intervention Predicts In-Hospital Mortality in Patients Undergoing Coronary Artery Bypass Graft Surgery Circulation, January 22, 2008; 117(3): 356 - 362. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. B. King III, T. Aversano, W. L. Ballard, R. H. Beekman III, M. J. Cowley, S. G. Ellis, D. P. Faxon, E. L. Hannan, J. W. Hirshfeld Jr, A. K. Jacobs, et al. ACCF/AHA/SCAI 2007 Update of the Clinical Competence Statement on Cardiac Interventional Procedures: A Report of the American College of Cardiology Foundation/American Heart Association/American College of Physicians Task Force on Clinical Competence and Training (Writing Committee to Update the 1998 Clinical Competence Statement on Recommendations for the Assessment and Maintenance of Proficiency in Coronary Interventional Procedures) J. Am. Coll. Cardiol., July 3, 2007; 50(1): 82 - 108. [Full Text] [PDF] |
||||
![]() |
A Siotia and J Gunn Risk scoring for percutaneous coronary intervention: let's do it! Heart, November 1, 2006; 92(11): 1539 - 1540. [Full Text] [PDF] |
||||
![]() |
A D Grayson, R K Moore, M Jackson, S Rathore, S Sastry, T P Gray, I Schofield, A Chauhan, F F Ordoubadi, B Prendergast, et al. Multivariate prediction of major adverse cardiac events after 9914 percutaneous coronary interventions in the north west of England Heart, May 1, 2006; 92(5): 658 - 663. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. E. Matheny, L. Ohno-Machado, and F. S. Resnic Monitoring Device Safety in Interventional Cardiology J. Am. Med. Inform. Assoc., March 1, 2006; 13(2): 180 - 187. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Moscucci, E. K. Rogers, C. Montoye, D. E. Smith, D. Share, M. O'Donnell, A. Maxwell-Eward, W. L. Meengs, A. C. De Franco, K. Patel, et al. Association of a Continuous Quality Improvement Initiative With Practice and Outcome Variations of Contemporary Percutaneous Coronary Interventions Circulation, February 14, 2006; 113(6): 814 - 822. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. Wu, E. L. Hannan, G. Walford, J. A. Ambrose, D. R. Holmes Jr, S. B. King III, L. T. Clark, S. Katz, S. Sharma, and R. H. Jones A Risk Score to Predict In-Hospital Mortality for Percutaneous Coronary Interventions J. Am. Coll. Cardiol., February 7, 2006; 47(3): 654 - 660. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. R. Holmes Jr, P. Hodgson, and M. Singh Guidelines, Lighthouses, and a Toe in the Water Circulation, November 1, 2005; 112(18): 2754 - 2755. [Full Text] [PDF] |
||||
![]() |
M. Moscucci, D. Share, D. Smith, M. J. O'Donnell, A. Riba, R. McNamara, T. Lalonde, A. C. Defranco, K. Patel, E. Kline Rogers, et al. Relationship Between Operator Volume and Adverse Outcome in Contemporary Percutaneous Coronary Intervention Practice: An Analysis of a Quality-Controlled Multicenter Percutaneous Coronary Intervention Clinical Database J. Am. Coll. Cardiol., August 16, 2005; 46(4): 625 - 632. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. A. Brinker, C. J. Davidson, and W. Laskey Preventing in-hospital cardiac and renal complications in high-risk PCI patients Eur. Heart J. Suppl., August 1, 2005; 7(suppl_G): G13 - G24. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Moscucci, K. A. Eagle, D. Share, D. Smith, A. C. De Franco, M. O'Donnell, E. Kline-Rogers, S. M. Jani, and D. L. Brown Public Reporting and Case Selection for Percutaneous Coronary Interventions: An Analysis From Two Large Multicenter Percutaneous Coronary Intervention Databases J. Am. Coll. Cardiol., June 7, 2005; 45(11): 1759 - 1765. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Vanholder, Z. Massy, A. Argiles, G. Spasovski, F. Verbeke, N. Lameire, and for the European Uremic Toxin Work Group (EUTox) Chronic kidney disease as cause of cardiovascular morbidity and mortality Nephrol. Dial. Transplant., June 1, 2005; 20(6): 1048 - 1056. [Abstract] [Full Text] [PDF] |
||||
![]() |
F. S. Resnic, K. H. Zou, D. V. Do, G. Apostolakis, and L. Ohno-Machado Exploration of a Bayesian Updating Methodology to Monitor the Safety of Interventional Cardiovascular Procedures Med Decis Making, August 1, 2004; 24(4): 399 - 407. [Abstract] [PDF] |
||||
![]() |
M. Singh, C. S. Rihal, R. J. Lennon, K. N. Garratt, and D. R. Holmes Jr Comparison of Mayo Clinic risk score and American College of Cardiology/American Heart Association lesion classification in the prediction of adverse cardiovascular outcome following percutaneous coronary interventions J. Am. Coll. Cardiol., July 21, 2004; 44(2): 357 - 361. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. S. McKechnie, D. Smith, C. Montoye, E. Kline-Rogers, M. J. O'Donnell, A. C. DeFranco, W. L. Meengs, R. McNamara, J. G. McGinnity, K. Patel, et al. Prognostic Implication of Anemia on In-Hospital Outcomes After Percutaneous Coronary Intervention Circulation, July 20, 2004; 110(3): 271 - 277. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. A. Eagle, M. J. Lim, O. H. Dabbous, K. S. Pieper, R. J. Goldberg, F. Van de Werf, S. G. Goodman, C. B. Granger, P. G. Steg, J. M. Gore, et al. A Validated Prediction Model for All Forms of Acute Coronary Syndrome: Estimating the Risk of 6-Month Postdischarge Death in an International Registry JAMA, June 9, 2004; 291(22): 2727 - 2733. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. A. Qureshi, R. D. Safian, C. L. Grines, J. A. Goldstein, D. C. Westveer, S. Glazier, M. Balasubramanian, and W. W. O'Neill Simplified scoring system for predicting mortality after percutaneous coronary intervention J. Am. Coll. Cardiol., December 3, 2003; 42(11): 1890 - 1895. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. E. Cutlip, K. K. L. Ho, R. E. Kuntz, and D. S. Baim Risk assessment for percutaneous coronary intervention: our version of the weather report? J. Am. Coll. Cardiol., December 3, 2003; 42(11): 1896 - 1899. [Full Text] [PDF] |
||||
![]() |
M. Singh, C. S. Rihal, F. Selzer, K. E. Kip, K. Detre, and D. R. Holmes Jr Validation of Mayo clinic risk adjustment model for in-hospital complications after percutaneous coronary interventions, using the National Heart, Lung, and Blood Institute dynamic registry J. Am. Coll. Cardiol., November 19, 2003; 42(10): 1722 - 1728. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. J. de Feyter and E. McFadden Risk score for percutaneous coronary intervention: forewarned is forearmed J. Am. Coll. Cardiol., November 19, 2003; 42(10): 1729 - 1730. [Full Text] [PDF] |
||||
![]() |
M. Moscucci, K.A.A. Fox, C. P. Cannon, W. Klein, J. Lopez-Sendon, G. Montalescot, K. White, R.J. Goldberg, and for the GRACE Investigators Predictors of major bleeding in acute coronary syndromes: the Global Registry of Acute Coronary Events (GRACE) Eur. Heart J., October 2, 2003; 24(20): 1815 - 1823. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. R. Holmes, F. Selzer, J. M. Johnston, S. F. Kelsey, R. Holubkov, H. A. Cohen, D. O. Williams, and K. M. Detre Modeling and Risk Prediction in the Current Era of Interventional Cardiology: A Report From the National Heart, Lung, and Blood Institute Dynamic Registry Circulation, April 15, 2003; 107(14): 1871 - 1876. [Abstract] [Full Text] [PDF] |
||||
![]() |
Predicting Post-PCI In-Hospital Mortality Journal Watch Cardiology, September 14, 2001; 2001(914): 6 - 6. [Full Text] |
||||
| ||||||