Development and Validation of the MCRS

The development of the new MCRS has been described previously (Figure).⁴ Briefly, 9 independent variables were selected and entered into a logistic regression model to determine their relationship with PCI mortality. Regression coefficients from the logistic regression model were then converted to integers to create a simple bedside risk score. The final risk score was validated with data collected between January 2000 and April 2005 from 7457 PCI patients. The MCRS demonstrated excellent ability to discriminate between high- and low-risk PCI patients (c-index=0.90) and good calibration. The probabilities estimated from the logistic model matched the observed data well, as indicated by a nonsignificant Hosmer-Lemeshow goodness-of-fit test (P=0.44).

• Download figure
• Open in new tab
• Download powerpoint

Figure. New Mayo Clinic risk model⁴ for prediction of in-hospital death. Coefficients for age, left ventricular ejection fraction (LV EF), and serum creatinine can be determined from the nomograms at the bottom. Note that congestive heart failure (CHF) needs to be entered only for patients not presenting with myocardial infarction (MI) or shock. The equation for log-odds of death is LogOdds=−6.082+0.2883×score. AMI indicates acute myocardial infarction. For missing values: If creatinine is unavailable, do the following: Add 1 point if patient is male, and 1 point if patient has CHF. If ejection fraction is unavailable, do the following: Add 1 point if patient has CHF. For other variables, if a risk factor is unknown, no points are added.

Study Population

The Society of Thoracic Surgeons (STS) National Cardiac Database includes the records of millions of patients who have undergone cardiac surgery at participating sites across the United States. For the present study, we only included patients who underwent isolated CABG surgery, and we excluded patients undergoing CABG after failed PCI (n=3601) or other concomitant procedures (eg, valve replacements [n=105 457]). The present analysis is thus confined to 370 793 patients undergoing CABG only from 2004 to 2006 who had their data collected under the most recent STS data collection form (version 2.52). Approval for the development and validation of the MCRS model was obtained from the Institutional Review Board of the Mayo Foundation.

Definitions

The outcome for the present analysis was the incidence of in-hospital mortality after CABG surgery. Age, ejection fraction, and serum creatinine were entered as numerical values and then mapped with closest integer in the model. The definitions of myocardial infarction, congestive heart failure, and peripheral vascular disease were similar in the 2 original data sets. Shock was defined more stringently (Appendix 1) in the STS database than in the Mayo Clinic database.

Statistical Analysis

Crude incidence rates of in-hospital mortality after CABG were compared across levels of demographic and clinical variables. Each patient was assigned a risk score by application of the MCRS model. The relationship between the MCRS and mortality risk was estimated by grouping together all patients who had the same value of the MCRS. Within each MCRS group, the observed in-hospital mortality rate was calculated and presented along with approximate 95% binomial confidence intervals (CIs). The discrimination of the MCRS as a predictor of CABG mortality was assessed by the C-statistic (also known as the area under the receiver operating characteristics curve). The C-statistic represents the probability that a randomly selected patient who died in the hospital had a higher predicted risk of mortality than a randomly selected patient who survived until discharge. The C-statistic generally ranges from 0.5 to 1.0, with 0.5 representing no discrimination (ie, a coin flip) and 1.0 representing perfect discrimination. This analysis was initially conducted in the overall study population and subsequently repeated in selected subgroups based on variables such as age and ejection fraction. To provide a context for interpreting the C-statistic, we also qualitatively compared the discrimination of the MCRS to the published STS CABG mortality model as applied to the present study population. The difference in the C-statistics for the MCRS versus STS models was tested with the method of DeLong et al.⁵ We anticipated that the C-statistic of the MCRS would be somewhat smaller because the STS model contains a larger number of covariates and was developed specifically for CABG patients.

Missing Data

The approach used to handle missing data was similar to that proposed in the original MCRS (Figure). Missing values were rare in the STS database for the variables included in the MCRS. The percentage of missing values was <1%, with the exception of ejection fraction, for which values were missing in 4.6% of cases.

The authors had full access to and take full responsibility for the integrity of the data. All authors have read and agree to the manuscript as written.

Previous Studies

All previous attempts in the development of the model were predicated on the selection of either surgical or percutaneous revascularization therapy. Excellent models predicting not only mortality but also other major adverse cardiovascular complications are available.^2,3,6–8 Several available CABG risk models are either derived from older data sets or derived from European data, such that they may have limited applicability to patients undergoing CABG in the United States.^9–13 More recently, integer-based risk indices have facilitated quick estimation of in-hospital mortality and other adverse cardiovascular events.¹ New York State models for both PCI and CABG reported surprisingly similar risk factors, such that 7 of the 10 variables in the CABG risk score were listed in both scores.^1,2 Six of the 7 listed MCRS risk variables are similar to those in the New York State model, a fact that underscores the commonality of risk variables in prediction of outcome irrespective of the revascularization strategy. Besides demonstrating the utility of predicting outcome in patients undergoing PCI and CABG, the MCRS has several advantages over the current New York State CABG model. First, the MCRS does not use subjective covariates, thus minimizing inconsistencies in data collection and collinearity. For example, the new MCRS model does not refer to “urgent” interventions or an “extensively calcified” ascending aorta. Second, age, ejection fraction, and creatinine are entered as continuous variables, thus helping providers calculate risk gradients with minimal change in the value of these variables. For example, renal failure that requires dialysis is a risk variable in the New York State model; however, lesser elevation of creatinine is known to be associated with adverse outcome.¹⁴ The only variable from the MCRS that was not consistently included in the STS CABG mortality model was presentation with acute myocardial infarction, which likely represents the performance of urgent or emergent PCI in such situations.

Previous attempts at any risk model development have been geared either toward surgical or percutaneous coronary revascularization, which hindered objective outcome assessment by patients or healthcare providers for patients who were potentially suitable for either revascularization therapy. The present analyses create a window of opportunity to use a single risk score to assess the risk of coronary revascularization therapies based on easily obtainable demographic and laboratory parameters. There appears to be a high degree of overlap between risk factors that are predictive of outcome regardless of mode of coronary revascularization. For example, increasing age, renal disease, measures of left ventricular dysfunction (ejection fraction, congestive heart failure), cardiogenic shock, and peripheral vascular disease are a subset of risk factors that are predictive across several models developed for outcomes after CABG or PCI.¹⁵ This core set of variables is included in the current MCRS. Most of the available models for CABG include similar variables and demonstrate discriminatory ability similar to that of the MCRS.

We acknowledge that the existing STS mortality model, which was developed specifically for CABG, has better discrimination than the MCRS, and one could improve on the existing MCRS by choosing new weights based on the CABG data. The practical advantage of using a single, parsimonious model for both populations (PCI and CABG) is attractive; however, further studies are needed to demonstrate the applicability of such models in guiding the choice of revascularization strategy. We have demonstrated that the MCRS can be used by healthcare providers for bedside risk assessment during the first contact with patients. It conveys risks of in-hospital mortality from either CABG or PCI and has potential value to assist patients in selecting a mode of coronary revascularization. It remains to be seen whether assignment of risk by the model changes the selection of therapy and whether it impacts the prognosis of the patient. We realize, however, that personal or physician preferences and angiographic characteristics will likely determine the ultimate decision. We also want to underscore that the model is unlikely to convey the superiority of a specific revascularization therapy and that additional factors, including patients’ preferences, are required for decision making. Yet, given the opportunity to exaggerate or underestimate risks from procedures with which a given provider is less familiar, we thought that the MCRS could be a valuable tool in “fairly” benchmarking patients’ risks.

Study Limitations

The MCRS was developed to predict mortality after PCI and may weight risk factors differently than a risk score that was developed specifically for CABG patients.¹⁶ Yet, the finding that the MCRS has modest discrimination for predicting mortality after CABG supports the generalizability of the original MCRS across procedures. Second, we did not address the additional prognostic value of several variables known to be associated with adverse outcomes after CABG, namely, left main or multivessel disease. We deliberately omitted these variables in an attempt to retain the simplicity of the model and to keep the perspective of using the preprocedure risk variables (ie, before coronary angiography, because for many patients, the decision about subsequent revascularization is made in the cardiac catheterization laboratory immediately after coronary angiography). We also excluded patients undergoing valve surgery or emergent CABG as a consequence of failed PCI, limiting its role only to patients undergoing isolated CABG; however, given that the need for concomitant valve surgery minimizes the utility of concurrent evaluation of the risks of PCI, we do not believe that this is a significant limitation. Because of differences in definitions and variations in reporting of periprocedural myocardial infarction among patients undergoing CABG or PCI, this outcome measure and other measures that increase morbidity or length of hospital stay were not included in the final analyses. Although this report was accurate in predicting in-hospital mortality, no predictive model can overcome the effect of chance and uncertainty, which are inherent in invasive treatments.

Shock Definition

In the STS database: Indicate whether the patient was, at the time of procedure, in a clinical state of hypoperfusion according to either of the following criteria: (1) systolic blood pressure <80 mm Hg and/or cardiac index <1.8 L · min⁻¹ · m⁻² despite maximal treatment; (2) intravenous inotropes and/or intra-aortic balloon pump necessary to maintain systolic blood pressure >80 mm Hg and/or cardiac index >1.8 L · min⁻¹ · m⁻².

In the Mayo Clinic database: Prolonged hypotension, with blood pressure <95 mm Hg without inotropes or intra-aortic balloon pump support or <110 mm Hg with inotropes or intra-aortic balloon pump support, which required treatment with inotropes or intra-aortic balloon pump.