Reply to Letters Regarding Article, “Risk of Assessing Mortality Risk in Elective Cardiac Operations: Age, Creatinine, Ejection Fraction, and the Law of Parsimony”
We thank Dr Lown et al and Dr Miceli et al for their comments about our article.1 They raise several different and interesting points.
It is of course possible that the selection of a limited number of variables for the risk model may better benefit from other more sophisticated statistical approaches. However, even if our model is apparently simplistic, it seems to work better than other complex models. We agree that an area under curve value of 0.744 is probably inadequate for clinical purposes, but this is the value reached by the widely used EuroSCORE in many clinical settings.2 Actually, in the validation series, the area under curve value of the ACEF score (age, creatinine, ejection fraction) was higher than 0.8.2 As stated in our article, it is a matter of terminology. We could say that the ACEF and the EuroSCORE are equally good or equally bad models. To answer the authors’ question, we did sensitivity analyses in a subgroup of patients at low, medium, and high risk, and the worst accuracy was achieved in the high-risk patients (EuroSCORE >5). However, it should be considered that the ACEF is built for elective patients. When addressing the whole range of cardiac surgery population, more variables are needed to improve the accuracy of the model, up to 5 for medium-risk patients and even 12 in high-risk patients.2
Dichotomization of continuous variables is always arbitrary in statistics but is a common practice in the development of risk scores. The important point is to dichotomise according to a method that may guarantee the best specificity and sensitivity of the identified cutoff value. We did dichotomise the serum creatinine value (as in the great majority of the existing scores) using the sensitivity and specificity values of the receiver operating characteristics curve coordinates and the Youden index. We agree that every kind of preoperative renal function impairment may strongly affect the operative mortality, and we cannot exclude that using serum creatinine as a continuous variable (as we did for age and ejection fraction) may improve the accuracy of the model, however at the expenses of more complex calculations.
We agree with Lown et al about the limitations of area under curve–based systems and the risk of overfitted models, availability of data, and subjectivity, and these last reasons led us to propose a parsimonious model. Conversely, we are not sure that simple models may be more adequate for acute settings and complex models for elective settings. Actually, our experience is that when acute risk conditions intervene, more factors are needed to stratify the risk.2 It was demonstrated that when severe acute conditions are present, the current scores are inadequate to stratify mortality risk.3
When developing this model, we considered many potential factors that could be independently associated with operative mortality after cardiac surgery. Among these factors are those mentioned by Miceli et al (gender, chronic obstructive pulmonary disease, operations other than isolated coronary artery bypass graft, and redo operations). We think these factors should be considered independent predictors of operative mortality. However, as demonstrated in another recently published article,2 adding these factors to the ACEF does not improve the models’ accuracy.
The main point, from our perspective, is that risk scores should be used to address a surgical population risk, check the internal quality of an institution, and follow the quality of the results year by year. If used to establish the mortality risk of a single patient, all the existing risk scores suffer from a severe lack of sensitivity (never exceeding 10%), and the ACEF is not an exception to this rule. When analyzing a single patient’s mortality risk, other specific conditions not usually considered in any risk score should be taken into account. Chronic obstructive pulmonary disease, for example, should be considered according to its severity, and it does not make any sense to attribute a single risk point to this condition regardless of the fact that the patient is simply under steroid therapy or is dependent on oxygen administration at home. Other extreme conditions (eg, morbid obesity, severe liver failure, chronic dialytic treatment) play important roles but are not included in many existing risk scores.
In conclusion, we greatly appreciate this ongoing debate on risk stratification in cardiac surgery. New challenges are coming, and a good example is patient selection for transcatheter aortic valve implantation. We must be ready to rediscuss all the existing information and open our minds to new and probably more adequate approaches.
Ranucci M, Castelvecchio S, Menicanti L, Frigiola A, Pelissero G. Risk of assessing mortality risk in elective cardiac operations: age, creatinine, ejection fraction, and the law of parsimony. Circulation. 2009; 119: 3053–3061.
Ranucci M, Castelvecchio S, Menicanti L, Frigiola A, Pelissero G. Accuracy, calibration and clinical performance of the EuroSCORE: can we reduce the number of variables? Eur J Cardiothorac Surg. 2009, Epub ahead of print October 9, 2009, doi: 10.1016/j.ejcts.2009.08.033.