Cardiovascular Surgery

Prediction Models for Prolonged Intensive Care Unit Stay After Cardiac Surgery

Systematic Review and Validation Study

Roelof G.A. Ettema, Linda M. Peelen, Marieke J. Schuurmans, Arno P. Nierich, Cor J. Kalkman, Karel G.M. Moons
https://doi.org/10.1161/CIRCULATIONAHA.109.926808
Circulation. 2010;122:682-689
Originally published August 16, 2010

Jump to

Abstract

Background— Several models have been developed to predict prolonged stay in the intensive care unit (ICU) after cardiac surgery. However, no extensive quantitative validation of these models has yet been conducted. This study sought to identify and validate existing prediction models for prolonged ICU length of stay after cardiac surgery.

Methods and Results— After a systematic review of the literature, the identified models were applied on a large registry database comprising 11 395 cardiac surgical interventions. The probabilities of prolonged ICU length of stay based on the models were compared with the actual outcome to assess the discrimination and calibration performance of the models. Literature review identified 20 models, of which 14 could be included. Of the 6 models for the general cardiac surgery population, the Parsonnet model showed the best discrimination (area under the receiver operating characteristic curve=0.75 [95% confidence interval, 0.73 to 0.76]), followed by the European system for cardiac operative risk evaluation (EuroSCORE) (0.71 [0.70 to 0.72]) and a model by Huijskes and colleagues (0.71 [0.70 to 0.73]). Most of the models showed good calibration.

Conclusions— In this validation of prediction models for prolonged ICU length of stay, 2 widely implemented models (Parsonnet, EuroSCORE), although originally designed for prediction of mortality, were superior in identifying patients with prolonged ICU length of stay.

Received November 27, 2009; accepted June 4, 2010.

In the past decades, mortality during or shortly after cardiac surgery has decreased.1 However, morbidity has increased,2 mainly because cardiac surgery is increasingly utilized in older and more vulnerable patients. This often results in more complications after surgery and potential reduction in quality of life.3–5 One method of assessing complications occurring directly after cardiac surgery is a prolonged stay in the intensive care unit (ICU).6–9 Prolonged ICU stay also leads to incremental use of resources. In practice, prediction models are being used for efficient use of ICU resources. Patients with a low risk of complications are being scheduled for surgery before patients with a high risk.5–13 Various prediction models have been developed to preoperatively identify patients with an increased risk for postoperative complications and prolonged ICU stay.12–28 Interestingly, all of these prediction models were derived from samples including different patients, as reflected by the different distributions of patient and outcome characteristics. Hence, which model should be preferred in which situation is still unclear. Recently, in a qualitative review, Messaoudi and colleagues14 reviewed 13 of these prediction models by comparing their published prognostic values for predicting ICU stay. They found that the 13 different prediction models indeed used different definitions of prolonged ICU stay and different definitions of predictors.

Clinical Perspective on p 689

Even though it is widely accepted that no prediction model should be applied in practice before being formally validated on its predictive accuracy in new patients,29–31 no study has previously performed a formal, quantitative (external) validation of these prediction models in an independent patient population. Therefore, we first conducted a systematic review to identify all existing prediction models for prolonged ICU length of stay (PICULOS) after cardiac surgery. Subsequently, we validated the performance of the identified models in a large independent cohort of cardiac surgery patients.

Methods

Systematic Literature Review

In February 2008, the MEDLINE and PreMEDLINE databases were searched for studies on prediction models for PICULOS after cardiac surgery that were published after 1980. The precise search query is presented in Appendix I in the online-only Data Supplement.

The retrieved articles were reviewed by 2 reviewers (R.G.A.E. and L.M.P.) and retained when they presented a formally developed prediction model. There is no consensus on the exact definition of PICULOS.14 To relate to clinical practice,2,11,13,15–28 we further restricted our analysis to prediction models that used a threshold for PICULOS within the bounds of 24 to 72 hours.

Application of the Models to an Independent Cohort

The validation of the retrieved models was then performed on a large cohort of cardiac surgery patients who underwent surgery between January 1, 2000, and July 31, 2008, at the Isala Clinics, Zwolle, Netherlands (1400 cardiac surgery procedures per year). The data had been collected prospectively as part of a continuous data registry for the national cardiac surgery patient registration. All patients provided informed consent to use the data for research. Patients’ identifying information was removed before the analysis.

When the original articles did not provide sufficient information on the included predictors or regression coefficients (log odds ratios) in the model, the authors were asked to personally provide this information. If the information obtained was insufficient to apply the model to our data, the study was excluded from the analysis.

To validate the performance of the retrieved models, we used the original formulas and applied them to our patients using their observed predictor values. This yielded a predicted probability of PICULOS for each patient based on each model. To do this, we first matched the predictors in each prediction model to the variables in our data set. When a predictor was not available in our data set, we proceeded as follows. First, we sought to replace the variable with a proxy variable. Second, if a proxy was not available, we imputed the incidence or mean value reported in the literature for these predictors.32–34 To prevent overimputation, this option was applied only when the weight of the predictor in the corresponding prediction model was relatively low compared with the other predictors in that model because it has a tempering effect on the predictive ability of the model. As a consequence, we only used this method for the predictors “family history” in the Parsonnet model35,36 and “preoperative hemoglobin level” in the model of Huijskes et al.17,37 If neither of these methods could be applied, the model was excluded from the analysis.13,25

Data Analysis

To analyze the performance of each prediction model, each patient’s predicted probability of PICULOS in each model was compared with the observed outcome (ie, whether the patient had actually experienced PICULOS [yes/no]). To allow for a fair comparison of the models, a threshold for observed PICULOS had to be chosen. On the basis of the literature15,17,21,22,25,26,28 and current clinical practice, we defined observed PICULOS as an ICU length of stay of >48 hours.

In comparing the performance of the models, we focused on discrimination and calibration. The discrimination performance of a model indicates the extent to which the model distinguishes between patients with and without prolonged ICU stay. The discrimination performance of the models was expressed by constructing receiver operating characteristic curves for each of the models and calculating the area under the curve (AUC) with a 95% confidence interval.38 Theoretically, the AUC ranges from 0.5 (no predictive ability at all) to 1 (perfect predictive ability). In practice, however, the AUC can be well below the theoretical maximum of 1 even if the prediction model is perfectly calibrated, especially in complex diseases.39

The calibration performance of a model describes the extent to which the predicted probability of prolonged ICU stay reflects the true probability of prolonged ICU stay. The calibration of the models was judged by constructing calibration plots,40 relating the predicted and observed probabilities. The calibration performance of a prediction model in an independent data set (external validation set) is commonly influenced by the incidence of the outcome in the validation set.

To allow for a fair comparison of the models, we adjusted the intercept of each model before applying it to the data, such that the mean predicted probability was equal to the observed outcome frequency.34,41 Calibration plots were constructed subsequently. For each model, the U statistic (which compares the actual slope and intercept of the calibration plot to the ideal values of 1 and 0, respectively) was calculated and tested against a χ2 distribution with 2 degrees of freedom.33

To further measure the accuracy of the models, we calculated the Yates slope (difference between the mean predicted probabilities for the patients with and without actual prolonged ICU stay) and the Brier scores (quadratic difference between predicted probability and actual outcome [0 or 1] for each patient) for each of the models.42 All of these measures provide insight into the distance the model creates between the patients with and without a prolonged ICU stay.

Missing values occurred for the variables “gender” (0.05%), “myocardial infarction” (0.14%), “serum creatinine” (2.86%), “smoking” (0.13%), “height” and “weight” (both 45% of cases), “New York Heart Association classes” (0.92%), and the outcome variable “ICU length of stay” (1.71%). Missing values were substituted by means of single regression and weighted mean imputation, both of which are widely known methods for the substitution of missing values to reduce bias and increase statistical power.32 Two-sided statistical tests were conducted with a significance level of 0.05. The statistical package R (version 2.10.1 [2009-12-14], The R Foundation for Statistical Computing, Vienna, Austria) was used for statistical analysis.

Results

Systematic Literature Review

Figure 1 shows the flow chart of the systematic literature review. From the 56 articles that matched the initial search query, 25 articles described 20 different prediction models.* Two models were excluded because they used a threshold of >72 hours to define prolonged ICU stay.8,12 Additional information on intercepts, coefficients, and definitions of predictors in the models was requested from the authors for 7 models.11,15,16,24,25,27,35 Two authors responded with the requested information,11,16 2 authors responded but were not able to provide the requested information,15,27 and 4 authors did not respond. Three models were excluded because necessary information in regard to the definitions of the variables used was missing,10,15,25 and 1 model was excluded because no adequate information was available in the database.13 Finally, 14 prediction models could be included in our validation study.

Figure1

Figure 1. Flowchart of the systematic review of prediction models for prolonged ICU stay after cardiac surgery.

Six of these 14 models were developed for patients undergoing cardiac surgery in general,17,21,26,27,35,43,44 whereas the 8 other models focused on patients undergoing isolated coronary artery bypass grafting (CABG) surgery.2,16,18,20,23,24,27 Two of the 14 prediction models, the Parsonnet model35 and the European system for cardiac operative risk evaluation (EuroSCORE),43,44 were originally designed for the prediction of mortality after cardiac surgery but have been used and validated for prolonged ICU stay.7,8,11,21,28 Therefore, these models were also included in our study.

Table 1 describes the general characteristics of the 14 selected prediction models. Appendix II in the online-only Data Supplement provides a more extensive overview of the characteristics of the prediction models according to the framework established by Laupacis and colleagues.45

View this table:

Table 1. General Characteristics of the Studied Prediction Models

Predictive Performance

Table 2 describes the baseline characteristics of the patients in our cohort. We tested the prediction models in our cohort on the type of patients for which they were developed; prediction models developed for cardiac surgery in general were evaluated on all patients (n=11 395), and prediction models developed for isolated CABG patients were evaluated on patients who underwent isolated CABG (n=6463) only.

View this table:

Table 2. Baseline Characteristics of Patients in the Database

Figure 2 depicts the receiver operating characteristic curves for each of the models, and Table 3 depicts the accompanying statistics. Among models including all cardiac surgeries, the Parsonnet model8,11,35 showed the best discrimination (AUC 0.75 [95% confidence interval, 0.73 to 0.76]), followed by the EuroSCORE7,21,28,43,44 (0.71 [0.70 to 0.72]) and a model by Huijskes and colleagues17 (0.71 [0.70 to 0.73]). Among the models specifically developed for patients undergoing isolated CABG, the models by Wong et al,23 Ivanov et al,20 and Tuman et al27 showed the best discrimination, with AUCs of 0.68 (0.65 to 0.70), 0.67 (0.65 to 0.70), and 0.66 (0.64 to 0.68], respectively.

Figure2

Figure 2. Receiver operating characteristic (ROC) curves for all 14 prediction models. The diagonal line represents zero discriminative value and corresponds to an AUC of 0.50. ICULOS indicates ICU length of stay.

View this table:

Table 3. Predictive Performance of Prediction Models in the Study Cohort

Figure 3A and 3B show calibration plots of the 2 best- and the 2 least-performing models after adjustment of the intercept of each model for all cardiac surgery patients and isolated CABG patients, respectively. For most of the models, the calibration line in the plot closely followed the ideal calibration line, except for the models of Wong et al23 and Abrahamyan et al.16 The 6 models for the general cardiac surgery population had low P values for the U statistic (Table 3), indicating that the 6 models do not provide accurate probabilities. For the isolated CABG surgery patients, only the models of Tuman et al27 and Christakis I (containing only preoperative predictors)24 had nonsignificant P values.

Figure3

Figure 3. A, Calibration plots for models for all cardiac surgery. One plot for the 2 best-performing models (Parsonnet [solid red line] and EuroSCORE [dashed blue line]) and 1 plot for the 2 least-performing models (Tu [solid red line] and Pitkänen [dashed blue line]) are shown. The dotted line represents ideal calibration (with intercept 0 and regression coefficient 1); n=11 395. B, Calibration plots for models for isolated CABG surgery. One plot for the 2 best-performing models (Tuman [solid red line] and Ivanov [dashed blue line]) and 1 plot for the 2 least-performing models (Wong [solid red line] and Abrahamyan [dashed blue line]) are shown The dotted line represents ideal calibration (with intercept 0 and regression coefficient 1); n=6463.

Discussion

We conducted a systematic review and validated the performance of 14 retrieved prediction models to identify patients with prolonged ICU stay after cardiac surgery, using a large cohort of cardiac surgery patients. In this first quantitative comparison of all prediction models to identify patients who are likely to have a prolonged ICU stay, the Parsonnet model and the EuroSCORE show the best performance in terms of discrimination, accuracy, and calibration. Although both models were originally developed to predict mortality, we found that they are also superior in identifying patients with an increased risk of prolonged ICU stay. A major explanation lies in the fact that, in current practice, mortality has decreased but morbidity has increased.1,2 Because of advances in perioperative care in cardiac surgery,46 most of the patients who were likely to die in the era when the Parsonnet model and the EuroSCORE were developed will now survive, but they still have a higher probability of developing complications. This is also supported by Parolari and colleagues,47 who noted a significant overestimation of mortality with the EuroSCORE. Because both models overestimate mortality in current practice, these models for mortality need to be corrected for improved level of care in the future.

In the systematic review, we found 20 prediction models for prolonged ICU stay, 14 of which we could include in our analysis. In accordance with Messaoudi et al,14 we found considerable differences in the definitions of the predictors and outcomes. We chose to restrict our systematic review to prediction models that used a threshold for PICULOS within the bounds of 24 to 72 hours. Afterward, in our validation study, we used the threshold of 48 hours because this correlates best with clinical practice. To verify the extent to which this difference has influenced our findings, we repeated the validation analysis using threshold values of 24 and 72 hours. This did not influence the ranking of the models based on their performance.

Substantial differences between the models were also found in the sizes of the databases used to develop the prediction models and in the number of predictors in the models. Only 10 of the 14 models were initially validated, 9 of which used an independent validation set,† and 1 was validated by means of bootstrapping.23 Prospective validation, however, was done for only 4 models.21,35,44 In every case, the validation of the models was done in relatively small data sets (sizes ranging from 39427 to 243917). Only Parsonnet’s model,8,11 the EuroSCORE,7,21,28 and Tu’s model8 were validated by other authors in a different geographic region. Because of all of these differences, the results of these original analyses are difficult to compare.

Our analysis is the first extensive quantitative validation of existing models for prolonged ICU stay after cardiac surgery in a large data set including >11 000 patients. All models were validated on the same data set, which allows for a proper comparison of the performance of the models.

To determine the calibration of the models, we made calibration plots and calculated the U statistic and the Hosmer-Lemeshow statistic. At first sight, these approaches gave contradictory results. In most of the models, the Hosmer-Lemeshow and U statistics had a P value <0.05, suggesting that the predictions based on the model deviated significantly from the observed data. In contrast, the calibration lines in the plots were very close to the 45° line, suggesting near-perfect calibration. To gain insight into the cause of these large statistics, we furthermore calculated the t values for the slopes of the models. This revealed that the slopes of the models in this data set deviate significantly from the ideal slope of 1. This would explain the large χ2 values even after recalibration by adjusting the intercepts only. Whereas calibration statistics are merely summary measures, calibration plots directly reveal the variation of the performance of the model over the entire range of probabilities.48

Table 3 also shows the importance of recalibrating a model by adjusting the intercept34,41 before calibration of the model is assessed. The mean predicted risks of the original models do not even approach the observed outcome frequency, whereas after recalibration this problem is solved. This allows for a more fair comparison of the models and a better performance when the models are applied in daily practice.

To determine the discrimination performance of the models, we calculated the AUCs. The 6 models for the general cardiac surgery population yielded AUCs ranging from 0.68 to 0.74. In the models specifically developed for patients with isolated CABG surgery, substantially lower AUCs (0.56 to 0.67) were found. In general, values for the AUC <0.70 indicate that use of the model in clinical practice should be done with caution49 because the theoretical maximum value of the AUC is 1.0. However, it is also known that in practice this maximum depends not only on the model but also on characteristics of the data.39 To allow for better interpretation of our findings and provide a “benchmark value,” we fitted 2 reference models on the data (1 for all patients, 1 for patients undergoing isolated CABG only), which yielded AUCs of 0.80 and 0.73, respectively. These models are likely to be overfit but give a reference value for interpretation of the AUCs of the prediction models found in the literature.

We also found a considerable difference in AUC between the models for all patients and the models predicting prolonged ICU stay after isolated CABG procedures. To investigate whether this was due to the models or due to the differences in population characteristics (isolated CABG patients versus all patients), the 6 models for the general population were also applied to the isolated CABG patients only, resulting again in AUCs varying from 0.55 to 0.69. These AUCs are comparable to the AUCs of the models specially developed for isolated CABG surgery patients. This suggests that it is more complicated to predict prolonged ICU stay in isolated CABG surgery patients than in the cardiac surgery population as a whole. The Parsonnet model and the EuroSCORE showed the best discrimination (0.69 and 0.68, respectively) in the CABG surgery population. The Parsonnet model performed even better than the best-performing model (Wong, 0.68) that was specially developed for CABG surgery patients only.

Limitations

Obviously, prolonged ICU stay is intrinsically a continuous variable (length of stay). Accordingly, as with most continuous variables in medicine, one would rather not dichotomize14 but would rather predict the original length of stay value itself. However, all published models used as outcome dichotomized prolonged stay (length of stay with some threshold value), and our purpose was to validate these models as published.

We made use of a prospective continuous data registry that includes all patients who underwent surgery and systematically recorded a large amount of information on preoperative, perioperative, and postoperative characteristics. A disadvantage of using registry data is that not all predictors of the models are available in the registry with exactly the same definition as used to develop these models. We have solved this problem in part by using proxy variables and by replacing missing variables with the incidence or mean of the predictor based on the literature. When too many concessions had to be made before the model could be applied to our data, we excluded the prediction model from this validation study.8,10,12,13,15,25 Therefore, we do not think that the use of registry data has significantly influenced our conclusions. On the contrary, by using registry data we validated the performance of the models in daily clinical practice, which was specifically the aim of our study.

For most of the variables in the data set, the percentage of missing data was small. For height and weight, however, data were missing in 45% of the cases. Deleting 45% of the patient records (doing a complete case analysis) is widely known to yield biased results.32 We thus applied the best available methods to properly deal with these missing data and minimize this bias and explicitly chose to impute the data by fitting a model.32,50 With a percentage as high as 45% for missing data for 2 variables, theoretically multiple imputation is to be preferred over single imputation. However, in the context of multiple imputation, the manner in which to estimate the standard errors of part of the performance measures we used in this study is not straightforward. We have performed multiple imputation as a sensitivity analysis and found similar results for the point estimates, indicating that the numbers presented in this article are not influenced by the choice of the imputation strategy. We realize that we made use of data from a single center over a longer time period, which must be taken into account when our findings are generalized.

Conclusions

This extensive quantitative validation study demonstrates that the widely implemented Parsonnet and EuroSCORE models are superior to other models in predicting prolonged ICU stay after cardiac surgery. In current daily practice, Parsonnet’s model and the EuroSCORE are widely implemented for the prediction of mortality risk. This allows for the relatively straightforward application of our findings in clinical practice. The predictions that have already been made for mortality can also be used to identify patients with a high probability of prolonged ICU stay. This knowledge, when available before surgery, can be used for timely planning of postoperative care and ICU management.

Acknowledgments

Professor Moons gratefully acknowledges The Netherlands Organization for Scientific Research for their financial support (project nr. 016.046.360 and 912.08.004).

Disclosures

None.

References

  1. ↵
    Northrup WF, Emery RW, Nicoloff DM, Lillehei TJ, Holter AR, Blake DP. Opposite trends in coronary artery and valve surgery in a large multisurgeon practice, 1979–1999. Ann Thorac Surg. 2004; 77: 488–495.
    OpenUrlCrossRefPubMed
  2. ↵
    Ghotkar SV, Grayson AD, Fabri BM, Dihmis WC, Pullan DM. Preoperative calculation of risk for prolonged intensive care unit stay following coronary artery bypass grafting. Eur J Cardiothorac Surg. 2006; 31: 1–14.
    OpenUrlCrossRefPubMed
  3. ↵
    Litovski DS, Dacey LJ, Baribeau YR, Leavitt BJ, Clough R, Cochran RP, Quinn R, Sisto DA, Charlesworth DC, Malenka DJ, MacKenzie TA, Olmstead EM, Ross CS, O'Connor GT. Long-term survival of the very elderly undergoing coronary artery bypass grafting. Ann Thorac Surg. 2008; 85: 1233–123.
    OpenUrlCrossRefPubMed
  4. ↵
    Pätilä T, Kukkonen S, Vento A, Pettilä V, Suojaranta-Ylinen R. Relation of the sequential organ failure assessment score to morbidity and mortality after cardiac surgery. Ann Thorac Surg. 2006; 82: 2072–2079.
    OpenUrlCrossRefPubMed
  5. ↵
    Stoica SC, Sharples LD, Ahmed I, Roques F, Large SR, Nashef SAM. Preoperative risk prediction and intraoperative events in cardiac surgery. Eur J Cardiothorac Surg. 2002; 21: 41–46.
    OpenUrlAbstract/FREE Full Text
  6. ↵
    Bucerius J, Gummert JF, Walther T, Doll N, Falk V, Schmitt DV, Mohr FW. Predictors of prolonged ICU stay after on-pump versus off-pump coronary artery bypass grafting. Intensive Care Med. 2004; 30: 88–95.
    OpenUrlCrossRefPubMed
  7. ↵
    Pinna Pintor P, Bobbio M, Colangelo S, Veglia F, Marras R, Diena M. Can EuroSCORE predict direct costs of cardiac surgery? Eur J Cardiothorac Surg. 2003; 23: 595–598.
    OpenUrlAbstract/FREE Full Text
  8. ↵
    Hsieh CH, Peng SK, Tsai TC, Shih YR, Peng SY. Prediction for major adverse outcomes in cardiac surgery: comparison of three prediction models. J Formos Med Assoc. 2007; 106: 759–767.
    OpenUrlCrossRefPubMed
  9. ↵
    Scott BH, Seifert FC, Grimson R, Glass PSA. Octogenarians undergoing coronary artery bypass graft surgery: resource utilization, postoperative mortality, and morbidity. J Cardiothorac Vasc Anesth. 2005; 19: 583–588.
    OpenUrlCrossRefPubMed
  10. ↵
    Salamonsen RS, Bailey MJ, Salamonsen RF. Can ICU stay be predicted accurately enough for fast-tracking cardiac surgical patients? Crit Care Resusc. 2008; 10: 194–201.
    OpenUrlPubMed
  11. ↵
    Lawrence DR, Valencia O, Smith EEJ, Murday A, Treasure T. Parsonnet score is a good predictor of the duration of intensive care unit stay following cardiac surgery. Heart. 2000; 83: 429–432.
    OpenUrlAbstract/FREE Full Text
  12. ↵
    Rosenfeld R, Smith JM, Woods SE, Engel AM. Predictors and outcomes of extended intensive care unit length of stay in patients undergoing coronary artery bypass graft surgery. J Card Surg. 2006; 21: 146–150.
    OpenUrlCrossRefPubMed
  13. ↵
    Hein OV, Birnbaum J, Wernecke K, England M, Konertz W, Spies C. Prolonged intensive care unit stay in cardiac surgery: risk factors and long-term survival. Ann Thorac Surg. 2006; 81: 880–885.
    OpenUrlCrossRefPubMed
  14. ↵
    Messaoudi N, De Cocker J, Stockman B, Bossaert LL, Rodrigus IER. Prediction of prolonged length of stay in the intensive care unit after cardiac surgery: the need for a multi-institutional risk scoring system. J Card Surg. 2009; 24: 127–133.
    OpenUrlCrossRefPubMed
  15. ↵
    Atoui R, Ma F, Langlois Y Morin JF. Risk factors for prolonged stay in the intensive care unit and on the ward after cardiac surgery. J Card Surg. 2008; 23: 99–106.
    OpenUrlCrossRefPubMed
  16. ↵
    Abrahamyan L, Demirchyan A, Thompson ME, Hovaguimian H. Determinants of morbidity and intensive care unit stay after coronary surgery. Asian Cardiovasc Thorac Ann. 2006; 14: 114–118.
    OpenUrlAbstract/FREE Full Text
  17. ↵
    Huijskes RV, Rosseel PM, Tijssen JG. Outcome prediction in coronary artery bypass grafting and valve surgery in the Netherlands: development of the Amphiascore and its comparison with the EuroSCORE. Eur J Cardiothorac Surg. 2003; 24: 741–749.
    OpenUrlAbstract/FREE Full Text
  18. ↵
    Janssen DP, Noyez L, Wouters C, Brouwer RM. Preoperative prediction of prolonged stay in the intensive care unit for coronary bypass surgery. Eur J Cardiothorac Surg. 2004; 25: 203–207.
    OpenUrlAbstract/FREE Full Text
  19. ↵
    Geissler HJ, Hözl P, Marohl S, Kuhn-Régnier F, Mehlhorn U, Südkamp M, Rainer de Vivie E. Risk stratification in heart surgery: comparison of six score systems. Eur J Cardiothorac Surg. 2000; 17: 400–406.
    OpenUrlAbstract/FREE Full Text
  20. ↵
    Ivanov J, Borger MA, David TE, Cohen G, Walton N, Naylor CD. Predictive accuracy study: comparing a statistical model to clinicians’ estimates of outcomes after coronary bypass surgery. Ann Thorac Surg. 2000; 70: 162–168.
    OpenUrlCrossRefPubMed
  21. ↵
    Pitkänen O, Niskanen M, Rehnberg S, Hippeläinen M, Hynynen M. Intra-institutional prediction of outcome after cardiac surgery: comparison between a locally derived model and the EuroSCORE. Eur J Cardiothorac Surg. 2000; 18: 703–710.
    OpenUrlAbstract/FREE Full Text
  22. ↵
    Ivanov J, Tu JV, Naylor DC. Ready-made, recalibrated, or remodeled? Issues in the use of risk indexes for assessing mortality after coronary artery bypass graft surgery. Circulation. 1999; 99: 2098–2104.
    OpenUrlAbstract/FREE Full Text
  23. ↵
    Wong DT, Cheng DCH, Kustra R, Tibshirani R, Karski J, Carroll-Munro J, Sandler A. Risk factors of delayed extubation, prolonged length of stay in the intensive care unit, and mortality in patients undergoing coronary artery bypass graft with fast-track cardiac anesthesia: a new cardiac risk score. Anesthesiology. 1999; 91: 936–944.
    OpenUrlCrossRefPubMed
  24. ↵
    Christakis GT, Fremes SE, Naylor CD, Chen E, Rao V, Goldman BS. Impact of preoperative risk and perioperative morbidity on ICU stay following coronary bypass surgery. Cardiovasc Surg. 1996; 4: 29–35.
    OpenUrlCrossRefPubMed
  25. ↵
    Michalopoulos A, Tzelepis G, Pavlides G, Kriaras J, Dafni U, Geroulanos S. Determinants of duration of ICU stay after coronary artery bypass graft surgery. Br J Anaesth. 1996; 77: 208–212.
    OpenUrlAbstract/FREE Full Text
  26. ↵
    Tu JV, Mazer CD, Levinton C, Armstrong PW, Naylor CD. A predictive index for length of stay in the intensive care unit following cardiac surgery. Can Med Assoc J. 1994; 151: 177–185.
    OpenUrlAbstract
  27. ↵
    Tuman KJ, McCarthy RJ, March RJ, Najafi H, Ivankovich AD. Morbidity and duration of ICU stay after cardiac surgery: a model for preoperative risk assessment. Chest. 1992; 102: 36–44.
    OpenUrlCrossRefPubMed
  28. ↵
    Nilsson J, Algotsson L, Höglund P, Lührs C, Brandt J. EuroSCORE predicts intensive care unit stay and costs of open heart surgery. Ann Thorac Surg. 2004; 78: 1528–1535.
    OpenUrlCrossRefPubMed
  29. ↵
    Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med. 1999; 130: 515–524.
    OpenUrlCrossRefPubMed
  30. ↵
    Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006; 144: 201–209.
    OpenUrlCrossRefPubMed
  31. ↵
    Moons KG, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? BMJ. 2009; 338: b375.
    OpenUrlFREE Full Text
  32. ↵
    Janssen KJ, Donders AR, Harrell FE Jr, Vergouwe Y, Chen Q, Grobbee DE, Moons KG. Missing covariate data in medical research: to impute is better than to ignore. J Clin Epidemiol. 2010; 63: 721–727.
    OpenUrlCrossRefPubMed
  33. ↵
    Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic model issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996; 15: 361–387.
    OpenUrlCrossRefPubMed
  34. ↵
    Steyerberg EW, Eijkemans MJC, Van Houwelingen JC, Lee KJ, Habbema JDF. Prognostic models based on literature and individual patient data logistic regression analysis. Stat Med. 2000; 19: 141–160.
    OpenUrlCrossRefPubMed
  35. ↵
    Parsonnet V, Dean D, Bernstein AD. A method of uniform stratification of risk for evaluating the results of surgery in acquired adult heart disease. Circulation. 1989; 79 (suppl 1): I-3–I-12.
    OpenUrl
  36. ↵
    Nasir K, Budoff MJ, Wong ND, Scheuner M, Herrington D, Arnett DK, Szklo M, Greenland P, Blumenthal RS. Family history of premature coronary heart disease and coronary artery calcification: Multi-Ethnic Study of Atherosclerosis (MESA). Circulation. 2007; 116: 619–626.
    OpenUrlAbstract/FREE Full Text
  37. ↵
    Ouattara A, Niculescu M, Boccara G, Landi M, Vaissier E, Léger P, Riou B, Gandjbakch I, Coriat P. Identification of risk factors for allogenic transfusion in cardiac surgery from an observational study. Ann Fr Anesth Reanim. 2003; 22: 278–283.
    OpenUrlCrossRefPubMed
  38. ↵
    Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982; 143: 29–36.
    OpenUrlCrossRefPubMed
  39. ↵
    Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007; 115: 928–935.
    OpenUrlAbstract/FREE Full Text
  40. ↵
    Spiegelhalter DJ. Probabilistic prediction in patient management and clinical trials. Stat Med. 1986; 5: 421–433.
    OpenUrlCrossRefPubMed
  41. ↵
    Janssen KJM, Moons KGM, Kalkman CJ, Grobbee DE, Vergouwe Y. Updating a clinical prediction model improved the performance in new patients. J Clin Epidemiol. 2008; 61: 76–86.
    OpenUrlCrossRefPubMed
  42. ↵
    Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattane MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010; 21: 128–138.
    OpenUrlCrossRefPubMed
  43. ↵
    Nashef SA, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon R. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg. 1999; 16: 9–13.
    OpenUrlAbstract/FREE Full Text
  44. ↵
    Roques F, Nashef SAM, Michel P, Gauducheau E, de Vincentiis C, Baudet E, Cortina J, David M, Faichney A, Gabrielle F, Gams E, Harjula A, Jones MT, Pinna Pintor P, Salamon R, Thulin L. Risk factors and outcome in European cardiac surgery: analysis of the EuroSCORE multinational database of 19030 patients. Eur J Cardiothorac Surg. 1999; 15: 816–823.
    OpenUrlAbstract/FREE Full Text
  45. ↵
    Laupacis A, Sekar N, Stiell IG. Clinical prediction rules: a review and suggested modifications of methodological standards. JAMA. 1997; 277: 488–494.
    OpenUrlCrossRefPubMed
  46. ↵
    Svircevic V, Nierich AP, Moons KG, Brandon Bravo Bruinsma GJ, Kalkman CJ, van Dijk D. Fast-track anesthesia and cardiac surgery: a retrospective cohort study of 7989 patients. Anesth Analg. 2009; 108: 727–733.
    OpenUrlCrossRefPubMed
  47. ↵
    Parolari A, Pesce LL, Trezzi M, Loardi C, Kassem S, Brambillasca C, Miguel B, Tremoli E, Biglioli P, Alamanni F. Performance of EuroSCORE in CABG and off-pump coronary artery bypass grafting: single institution experience and meta-analysis. Eur Heart J. 2009; 30: 297–304.
    OpenUrlAbstract/FREE Full Text
  48. ↵
    Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation and Updating. New York, NY: Springer; 2009.
  49. ↵
    Harrell FE Jr. Regression Modelling Strategies. New York, NY: Springer; 2001.
  50. ↵
    Donders AR, van der Heijden GJ, Stijnen T, Moons KG. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol. 2006; 59: 1087–1091.
    OpenUrlCrossRefPubMed

CLINICAL PERSPECTIVE

Prolonged intensive care unit (ICU) stay after cardiac surgery leads to potential reduction in quality of life and incremental use of resources. For efficient use of ICU resources and to schedule patients with a low risk of postoperative complications before patients with a higher risk, preoperative estimation of the risk of prolonged ICU stay is necessary. Various prediction models have been developed to preoperatively identify patients with an increased risk for prolonged ICU stay. It is widely accepted that no prediction model should be applied in practice before being formally validated in new patients. In the domain of prolonged ICU stay after cardiac surgery, however, no study has thus far conducted such a formal validation and comparison study. The present analysis is the first extensive quantitative validation of existing models for prolonged ICU stay after cardiac surgery in a large data set of 11 395 patients. The results show that the Parsonnet model and the European system for cardiac operative risk evaluation (EuroSCORE) have the overall best performance. Although both models were originally developed to predict mortality, they are also superior in identifying patients with an increased risk of prolonged ICU stay. Because in current daily practice both models are widely implemented for the estimation of mortality risk, this allows for a relatively straightforward application of our findings in clinical practice. The risk stratification for mortality based on these models can also be used to identify patients with an increased risk of prolonged ICU stay, which is useful for timely planning of postoperative care and ICU management.

Footnotes

View Abstract

This Issue

Jump to

Article Tools

Share this Article

  • Email
  • Prediction Models for Prolonged Intensive Care Unit Stay After Cardiac Surgery
    Roelof G.A. Ettema, Linda M. Peelen, Marieke J. Schuurmans, Arno P. Nierich, Cor J. Kalkman and Karel G.M. Moons
    Circulation. 2010;122:682-689, originally published August 16, 2010
    https://doi.org/10.1161/CIRCULATIONAHA.109.926808
    Permalink:
    del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo

Related Articles

Cited By...

Subjects