Screening Scale Predicts Patients Successfully Receiving Long-term Implantable Left Ventricular Assist Devices
Background Although use of long-term implantable left ventricular assist devices (LVAD) is becoming more popular, further reduction of the mortality rate accompanying device insertion through improved patient selection would make this alternative even more appealing. We sought to develop a scoring system that was based on criteria obtainable at the time of evaluation and predictive of successful early outcome and simple to apply.
Methods and Results Patients (n=56) undergoing LVAD insertion between 1990 and 1994 were screened for easily obtainable preoperative risk factors. To test the association between survival and each risk factor, a χ2 analysis was performed, and relative risks were estimated. Oliguria, ventilator dependence, elevated central venous pressure, elevated prothrombin time, and reoperation status had low probability values and high estimated relative risks. On the basis of these relations, a risk factor–selection scale (RFSS) (range, 0 to 10) was developed by computing appropriate weights for each risk factor. The distribution of patients for each scale score reveal that with RFSS ≥5, most device recipients will die (P<.001). The average RFSS (±SD) of survivors (n=42) was 2.45±1.73 compared with 5.43±2.85 in nonsurvivors (n=14) (P<.0001). Univariate logistical regression was also significant (score statistic, 16.2; df=1; P=.001).
Conclusions The RFSS is simple, easy to apply, and statistically valid. Physicians could use the scale as a starting point in discussing the suitability for LVAD implantation in a specific patient and as a basis for comparing patient outcomes.
The recent Food and Drug Administration (FDA) approval of a long-term implantable left ventricular assist device1 (LVAD) for bridge-to-transplantation support will increase the ability of open heart surgical facilities to implant these devices. The FDA was prompted to allow widespread use of LVADs by trials that demonstrated safety and efficacy of the interventions.2 For long-term implantable devices such as the ThermoCardiosystems Heartmate and Baxter Novacor, survival to transplantation is approximately 65%3 and survival after transplantation approaches 90%. Patients are prepared for their transplants by device insertion followed by aggressive physical rehabilitation. These results are encouraging, especially in light of the dismal prognosis of patients without support.4 However, further reduction of the mortality and morbidity accompanying device insertion would make this alternative more appealing and possibly lead to LVAD implantation into healthier patients. The major causes of perioperative mortality—hemorrhage and right heart failure—have been reduced with increasing experience with devices and use of aprotinin.5 However, major additional improvements will probably result only from improved patient screening.
Although several authors6 7 have studied the risk factors for patients undergoing long-term LVAD insertion, the results are usually expressed as a list of items and do not provide specific guidelines, including scale cutoffs, to allow stratification of patients into high- and low-risk LVAD candidates. In part, this results from the relatively small number of long-term implantable devices that have been inserted at any one center. The more widespread use of LVADs over the past several years has facilitated such an analysis.
We sought to develop a scoring system that was based on criteria obtainable at the time of evaluation and predictive of successful early outcome and simple to apply. This clinically usable scale should predict which patients are unlikely to survive device insertion and on whom intervention would likely be futile. Physicians could use the scale as a starting point in discussing the suitability of a specific patient and as a basis for comparing patient outcomes.
Patients and Device
Between August 1, 1990, and September 30, 1994, 56 patients underwent Thermo Cardiosystems Heartmate 1000IP or 1205VE LVAD placement as a bridge to transplantation at the Columbia-Presbyterian and Cleveland Clinic Foundation Medical Centers. These individuals signed an institutional review board–approved consent form to participate in a trial studying the efficacy of mechanical ventricular assistance. This pusher-plate device, with a maximum stroke volume of 85 mL, is implanted through a median sternotomy with an inflow cannula inserted into the ventricular apex and an outflow graft anastomosed to the ascending aorta (Fig 1⇓). The pumping chamber is placed in the abdomen in either a preperitoneal or intraperitoneal position. An internal sensor measures pump filling volume and averages pump output (pump rate multiplied by filling volume) over 30 seconds. The device decompresses the left ventricle and ejects blood independent of the patient’s native rhythm; the aortic valve rarely opens.
Clinical charts were reviewed, and preoperative hemodynamic, laboratory, and historical data were retrieved for all 56 patients. The point of data gathering was standardized as the time at which the decision was made to insert the device. There was 100% data retrieval for each criteria. The mean age of the patients was 50 years (range, 18 to 64 years). Eleven were women, and 45 were men. The mean duration of support among the survivors was 84 days.
The hemodynamic criteria for LVAD insertion first proposed by Norman et al8 were met by all patients; these included a cardiac index of <2.0 L/m2 per minute with a left atrial pressure or pulmonary capillary wedge pressure of >20 mm Hg. In addition, since the FDA approved trial-mandated use of the device as a bridge to transplantation, all patients receiving devices were approved for transplantation.
Patients who died without leaving the intensive care unit after LVAD implantation formed the nonsurvivor group. Patients who died from causes unrelated to the operation after the first postoperative week were included as perioperative survivors because appropriate preoperative screening would not have predicted these outcomes. As an example, a patient who died from a small bowel obstruction 4 weeks after successful device insertion was included in the perioperative survivors group.
Several strict contraindications to long-term implantable device placement were established before this study; these included significant aortic valve insufficiency, major cerebrovascular accident, body surface area <1.5 M2, and sepsis. Patients were also excluded if compliance problems were present or if comorbid conditions existed that would prove independently to be life limiting.
Screening criteria were selected to include an assessment of end-organ dysfunction and technical difficulty during the procedure. The three end organs screened were the liver, kidney, and lung, and the criteria chosen were prothrombin time, oliguria, and endotracheal intubation. The technical challenge of the operation was increased by prior cardiac operation, elevated central venous pressure, and preoperative coagulopathy. These criteria predispose the patient to hemorrhage, a major cause of right-side circulatory failure and subsequent death after LVAD insertion.4 In addition, because infection is often a cause of acute decompensation in a patient with limited cardiac reserve and because sepsis is poorly treated by LVAD implantation, we investigated the predictive value of leukocyte count and temperature.
The roles of additional potential criteria, including age, sex, origin of cardiomyopathy, length of hospital stay before device insertion, platelet count, cardiac index, and intra-aortic balloon pump dependence, were excluded early in the experience at Columbia-Presbyterian. Additional potential criteria were excluded because they were believed to be too insensitive in this group of patients. For example, the activated partial thromboplastin time was often elevated by the heparin administered to patients during cardiopulmonary bypass or intra-aortic balloon pump support. Likewise, serum creatinine changes lag behind significant changes in renal function and are insensitive compared with urine output in assessing end-organ perfusion.
Each of the seven studied risk factors for survival was either binary (ie, yes or no) or continuous. The binary risk factors were reoperation status and need for preoperative mechanical ventilation. The five continuous variables were dichotomized into yes-or-no categories; these included central venous pressure >16 mm Hg, prothrombin time >16 seconds, temperature >101.5°F, white blood cell count >15 000/mm3, and urinary output <30 mL/h over 12 hours despite diuretic administration. The selected category cutoff levels were believed to represent rational clinical criteria for determining patient outcomes.
A χ2 analysis to test the association between outcome and each risk factor was carried out. Also calculated was the measure of relative risk (RR) for each risk factor estimated as follows: (percent of deaths if risk factor is present)/(percent of deaths if risk factor is absent). On the basis of these statistical relations, a risk factor–selection scale (RFSS) was developed by computing appropriate weights (W) for each risk factor. Weights for each risk factor were assigned as follows: RR<1.5, W=0; 1.5≤RR<2.4, W=1; 2.4≤RR<3.5, W=2; and RR≥3.5, W=3. These weights were summed to obtain a continuous variable score ranging from 0 (least risk) to 10 (most risk) for each patient.
To model the relation between survival and the continuous variable scale, univariate logistic regression analysis was performed. An alternative technique that is commonly used to evaluate screening instruments is a receiver-operator characteristic (ROC) analysis.9 Originally developed in the field of signal detection theory, the ROC assesses whether a diagnostic system (ie, the RFSS) is accurate in predicting outcomes. At each level of the scale, true-positive values (sensitivity) are plotted against false-positive values (1 minus specificity). A suitable single-valued measure of test accuracy is the area of the entire graph that lies beneath the curve.10 A perfect diagnostic test would have an area of 1.0 because only true-positive results would be identified. ROC curves are also valuable in choosing the appropriate scale point above which device insertion would be prohibitively high risk.
Student’s t test was used to compare mean scale scores between LVAD-treated and untreated patients and between survivors and nonsurvivors. Differences were considered statistically significant when P<.05.
As defined, 42 of the patients (75%) survived the operation and 14 died (25%). Four patients (7%) died before transplantation from causes unrelated to the initial operation and were included in the survivors group. The seven 2×2 tables that tested the association between survival and the studied risk factor along with χ2 statistics are summarized in Table 1⇓. The calculation of relative risk allowed assignment of risk factor weights that are depicted in Table 2⇓.
Oliguria, ventilator dependence, elevated central venous pressure, and elevated prothrombin time had low P values and high estimated RR. Although reoperation status was not statistically significant, it was included in the scale with a small weight (W=1) due to the size of its RR. The lack of statistical significance and low RRs of leukocyte count and temperature suggest that these factors are not related to early perioperative mortality; accordingly, these criteria were not assigned weights (W=0) for further analysis. The influence of these criteria on overall survival to hospital discharge was not assessed in this study.
Using the assigned weights from Table 2⇑, a variable score was created by summing each patient’s risk factor weights. The resulting scale ranges between 0 and 10. The distribution of patients for each scale score is shown in Fig 2⇓ and reveals that at a scale score of >5, most device recipients will die. With an RFSS cutoff of <5 for survivors, Table 2⇑ can be collapsed into a 2×2 table (Table 3⇓) and reveals the strong statistical relation between this cutoff value and survival (P<.001). The average risk factor summation score (±SD) of survivors (n=42) was 2.45±1.73 compared with 5.43±2.85 in nonsurvivors (n=14) (P<.0001).
To model the relation between a binary outcome variable (survival) and a continuous explanatory variable (RFSS), a univariate logistical regression was performed (Fig 3⇓). The regression analysis was significant (score statistic, 16.2; df=1; P=.001), and the estimated model equation was found to be as follows:
An ROC curve (Fig 4⇓) was created using the weights assigned in Table 2⇑ to examine the relation between a positive and negative decision based on human judgment and the ability of the test to detect the difference.11 At each scale point, we can compute the screen’s ability to identify survivors (true-positives) at the cost of misidentifying nonsurvivors (false-positives). For example, at RFSS=1, we identify only 13 of 42 survivors (true-positive, 0.31) while incorrectly including 2 of 14 nonsurvivors (false-positive, 0.14). Using this plot of true-positives versus false-positives at each scale point, we could select an optimal point of the RFSS that maximizes true-positives and minimizes false-positives. From the plot, the optimal RFSS appears to occur at <5 points. The area under this curve is 0.81, the highest obtainable with these risk factors.
Historically, the referral patterns for patients who were candidates for cardiac transplantation were influenced by surgical results of this procedure. As in many other scenarios, as outcomes of the operation improved, surgeons were referred healthier patients who did even better, thus prompting even healthier candidates. A similar trend is occurring with patients referred for LVAD and will accelerate as selection criteria improve. If individuals without evidence of significant end-organ dysfunction undergo LVAD insertion, results should approach the 90% 1-year survival rate offered by orthotopic cardiac transplantation12 rather than the currently reported 70% survival rate during LVAD implantation.2
The RFSS proposed is logistically simple to apply and, in this limited study, allowed stratification of high-, medium-, and low-risk patients. Although no one factor should dissuade a physician from pursuing device insertion, the combination of particularly significant risk factors may indicate a higher-than-acceptable expected mortality rate. In medium-risk cases (ie, scores of 4 to 7), LVAD insertion may be postponed until the patient becomes a better candidate. In fact, aggressive medical treatment, intra-aortic balloon pump placement, or other temporary cardiac support may decrease central venous pressure, improve oxygenation, and increase urine output, thus rendering a patient suitable for LVAD insertion. Also, the use of the RFSS may allow physicians to refuse to operate on high-risk patients by demonstrating that the expected survival is unacceptably low. Likewise, patients who are too sick for heart transplantation might still have a reasonable score and could be argued to be acceptably low-risk candidates for LVAD implantation. If they survive and improve, these individuals may later again be candidates for heart transplantation.
The underlying principles of a RFSS to screen patients for LVAD insertion should include an assessment of intraoperative technical limitations and an evaluation of end-organ injury. Technically, device insertion is complicated by prior cardiac operation, preoperative coagulopathy, and the likelihood of right-side heart failure. These factors are interrelated; a patient who is coagulopathic and has had a prior operation is more likely to have substantial hemorrhage intraoperatively and postoperatively and thus be at higher risk for the development of right-side heart failure.5 Variations within these risk factor groups may affect the accuracy of the scale assessment. Reoperation on a patient who underwent open heart surgery many years earlier is relatively easier. Likewise, a patient whose coagulopathy was corrected with difficulty using transfusion of clotting factors is not expected to have the same clotting ability and operative complications as a patient whose baseline prothrombin time never wavered from normal limits. The origin of the coagulopathy is often severe hepatic congestion, and the liver may not be able to begin producing clotting factors immediately after separation from cardiopulmonary bypass, especially if significant right-side heart failure is evident.
End-organ injury indexes include pulmonary, hepatic, and renal insufficiency. Endotracheal intubation is a binomial criterion that is insensitive to different levels of oxygen requirement. However, the observation that a ventilator is required at all may be the most significant risk factor and is easy to quantify and follow between patients. Hepatocellular dysfunction after a period of shock is clinically most important if coagulation factor production is impaired. Especially after cardiopulmonary bypass, the dozens of units of fresh-frozen plasma transfusions required to overcome this degree of liver dysfunction can be difficult to administer in a timely fashion and may not result in adequate replenishment of short half-life components such as factor VII. Accordingly, the prothrombin time was selected as a screening criterion rather than bilirubin, which often lags behind in identifying hepatic injury, or transaminases, which may be too sensitive in revealing abnormalities of the liver. Urine output was selected as the appropriate criterion to reveal renal injury because it most closely reflects changes in renal perfusion. Creatinine is insensitive and lags behind renal function changes. Comparison of creatinine among institutions is also difficult because the value may be rising after a prolonged period of shock or may be recovering from a remote injury. Elevated creatinine may also reflect a baseline abnormality in function.
Dysfunction of each of these organs can be compensated for to a certain extent. Elevated peak end-inspiratory pressure, dialysis, and blood product administration may delay catastrophe; however, severe end-organ dysfunction will often prove lethal. In addition, although intensive postoperative care can salvage these patients, the margin for error is smaller, and a higher percentage of patients will have complicated courses, which may result in early death. In sum, patients with significant technical challenges and other end-organ injuries may not have the reserve needed to tolerate LVAD implantation.
To provide a specific cutoff point above which patients would be considered to be at high risk, an ROC curve was created. As used by previous investigators,13 ROC curves compare the sensitivity and specificity of a screening test and allow determinations of appropriate cutoff levels if they exist. In our model, at a scale score of >4 points, the gain in true-positives is minimal, whereas the false-positive rate triples from 4 to 7. Clinically, if a physician accepts a patient with a scale score of 5 rather than 4 points, an additional approximately 30% chance of dying will be accepted (false-positive increases 30%) in return for saving an additional 10% of patients (true-positive increases 10%). This tradeoff is higher than acceptable; therefore, we chose our cutoff at <5 points on the RFSS. The ROC curve is also useful in validating the LVAD screening scale. In general, a model is believed to accurately predict an outcome if the ROC curve stays far into the top left corner of the graph, demonstrating that the incidence of true-positive values increases rapidly with a much slower increase in false-positive values. Only after the RFSS score increases to a threshold (5 in our study) should the false-positive values increase, revealing that the RFSS score is now too high to differentiate well between false-positive and negative results.
Several investigators previously described their risk factors predicting the failure of mechanical assistance. Reedy et al7 reported 25 factors that they believed to have variable influence over survival after mechanical assistance. Although the small number of LVAD patients in this study (n=14) prevented statistical analysis of risk factors, the authors identified creatinine, urine output, prior heart surgery, and intubation as important criteria in screening patients for device insertion. We were unable to corroborate many of the other factors listed in this study because our study was underpowered to study 25 potential variables.
Farrar et al6 identified blood urea nitrogen and previous operations as risks to survival using univariate analysis. Total bilirubin levels approached statistical significance in this study of 186 patients who had received the Thoratec VAD (Thoratec Laboratories Corp). To achieve the required sample size, data were necessarily pooled from many centers, each of which may have had conflicting views on which criteria are most critical for patient selection. This was a concern raised by the authors and shared by us; we accordingly included only two centers’ results in this report.
Although the number of patients in the study and the completeness of data retrieval allow univariate analysis, additional patients must be included to perform the multivariate analysis required in these complex patients.
A second limitation of the study is the desire to assess only early perioperative mortality risk factors. Most of the deaths occur within 1 week of operation and are a direct result of complications that could have been predicted from preoperative risk factors. A similar relation is more difficult to identify between preoperative criteria and later postoperative death. Additional risk factors such as age and sex of the patient or leukocyte count may play an important role in determining who will be able to return home after these operations; however, our study was statistically underpowered to address risk factors for longer-term survival.
A third limitation is the ability of the clinician to reduce or increase the importance of certain risk factors. For example, only one patient with a temperature of >101.5°F received a device. Interestingly, this patient survived. All other febrile patients were refused device insertion on the assumption that an elevated temperature signaled a septic patient who could not be salvaged with LVAD support. Statistically, however, presence of a temperature does not increase the relative risk of device insertion. This issue could be addressed if more LVADs were implanted into febrile patients, although clinical enthusiasm for such a project may be limited.
The high operative mortality associated with long-term implantable LVAD insertion can be reduced with use of RFSS. The five criteria that were demonstrated to be statistically useful are obtainable at the time of presentation, predictive of successful early outcome, and simple to apply. Physicians could use the scale as a starting point in discussing the suitability of a specific patient and as a basis for comparing patient outcomes.
- Copyright © 1995 by American Heart Association
Medical Devices, Diagnostics and Instrumentation Reports. Chevy Chase, Md: F-D-C Reports, Inc; October 10, 1994.
Mancini DM, Eisen H, Kussmaul W, Mull R, Edmunds LH, Wilson JR. Value of peak exercise oxygen consumption for optimal timing of cardiac transplantation in ambulatory patients with heart failure. Circulation. 1991;83:778-786.
Goldstein DJ, Seldomridge JA, Chen JM, Catanese KA, DeRosa CM, Weinberg AD, Smith CR, Rose EA, Levin HR, Oz MC. Use of aprotinin in LVAD recipients reduces blood loss and associated right heart failure. Ann Thorac Surg.
Reedy JE, Swartz MT, Termuhlen DF, Pennington DG, McBride LR, Miller LW, Ruzevich SA. Bridge to heart transplantation: importance of patient selection. J Heart Tranplant. 1990;9:473-481.
Norman JC, Cooley DA, Igo SR, Hibbs CW, Johnson MD, Bennett JG, Fuqua JM, Trono R, Edmonds CH. Prognostic indices for survival during postcardiotomy intra-aortic balloon pumping: methods of scoring and classification, with implications for LVAD utilization. J Thorac Cardiovasc Surg. 1977;74:709-720.
Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Diagn Radiol. 1982;143:29-36.
Kraemer HC. Assessment of 2×2 association: generalization of signal-detection methodology. Am Stat. 1988;42:37-49.
United Network for Organ Sharing (UNOS). UNOS Update 7; May 1991;2.
O’Connor GT, Plume SK, Olmstead EM, for the Northern New England Cardiovascular Disease Study Group. Multivariate prediction of in-hospital mortality associated with coronary artery bypass graft surgery. Circulation. 1992;85:2110-2118.