Scores for Post–Myocardial Infarction Risk Stratification in the Community
Background— Several scores, most of which were derived from clinical trials, have been proposed for stratifying risk after myocardial infarctions (MIs). Little is known about their generalizability to the community, their respective advantages, and whether the ejection fraction (EF) adds prognostic information to the scores. The purpose of this study is to evaluate the Thrombolysis in Myocardial Infarction (TIMI) and Predicting Risk of Death in Cardiac Disease Tool (PREDICT) scores in a geographically defined MI cohort and determine the incremental value of EF for risk stratification.
Methods and Results— MIs occurring in Olmsted County were validated with the use of standardized criteria and stratified with the ECG into ST-segment elevation (STEMI) and non–ST-segment elevation (NSTEMI) MI. Logistic regression examined the discriminant accuracy of the TIMI and PREDICT scores to predict death and recurrent MI and assessed the incremental value of the EF. After 6.3±4.7 years, survival was similar for the 562 STEMIs and 717 NSTEMIs. The discriminant accuracy of the TIMI score was good in STEMI but only fair in NSTEMI. Across time and end points, irrespective of reperfusion therapy, the discriminant accuracy of the PREDICT score was consistently superior to that of the TIMI scores, largely because PREDICT includes comorbidity; EF provided incremental information over that provided by the scores and comorbidity.
Conclusion— In the community, comorbidity and EF convey important prognostic information and should be included in approaches for stratifying risk after MI.
Received May 6, 2002; revision received August 12, 2002; accepted August 16, 2002.
Several scores have recently been proposed, derived either from clinical trials (eg, the Thrombolysis in Myocardial Infarction [TIMI] score,1,2⇓ the Platelet glycoprotein IIb/IIIa in Unstable angina: Receptor Suppression Using Integrilin Therapy trial [PURSUIT] score,3 the Intravenous nPA Treatment of Infarcting Myocardium Early II [InTIME II] score,4 and the Global Utilization of Streptokinase and t-PA for Occluded coronary arteries [GUSTO] score5) or from registries and cohort studies (eg, the Predicting Risk of Death in Cardiac Disease Tool [PREDICT]6 and the Cooperative Cardiovascular Project scores7) (Table 1). They differ with regard to their derivation population, which likely plays a role in the difference of the components retained in the final score8 and their clinical applicability. Furthermore, the selection process inherent to study participation9 leads to results that often differ from those observed in free-living individuals. Thus, scores derived from clinical trials should be validated in less-selected patients to verify their clinical applicability. This was reported for the TIMI score in ST-segment elevation myocardial infarction (STEMI)10 but not thus far in a community cohort of non–ST-segment elevation myocardial infarction (NSTEMI). Furthermore, the scores have not been directly compared with one another in the same population, which is important for determining their respective value. Finally, although the Cooperative Cardiovascular Project score7 underscored the prognostic value of the ejection fraction (EF) after MI, most scores do not include it. Thus, it is unknown whether adding the EF to these scores would increase their predictive power. Knowledge about the incremental value of the EF in post-MI risk stratification is important because it is inconsistently measured in practice.7,11⇓
See p 2292
The present study was undertaken to address these gaps in knowledge in a rigorously defined, population-based MI cohort by measuring and comparing the discriminant accuracy of the TIMI and PREDICT scores and to determine the incremental prognostic value of the EF above that of the scores. We compared the TIMI with the PREDICT scores because these were derived from different populations and have different components. As shown in Table 1, the GUSTO and PURSUIT scores have similarities with the TIMI scores, whereas the Cooperative Cardiovascular Project (CCP) score shares components with PREDICT.
The study was conducted among the population of Olmsted County, Minn, the characteristics of which are similar to those of United States whites. The Mayo Clinic and Olmsted Medical Center provide medical care for this population. A unified system has accumulated comprehensive clinical records in which information is collected by physicians in a unit record system and is of high quality. These are easily retrievable because the Mayo Clinic maintained extensive indices which, through the Rochester Epidemiology Project (AR-30582), were extended to the records of other care providers to county residents, resulting in the linkage of all medical records from all sources of care through a centralized system.12,13⇓ This provides the ideal setting in which to assess outcomes of rigorously ascertained diseases.12,14⇓
Assembling the MI Incidence Cohort
The cohort of incident MI was assembled with the use of standardized surveillance methods.15,16⇓ Briefly, all cases discharged from Olmsted County hospitals between 1983 and 1994 with diagnoses compatible with an MI16 were reviewed by abstractors who verified residency in Olmsted County and collected clinical data. ECGs were assigned a Minnesota code by the ECG Reading Center at the University of Minnesota.17 Standard epidemiological criteria were applied to assign MI diagnosis on the basis of cardiac pain, creatine kinase (CK), and Minnesota code. The relevant quality-control data were detailed elsewhere.15
Baseline Characteristics, End Point Definitions, and Ascertainment
Because of the change over time in normal values, peak CK ratio was defined as the ratio of the maximum CK value to the upper limit of normal. ST-segment elevation was defined by the Minnesota code.
Comorbidity was measured by the Charlson index.18 This validated index, based on 17 disease categories, was incorporated into the analysis with the following approach: No comorbidity equates to 0 Charlson points; moderate comorbidity, 1 point; severe comorbidity, 2 points; and very severe comorbidity, ≥3 Charlson points.
Measurement of EF with the use of radionuclide angiography, echocardiography, or cardiac catheterization within 28 days after the index MI was ascertained from the medical records. When more than one method was used, one measurement was selected according to the following hierarchy: echocardiography; if not available, catheterization; if not available, then radionuclide ventriculography. EF was categorized as <30%, 30% to 49%, and ≥50%. Reperfusion therapy was defined as thrombolysis or coronary angioplasty within 24 hours after admission.
Follow-up, accrued through community (inpatient and outpatient) medical records, is quite complete, inasmuch as >90% of the county residents receive primary care at Mayo Clinic or Olmsted Medical Center, and nearly all are seen at the Mayo Clinic every 3 years.12 Recurrent MIs were defined clinically. The ascertainment of deaths, on the basis of a comparison of census estimates of the population and unique registration numbers of county residents at Mayo Clinic, includes several procedures. First, all death certificates for Olmsted County residents are obtained every year from the County office. Second, the Mayo Clinic registration office monitors the obituaries in the local newspapers and incorporates notices of deaths in the record. Finally, death tapes from the State of Minnesota are obtained every 2 to 3 years. The population of Olmsted County is stable, with low out-migration rates, and net in-migration to the County. Health status has little influence on migration, except among the very old who move into Rochester from surrounding areas for nursing home admission.
The Risk Scores
These were calculated with the use of the published components (Table 1). For STEMI, the TIMI risk score is a weighted integer score based on 8 clinical indicators that can be easily assessed on admission. The score is the sum of the points for each risk indicator present (range, 0 to 14).2 It was reported for the prediction of death at 30 days. For NSTEMI, the score is based on clinical indicators, each assigned one point and ranging from 0 to 7.1 It was reported to predict death, recurrent MI, or urgent revascularization at 14 days.1,2⇓ PREDICT, derived from a geographically defined cohort of acute coronary syndromes,6 includes 7 clinical factors measured on the day of admission. It assigns integer points, added across components, leading to values ranging from 0 to 24. It was reported to predict death at 30 days, 2 years, and 6 years.
The data are presented as frequency or mean±SD. Characteristics were compared across groups with χ2 tests for categorical variables and t tests for continuous variables. Survival was analyzed with the Kaplan-Meier method.
The prognostic discriminatory capacity of the TIMI and PREDICT scores was measured and compared with the c-statistic representing the area under the receiver operating characteristic curves for prediction of death and death or recurrent MI at 28 days and at one year. A c-statistic >0.75 is considered to have good discriminant ability. To evaluate goodness of fit, we used logistic regression of the end points on the scores and reported the Hosmer-Lemeshow χ2 statistics for goodness of fit, in which a high-probability value corresponds to good fit.19 To determine what component of the scores increased the predictive accuracy of one score over the other and the incremental value of the EF above that of the most discriminant score, logistic regression models, constructed while adding the various components of the scores, were compared with the use of c-statistics. Missing values did not exceed 5% for the variables used in the prediction scores, except for the EF, which was not measured in 27% of the cases. For EF, an indicator variable reflecting missing values was included in models analyzing the incremental contribution of EF over the scores.7 Ancillary analyses compared this approach to the complete case analysis and to imputation.
Analyses were performed with the use of SAS, version 8.0 (SAS Institute Inc). The Mayo Foundation Institutional Review Board approved this study.
Between 1983 and 1994, 1296 persons had an incident MI in Olmsted County. Of these, 1279 had ECG data available and were included in the analysis. Among these, 562 (44%) had STEMI and 717 (56%) NSTEMI. Their baseline characteristics are shown in Table 2. No difference in the distribution of age or cardiovascular risk factors was detected between STEMI and NSTEMI. The systolic blood pressure and heart rate were lower and peak CK ratio was higher among patients with STEMI. Patients with NSTEMI had more comorbidity.
After a follow-up of 6.3±4.7 years, 304 (54%) patients with STEMI and 372 (52%) patients with NSTEMI died, and 237 nonfatal recurrent MIs were observed, 99 among STEMI and 136 among NSTEMI. Figure 1 shows the Kaplan-Meier survival curves of time to death and time to death or recurrent MI, which did not differ by ST-elevation status (P=0.95 and 0.88, respectively). The overall survival was 0.88 (95% CI 0.86 to 0.91) at 30 days and 0.80 (95% CI 0.78 to 0.84) at one year. Survival free of death or recurrent MI was 0.85 (95% CI 0.83 to 0.88) at 30 days and 0.76 (95% CI 0.73 to 0.79) at one year.
Discriminant Accuracy of the TIMI and the PREDICT Scores
The TIMI Scores
The distribution of the TIMI scores for STEMI and NSTEMI are shown in Figure 2. For STEMI, the distribution of scores was skewed to the right with a median value of 5 (range 0 to 14, first quartile 3, third quartile 7). Thirteen of 14 possible values were represented. For NSTEMI, 5 of 7 possible values were represented. The median value was 2 (range 0 to 5, first quartile 2, third quartile 3).
To predict death after STEMI, the TIMI score provided good fit with the data (P for Hosmer-Lemeshow χ2=0.92 for death at 28 days and 0.44 at 1 year). For NSTEMI, the TIMI score provided good fit with the data for death at 28 days (P=0.51 for Hosmer-Lemeshow χ2). For death at 1 year, the probability value of the Hosmer-Lemeshow χ2 was 0.11, suggesting an adequate but not optimal fit. With regard to STEMI, the TIMI score showed a strong discriminant accuracy to predict death at 28 days (c=0.73), which was qualitatively similar to that measured in the InTime II trial (c=0.78),2 and which remained robust at one year (c=0.73) (Table 3). In NSTEMI, the discriminant accuracy of the TIMI score to predict death at 28 days was not as strong (c=0.59) and was qualitatively lower than that reported in the TIMI trial, in which the c-statistic was 0.74.1 The discriminant accuracy to predict death at one year was similar to that observed at 28 days (c=0.61). When recurrent MI was combined to death as an end point, the TIMI scores maintained a similar predictive accuracy.
When the analyses were stratified by reperfusion therapy, no difference in the discriminant accuracy of TIMI scores by reperfusion was detected for NSTEMI and STEMI.
The PREDICT Score
As shown in Figure 3, the median value of the PREDICT score was 9 (range 0 to 24, first quartile 5, third quartile 14) and similar for STEMI and NSTEMI (P=0.72). For STEMI, PREDICT provided good fit with the data for the prediction of death at 28 days (probability value for Hosmer-Lemeshow χ2 0.58) and at 1 year (probability value for Hosmer-Lemeshow χ2 0.89).
Similarly, for NSTEMI, PREDICT scores provided good fit with the data for the prediction of death at 28 days (probability value for Hosmer-Lemeshow χ2 0.36) and at 1 year (probability value for Hosmer-Lemeshow χ2 0.60).
With regard to discriminant accuracy (Table 3), for STEMI and NSTEMI, PREDICT showed a discriminant accuracy to predict death at 30 days, which was strong and remained robust over time and across end points. When the analyses were stratified by reperfusion therapy, no difference in the discriminant accuracy of PREDICT by reperfusion was detected.
Comparison of TIMI and PREDICT
The discriminant accuracy of PREDICT was consistently superior to that of the TIMI scores across time and end points (Table 3). One of the differences between the TIMI and PREDICT scores resides in the inclusion of comorbidity in PREDICT (Table 1). To formally test the hypothesis that adding comorbidity increases the predictive value of the TIMI score, logistic regression analyses assessed the improved discriminant accuracy related to the addition of the Charlson index. The c-statistics for the models including the TIMI scores alone and complemented by the Charlson index were compared with those for the models including PREDICT (Table 3). The results indicated a better discriminant accuracy for the model including the TIMI scores complemented by the Charlson index over the model including only the TIMI scores. PREDICT, however, had greater discriminant accuracy than the other two models.
Incremental Value of the EF for Risk Stratification
Adding EF to the models examining the association of the PREDICT score with outcomes improved the discriminant accuracy of PREDICT (Table 4).
In this community-based cohort, no difference in outcome was observed between STEMI and NSTEMI. The discriminant accuracy of the TIMI score was good for STEMI but only fair for NSTEMI; that of PREDICT was consistently superior to that of the TIMI scores across time and end points and irrespective of reperfusion status, largely because PREDICT includes comorbidity. The EF added incremental prognostic information over both scoring systems.
Outcome of STEMI Versus NSTEMI
Our results show similar outcomes for STEMI and NSTEMI, which is at odds with data from GUSTO-IIb, in which STEMI had a greater mortality rate at 30 days, and from data from the Worcester study, which indicated a higher risk of in-hospital death for Q-wave MI as compared with non–Q-wave MI.20,21⇓
The characteristics of this community-based cohort differ from those of trial patients in such a way that can influence the outcome. The reasons for the differences between the Worcester study and ours can only be speculative, yet it is important to underscore that our study examines 30-day mortality, which may differ from in-hospital mortality, and that one cannot directly equate ST elevation to the development of Q waves.
Risk Stratification Scores in the Community
The TIMI Scores
The score for STEMI was derived from a trial of fibrinolytic eligible patients. In the present community cohort that includes fibrinolytic-ineligible patients, the score retained good discriminant accuracy in predicting early outcomes that remained robust for late events. These data support and extend the results reported by Morrow et al10 by indicating that the discriminant accuracy of the TIMI score in STEMI did not differ according to reperfusion status. With regard to NSTEMI, the discriminant accuracy of the TIMI score was only fair in our cohort. Several points can explain these discrepancies. First, the study populations differ. In addition to the expected difference in baseline characteristics between community studies and clinical trial populations, all persons in the present cohort experienced a MI, whereas only one third of the TIMI 11B trial population experienced a MI. From the GUSTO II-b data, one would expect worse outcomes after NSTEMI as opposed to a group including predominantly unstable angina.21 This may in turn lead to greater discriminant accuracy of the score in populations combining NSTEMI and unstable angina who experience more diverse outcomes.
Second, our study did not include severe recurrent ischemia requiring revascularization as an end point, which may also affect the discriminant accuracy of the score.
Notwithstanding these considerations, the present data indicate that the TIMI score does not provide optimal prognostic information for NSTEMI in the community.
The PREDICT Score
In contrast to the TIMI scores, PREDICT was developed in a population-based cohort assembled with methodology similar to the one used in this study.6 Its greater discriminant accuracy indicates that it conveys information better suited than that conveyed by the TIMI scores to stratify risk in the community. This enhanced ability is largely related to the inclusion of comorbidity lacking from the TIMI scores. Given the inherent selection process related to the inclusion in clinical trials, participants have less comorbidity than nonparticipants do9; thus, the scores developed in trials are influenced by the selection process inherent to trials such that comorbidity is not integrated in such systems.
Role of Comorbidity and EF: Practical Implications
The importance of comorbidity to predict post-MI outcomes was recently underscored by the report of the Cardiovascular Cooperative Project, which focuses on Medicare beneficiaries.7 As the burden of coronary disease shifts toward older age groups, MI is increasingly becoming a disease of the elderly,15 such that the impact of comorbidity on outcome should be expected. Comorbidity, however, is not included in any of the scores derived from clinical trials, which typically include younger persons.2
The EF provides information not conveyed by the PREDICT score, thereby improving risk stratification. All scores, including PREDICT, focus on data acquired early on admission. Our results indicate that this early risk stratification approach is robust across time and end points but should be combined to the measurement of EF after MI for optimal risk stratification. This has important clinical implications, inasmuch as EF is not measured in practice in all cases after MI.7,11,22⇓⇓
These results should not be interpreted as advocating for one particular score, as the TIMI scores have the distinct advantage of their ease for bedside use. Rather, they underscore the complexity of risk stratification and the limitations of extrapolating data acquired in trial populations to clinical practice. To this end, comorbidity and EF should be integrated to post-MI risk stratification.
Potential limitations of the study include the racial and ethnic composition of Olmsted County, which limits the generalization of these data to groups underrepresented in the population. Although no single community can completely represent the nation as a whole, studies of chronic diseases in Olmsted County indicate that results from the county can be extrapolated to a large part of the population. However, because an unexpected association between blacks and the outcome of STEMI was recently reported, this study should be replicated in other ethnic groups.5
In part because of the period analyzed, the use of reperfusion likely increased more recently; however, our findings are robust irrespective of the use of reperfusion. Although approaches to missing values are challenging, however, ancillary analysis yielded similar results for the indicator variable approach, imputation of missing EF values, and complete case analysis.
In this community-based MI cohort, patients with STEMI and NSTEMI experienced similar outcomes. The discriminant accuracy of the TIMI score was good for STEMI but only fair for NSTEMI. That of the PREDICT score was superior to that of the TIMI scores across time, largely because PREDICT includes comorbidity. The EF added incremental prognostic information over both scores. This underscores the importance of comorbidity and left ventricular function for effective risk stratification in acute MI.
Supported in part by grants from the Public Health Service and the National Institutes of Health (AR30582 and RO1 HL 59205). Dr Roger is an Established Investigator of the American Heart Association. We also thank Kay Traverse, RN, and Susan Stotz, RN, for their assistance with data collection; Ryan Meverden for assistance with statistical analysis; and Kristie Shorter for secretarial assistance.
- ↵Morrow DA, Antman EM, Charlesworth A, et al. TIMI risk score for ST-elevation myocardial infarction: a convenient, bedside, clinical score for risk assessment at presentation: an intravenous nPA for treatment of infarcting myocardium early II trial substudy. Circulation. 2000; 102: 2031–2037.
- ↵Boersma E, Pieper KS, Steyerberg EW, et al. Predictors of outcome in patients with acute coronary syndromes without persistent ST-segment elevation: results from an international trial of 9461 patients. The PURSUIT Investigators. Circulation. 2000; 101: 2557–2567.
- ↵Califf RM, Pieper KS, Lee KL, et al. Prediction of 1-year survival after thrombolysis for acute myocardial infarction in the global utilization of streptokinase and TPA for occluded coronary arteries trial. Circulation. 2000; 101: 2231–2238.
- ↵Jacobs DR Jr, Kroenke C, Crow R, et al. PREDICT: a simple risk score for clinical severity and long-term prognosis after hospitalization for acute myocardial infarction or unstable angina: the Minnesota Heart Survey. Circulation. 1999; 100: 599–607.
- ↵Kurland LT, Elveback LR, Nobrega FT. Population studies in Rochester and Olmsted County, Minnesota, 1900–1968.In: Kessler IT, Levin ML, eds. The Community as an Epidemiologic Laboratory; A Casebook of Community Studies. Baltimore, Md: Johns Hopkins Press; 1970: 47–70.
- ↵Prineas R, Crow R, Blackburn H. The Minnesota Code Manual of Electrocardiographic Findings. Littleton, Mass: John Wright-PSG, Inc; 1982.
- ↵Lemeshow S, Hosmer DW Jr. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol. 1982; 115: 92–106.
- ↵Furman MI, Dauerman HL, Goldberg RJ, et al. Twenty-two year (1975 to 1997) trends in the incidence, in-hospital and long-term case fatality rates from initial Q-wave and non-Q-wave myocardial infarction: a multi-hospital, community-wide perspective. J Am Coll Cardiol. 2001; 37: 1571–1580.
- ↵Armstrong PW, Fu Y, Chang WC, et al. Acute coronary syndromes in the GUSTO-IIb trial: prognostic insights and impact of recurrent ischemia. The GUSTO-IIb Investigators. Circulation. 1998; 98: 1860–1868.
- ↵AHA/ACC conference proceedings: measuring and improving quality of care: a report from the American Heart Association/American College of Cardiology First Scientific Forum on Assessment of Healthcare Quality in Cardiovascular Disease and Stroke. Circulation. 2000; 101: 1483–1493.