Simple Scoring Scheme for Calculating the Risk of Acute Coronary Events Based on the 10-Year Follow-Up of the Prospective Cardiovascular Münster (PROCAM) Study
Background— The absolute risk of an acute coronary event depends on the totality of risk factors exhibited by an individual, the so-called global risk profile. Although several scoring schemes have been suggested to calculate this profile, many omit information on important variables such as family history of coronary heart disease or LDL cholesterol.
Methods and Results— Based on 325 acute coronary events occurring within 10 years of follow-up among 5389 men 35 to 65 years of age at recruitment into the Prospective Cardiovascular Münster (PROCAM) study, we developed a Cox proportional hazards model using the following 8 independent risk variables, ranked in order of importance: age, LDL cholesterol, smoking, HDL cholesterol, systolic blood pressure, family history of premature myocardial infarction, diabetes mellitus, and triglycerides. We then derived a simple point scoring system based on the β-coefficients of this model. The accuracy of this point scoring scheme was comparable to coronary event prediction when the continuous variables themselves were used. The scoring system accurately predicted observed coronary events with an area under the receiver-operating characteristics curve of 82.4% compared with 82.9% for the Cox model with continuous variables.
Conclusions— Our scoring system is a simple and accurate way of predicting global risk of myocardial infarction in clinical practice and will therefore allow more accurate targeting of preventive therapy.
Received August 22, 2001; revision received November 7, 2001; accepted November 8, 2001.
The risk of developing coronary heart disease (CHD) depends on several determinants—some associated with lifestyle—operating from early childhood on. Many of these determinants interact in nonlinear fashion, complicating assessment in the individual patient. Prediction rules developed through the use of clinical judgment include a large subjective component and are therefore difficult to standardize.1,2⇓ Statistical procedures such as stepwise logistic regression or the Cox proportional hazards model, which have been implemented in large prospective epidemiological studies such as the Framingham study in the United States3 and the Prospective Cardiovascular Münster (PROCAM) study in Europe4 provide a truer picture of a person’s overall or global risk of CHD than clinical classification but may prove cumbersome in clinical practice. For this reason, several groups have developed more simple point scoring schemes that take several variables into account.5,6⇓ However, these scoring schemes may not reflect either differences in the absolute contribution to risk of individual factors or the nonlinear progression of risk that may exist within a risk factor category. These drawbacks may limit the predictive power of such schemes. Recently, a more differentiated scoring scheme has been suggested based on the results of the Framingham study.7 However, in contrast to PROCAM, this scoring scheme used total cholesterol for 10-year risk assessment because of a larger and more robust Framingham database for total than for LDL cholesterol. Triglycerides and family history of premature CHD were also not used in generating this Framingham risk score. We have now designed an improved point-scoring scheme based on the newly completed 10-year follow-up of the cohort of middle-aged men in PROCAM. Use of this scoring scheme resulted in very little loss of information compared with the Cox proportional hazards models, using the continuous variables themselves, and was associated, within the PROCAM population, with greater predictive power than similar schemes that have been proposed by others.6,7⇓
Recruitment of Participants
Recruitment to the PROCAM study8,9⇓ was started in 1979 and completed in 1985. During that time, 20 060 employees of 52 companies and local government authorities were examined. Details of the examination procedure are reported elsewhere.4 Follow-up was by questionnaire every 2 years, with a response rate of 96%. In each case in which evidence of morbidity or mortality was entered in the questionnaire, hospital records and records of the attending physician were obtained, and, in the case of deceased study participants, an eyewitness account of death was sought. A “major coronary event” was defined as the occurrence of sudden cardiac death or a definite fatal or nonfatal myocardial infarction on the basis of ECG and/or cardiac enzyme changes. Participants were excluded from follow-up if at the time of recruitment they had a history of either myocardial infarction or stroke or if the ECG at recruitment showed signs of ischemic heart disease. Patients with a history of angina pectoris at recruitment, as defined by the Rose questionnaire,10 were excluded from the present analysis.
In the cohort of 5389 men (35 to 65 years of age) recruited before the end of 1985, 325 major coronary events occurred within 10 years, enough to allow statistically valid longitudinal analysis. Within the 10 years, 230 men were lost to follow-up, 14 had suspected coronary death, 218 died from other causes, and 46 nonfatal cases of stroke occurred. In addition, in 63 men who did not have an acute coronary event, CHD was diagnosed by angiography. All of these subjects were included in the population at risk until such time as death, loss to follow-up, or censoring occurred. Thus, 4493 middle-aged men survived 10 years without a major coronary event and without an event causing censoring.
Cox Proportional Hazards Model
To extract the maximum information from our study population, we constructed our risk algorithm using the Cox proportional hazards model. This model has several advantages over other techniques, such as logistic regression: It accounts for variable duration of follow-up, censoring of subjects, proportionality of event occurrence, and time-to-event.11,12⇓ Strictly speaking, the Cox model only allows calculation of relative risk. Therefore, to convert the results of the Cox model into absolute risk estimates, we calculated survival within our population by using Kaplan-Meier statistics, as previously described by Wilson and colleagues.6
Generation of Point Scoring Scheme
To increase the usefulness of the risk algorithm in clinical practice, we simulated it by means of a simple scoring scheme. Each risk factor was divided into the categories shown in Table 3. A regression equation was then calculated between the logarithm of global risk as calculated by use of the Cox model in conjuction with the survival curves and the categories of each risk factor. The coefficients calculated in this manner were then standardized and rounded such that only whole numbers remained.
Calibration measures how closely predicted outcomes agree with actual outcomes. For this, we used a version of the Hosmer-Lemeshow χ2 statistic as described by d’Agostino.13
Kaplan-Meier Survival Curve
To generate absolute risk estimates, a Kaplan-Meier survival curve was generated. Table 1 shows the time-to-event in steps of 1 year, the number entering each time interval, the number of participants withdrawn during each interval, the number of terminal events, the cumulative proportion surviving in each interval, and the percentage of incidences occurring within each interval (the hazard rate). The cumulative event-free survival after 10 years in this cohort of men in PROCAM was 93.7%
Construction of Cox Model
Of the 57 clinical and laboratory variables measured in the PROCAM study,14 8 were found to be independently predictive of event risk and were used to construct the risk algorithm. These variables, together with the β-coefficients of the Cox model, the hazard ratios, and the 95% confidence intervals, are shown in Table 2. This table also shows the mean levels of these risk variables in the total population. In the scoring of the variable “family history,” those men whose first-degree relatives were all younger than 60 years of age were coded as “negative family history” in both cases, providing no coronary events had occurred among those family members. Those men whose first-degree relatives died before age 60 years from a noncoronary cause such as a traffic accident were also classified as “negative family history.” In generating our algorithm, we also considered first-order interactions between these independent risk variables; however, none of these interactions exceeded the significance threshold of 0.05. To validate the Cox model, we divided our data into five equal and distinct sets. Combinations of four of these five sets were used for generating the model. The final set was used for testing performance of the model on unknown data. This cross-validation procedure was repeated for every possible 4+1 combination. This internal validation showed that the Cox model performed robustly with results in each subset that did not differ significantly from the model derived in the entire data set (not shown).
The PROCAM scoring system is shown in Table 3. The categories for the continuous variables in the PROCAM score were based, in the case of age, blood pressure, LDL cholesterol, and HDL cholesterol, on the National Cholesterol Education Program (NCEP) III guidelines,7 and in the case of triglycerides on the guidelines of the International Task Force for Prevention of CHD.15
The risk of a coronary event associated with each score and as calculated using the full PROCAM algorithm is shown in Table 4; at very low and very high PROCAM scores, incidences and case numbers were too small for individual scores to be meaningful; these scores are therefore grouped into two categories of very low and very high scores, respectively.
Comparison of Cox Model With Scoring Scheme
We examined if the risk of an acute coronary event as calculated with our scoring scheme agreed with the risk of an event as calculated with the Cox risk algorithm. To do this, we plotted the risk of an acute event in 10 years as calculated with the Cox model for each member of the cohort of middle-aged men in PROCAM against the PROCAM score (Figure 1).
We then compared the performance of our scoring system with that of the Cox model in calculating the relative risk of an acute coronary event using receiver-operating characteristics (ROC) curve analysis (Figure 2). Such ROC curves measure the discrimination of a prediction model, which is its ability to separate those who have hard CHD events from those who do not. Censored data (ie, individuals who did not complete the 10 years of follow-up without having a coronary event) were excluded from this analysis. Although the area under the ROC curve obtained by means of the Cox function was 82.9%, the area under the ROC curve obtained with the PROCAM scoring scheme did not differ significantly, at 82.4% (P=0.251), indicating that the ability of the PROCAM scoring scheme to predict the relative risk of an acute coronary event was equivalent to that of the full PROCAM Cox algorithm.
Calibration of Score
Finally, we compared the estimated risk of an acute coronary event calculated by use of the PROCAM score with the event rate that was actually observed for each score category (Table 5). As is clear from the table, the estimated risk showed very good agreement with the observed incidence (Hosmer-Lemeshow χ2=6.5, P>0.3). The regression equation between weighted score categories was observed incidence=0.983 · estimated risk−0.0007 (% in 10 years).
In recent years in Europe, a consensus has emerged that those patients with an absolute risk of a coronary event exceeding 20% in 10 years should be regarded as “high risk” and should receive special attention.16 This lead has recently been followed by the NCEP in the United States.7 As can be seen from Table 5, 7.5% of the population of middle-aged men in PROCAM fell into this category of high risk, the cut-off for which was a PROCAM score of >53. Thus the PROCAM score was able to confine the definition of high risk to a relatively small group of middle-aged men.
In this report, we present a relatively simple point-scoring scheme for calculating the risk of an acute coronary event (fatal or nonfatal myocardial infarction or acute coronary death). These scores were derived from a Cox proportional hazards model calculated from 10 years of follow-up of the cohort of middle-aged men in the PROCAM study. This scoring scheme contained nearly all the information present within the Cox function with continuous variables and performed very well in calculating the absolute risk of an acute coronary event.
An important feature of the PROCAM study is that it included only the “hard” end points of definite myocardial infarction or sudden coronary death. This definition was consciously chosen because it was considered unlikely that these end points would be overlooked or misdiagnosed in our participants. Inclusion of “new coronary heart disease” based on angiography or other criteria would have been problematic; how could we have been sure that we would not have failed to register men with new coronary heart disease in whom angiography or other diagnostic tests were not performed? For comparison purposes, however, we calculated our score including the 63 men in whom CHD was diagnosed by angiography or other means and the 14 “suspected coronary deaths.” The score for each risk factor was virtually identical to that calculated excluding these 77 men (data not shown). Because of the larger number of end points, however, the absolute risk associated with each point score was higher.
Recently, a point score similar to ours was developed on the basis of the data of the Framingham study and presented in the Third Report of the NCEP Expert Panel on Cholesterol.7 This score replaces a simpler scheme that was developed by Wilson and colleagues in 1998.6 We investigated the ability of the Framingham score to determine the relative risk of an acute coronary event by means of ROC curve analysis (Figure 2). The area under the ROC curve derived by use of the Framingham score (77.8%) was significantly less than that achieved with either the PROCAM Cox model (82.9%) or the PROCAM score (82.4%, P<0.001 for both comparisons). There are, however, substantial differences between the Framingham and the PROCAM data sets. In particular, the latest Framingham score does not include information on family history of CHD, triglycerides, or LDL cholesterol. The Framingham prediction function systematically overestimated risk in the PROCAM cohort. The Hosmer-Lemeshow χ2 statistics for evaluation of the calibration of the Framingham score applied to the PROCAM cohort was 43.8, corresponding to a probability value of <0.001.
In addition, it is not surprising that the PROCAM score performs better in the data set in which it was derived and optimized. True validation of both the Framingham and PROCAM scoring systems would require their application to a third independent data set.
Preliminary analysis of the 10-year follow up of women 45 to 65 years of age at entry into PROCAM indicates a 4-fold lesser absolute risk of coronary events in women compared with men of the same age (data not shown), whereas the score proposed by the latest Framingham score suggests that the difference in risk between the sexes is only 2-fold. However, as noted above, true comparison of Framingham and PROCAM scoring systems would require their application to a third data set.
The scoring system proposed by us is a simple and accurate means of predicting risk of myocardial infarction in clinical practice. Our scheme classified as high risk (ie, >20% risk of an acute coronary event in 10 years) a group of men comprising only 7.5% of the total cohort of middle-aged men in PROCAM. Thus, in addition to improving CHD prevention generally, use of the scoring scheme proposed by us may allow more accurate targeting of lipid-lowering therapy and considerable cost savings.
- ↵Anderson KM, Wilson PWF, Odell PM, et al. An updated coronary risk profile: a statement for health professionals. Circulation. 1991; 83: 356–362.
- ↵Wilson PWF, d’Agostino RB, Levy D, et al. Prediction of coronary heart disease using risk factors categories. Circulation. 1998; 97: 1837–1847.
- ↵Assmann G, Cullen P, Schulte H. The Munster Heart Study (PROCAM): results of follow-up at 8 years. Eur Heart J. 1998; 19: A2–A11.
- ↵Cullen P, Schulte H, Assmann G. The Münster Heart Study (PROCAM): total mortality in middle-aged men is increased at low total and LDL cholesterol concentrations in smokers but not in nonsmokers. Circulation. 1997; 96: 2128–2136.
- ↵World Health Organisation. Cardiovascular Survey Methods. Geneva: 1997.
- ↵Cox DR. Regression models and life tables. J R Stat Soc. 1972; 34: 187–220.
- ↵Assmann G, Schulte H. Results and conclusions of the Prospective Cardiovascular Münster (PROCAM) Study.In: Assmann G, ed. Lipid Metabolism Disorders and Coronary Heart Disease. München: MMV Medizin Verlag; 1993: 19–68.
- ↵Assmann G, Carmena R, Cullen P, et al, for the International Task Force for Prevention of Coronary Heart Disease. Coronary heart disease: reducing the risk: the scientific background for the primary and secondary prevention of coronary heart disease: a worldwide view. Nutr Metab Cardiovasc Dis. 1998; 8: 205–271.
- ↵Wood D, De Backer G, Faergeman O, et al. Prevention of coronary heart disease in clinical practice: recommendations of the second joint Task Force of European and other Societies on Coronary Prevention. Eur Heart J. 1998; 19: 1434–1503.