Continuous Probabilistic Prediction of Angiographically Significant Coronary Artery Disease Using Electron Beam Tomography
Background— We sought to incorporate electron beam tomography–derived calcium scores in a model for prediction of angiographically significant coronary artery disease (CAD). Such a model could greatly facilitate clinical triage in symptomatic patients with no known CAD.
Methods and Results— We examined 1851 patients with suspected CAD who underwent coronary angiography for clinical indications. An electron beam tomographic scan was performed in all patients. Total per-patient calcium scores and separate scores for the major coronary arteries were added to logistic regression models to calculate a posterior probability of the severity and extent of angiographic disease. These models were designed to be continuous, adjusted for age and sex, corrected for verification bias, and independently validated in terms of their incremental diagnostic accuracy. The overall sensitivity was 95%, and specificity was 66% for coronary calcium to predict obstructive disease on angiography. With calcium scores >20, >80, and >100, the sensitivity to predict stenosis decreased to 90%, 79%, and 76%, whereas the specificity increased to 58%, 72%, and 75%, respectively. The logistic regression model exhibited excellent discrimination (receiver operating characteristic curve area, 0.842±0.023) and calibration (χ2 goodness of fit, 8.95; P=0.442).
Conclusions— Electron beam tomographic calcium scanning provides incremental and independent power in predicting the severity and extent of angiographically significant CAD in symptomatic patients, in conjunction with pretest probability of disease. This algorithm is most useful when applied to an intermediate-risk population.
Received November 9, 2001; revision received February 1, 2002; accepted February 4, 2002.
Coronary artery calcification (CAC) scores derived from electron beam tomography (EBT) are predictive of subsequent cardiac events in both symptomatic1,2⇓ and asymptomatic populations.3–5⇓⇓ As opposed to other noninvasive modalities to diagnose coronary artery disease (CAD) focusing on physiological consequences of obstruction, EBT coronary calcium represents an anatomic measure of plaque burden.6 Studies comparing pathological and EBT findings have demonstrated that the degree of luminal narrowing is weakly correlated with the amount of calcification on a site-by-site basis,7 whereas total calcium score is more closely associated with the presence and severity of maximum angiographic stenosis.8–10⇓⇓ These studies have a substantial degree of verification bias as a consequence of the preferential referral of positive test responders to angiography and of negative test responders away from angiography.8
We sought to overcome these difficulties and thereby construct and validate a clinically relevant model to predict the likelihood of angiographically significant CAD based on the magnitude and distribution of calcification within each region of the coronary circulation. The rationale for this study was to create a new model based on a much larger data set than previous models, with certain design criteria. The a priori criteria for the design of this model were that it be (1) continuous, (2) adjusted for age and sex, (3) corrected for the effects of verification bias, (4) independently validated, and (5) capable of being integrated with additional diagnostic information.
The EBT and angiographic studies were conducted at Harbor-UCLA Medical Center, Torrance, Calif; EBT Research Foundation, Nashville, Tenn; and Saint Francis Hospital, Roslyn, New York, and approved by the respective Institutional Review Boards. Investigators recorded patient information using a case-report form that included biographical data including age, sex, dates, and results of EBT and angiographic studies. The data were forwarded to the coordinating and data elaboration center at Cedars-Sinai Medical Center in Los Angeles, Calif.
The patient population consisted of 1851 patients (1169 men, 682 women; mean age, 58±11 years; range, 21 to 86) undergoing coronary angiography for clinical indications. Nonconsecutive patients were enrolled from January 1996 to September 1999. After signing informed consent, EBT scanning was performed to assess presence and estimate extent of CAC. EBT was performed within 2 weeks of angiography in 92% of patients. Exclusion criteria included EBT scans performed >3 months from the angiogram and patients who had previous coronary interventional procedures or known CAD. The vast majority (>90%) of patients underwent angiography before EBT (81%) or were already scheduled to undergo angiography before EBT (10%). No patient was included in the previous multicenter publication.8
To adjust for verification bias, we evaluated 4103 asymptomatic persons referred for CAC assessment to measure cardiovascular risk. Persons were asymptomatic, with no history of cardiovascular disease.
All patients underwent coronary angiography and were referred on the basis of the primary physician’s concern for presence of myocardial ischemia. Positive noninvasive stress testing, abnormal echocardiograms, and clinical history were all used for referring patients for cardiac catheterization. The angiograms were analyzed by an experienced reader at each institution, blinded to the results of the EBT coronary scan. Each epicardial coronary vessel was assessed, and the visual estimation of the percent luminal stenosis of each lesion was reported as previously described.8 Angiographic abnormalities were considered significant if ≥50% luminal diameter stenosis was found in any vessel.
Electron Beam Tomography
The EBT studies were performed with an Imatron C-150 XL ultrafast CT scanner in the high-resolution volume mode, with a 100-ms exposure time. ECG triggering was used, corresponding to 80% of the R-R interval for standardized calcium scoring.11 Thirty to 40 consecutive, noncontrast images were obtained at 3-mm intervals to include the entire coronary tree. Total radiation exposure with the use of this technique was 0.6 Rad per patient, significantly less than multislice spiral CT (estimated between 2.8 to 10 Rad12,13⇓) or conventional angiography(4 to 8 Rad14).
The scans were scored by means of the algorithm developed by Agatston et al11 by a cardiologist blinded to the clinical, ECG, and angiographic information.
Multivariate Logistic Prediction Model
The baseline data set of 1851 patients was divided into two samples by a random number generator. The training sample of 932 patients was used for generation of 4 different logistic prediction models. The pretest model (1) was based on age, age squared, and sex. The test model (2) was based on the square root of each vessel-specific EBT score (left main, left anterior descending, left circumflex, and right coronary artery). The combined model (3) was based on sex, the 4 vessel-specific calcium scores, and 2 age-dependent interaction terms: age times square root of the total calcium score (when total calcium score >0) or age alone (when total calcium score=0). The adjusted model (4) used the same variables as the combined model but was corrected for verification bias15,16⇓ by use of the Harbor-UCLA data on the total population of 4982 patients referred for testing (879 referred for angiographic verification and 4103 who were not). The resultant prediction model was used to estimate the posterior or post-test probability of angiographically significant CAD in each of these 932 patients from which the model was derived (training sample) as well as in the independent 919 patients (validation sample).
All values are reported as mean±SD. Categorical data were analyzed by means of χ2 analysis and Fisher’s exact test. A χ2 goodness-of-fit statistic was used to assess the calibration of each model relative to the line of identity (the agreement between observed and expected prevalence of disease in each decile of predicted probability; a χ2 of zero indicating perfect calibration). A receiver operating characteristic (ROC) curve was created from each model to determine its discrimination (the ability to distinguish between those with and those without disease; a value of 0.5 indicating chance discrimination and a value of 1 indicating perfect discrimination).15
Statistical comparisons among the four models were made by computing the ratio of the χ2 statistics, which were then tested with of an F test for the appropriate degrees of freedom (1, the number of nonzero deciles) in the numerator and denominator. All tests of significance were 2-tailed, and significance was defined at the ≤0.05 level.
Electron Beam Tomography
Of the 1851 symptomatic patients, 1466 (79%) had a total calcium score >0. Total calcium scores ranged from 1 to 6649. The mean and median scores were 380 and 116, and 25th and 75th percentiles were 3 and 456, respectively. Overall sensitivity was 96%, and specificity was 40% for any CAC to predict obstructive angiographic disease. With calcium scores >20, >80, and >100, the sensitivity to predict stenosis decreased from 90% to 79% and 76%, respectively; the specificity increased from 58% to 72% and 75%, respectively.
The mean square root of the score in men was significantly higher than that in women (23.8±67.5 versus 13.3±51.3, P<0.0001), and each sex demonstrated significant increases in scores with age. We found no significant differences of sensitivity for obstructive disease (men, 97%; women, 96%), although women had significantly higher specificity (men, 36%; women, 47%; P<0.001). Participants had, on average, 2.6 cardiovascular risk factors.
Of 1851 patients, 983 (53%) had obstructive disease (luminal stenosis ≥50% in 1 or more vessels). In this cohort, mean total calcium score was 608 (range, 0 to 6646). Calcium scores were significantly lower for individuals without obstructive disease (n=868; 123; range, 0 to 3761; P<0.001).
Multivariate Prediction of the Probability of CAD
The variables incorporated into the model are prospectively defined and summarized in Table 1. The calibration and discrimination for each of the four prediction models are summarized in Table 2. Although the pretest model based only on age and sex was well calibrated, especially in the middle range of probability (Figure 1A), discrimination was relatively low (ROC area=0.672±0.019). In contrast, the test model based on the EBT calcium scores (Figure 1B) was significantly less well calibrated (F[7,5]=15.4, P=0.03) but higher in discrimination (ROC area=0.842±0.020). The combined model, incorporating all the variables, was well calibrated (Figure 1C) and provided a significant increase in discrimination over the pretest model (Figure 2) but also exhibited a significant amount of verification bias (the ratio of the angiographic referral rate in those with a calcium score >0 [628/3157=0.199] divided by the angiographic referral rate in those with a calcium score=0 [251/1825=0.138] was 1.44). Adjustment for this verification bias did not affect calibration (Figure 1D) or discrimination (Table 2) but significantly raised specificity (66%).
Coronary risk factors do not allow for discrimination of severity of angiographic CAD17 or adverse outcomes in individuals but are useful prognostic indicators in large patient populations. Detection of CAC by EBT has been demonstrated to be highly sensitive for the presence of significant CAD.8 A positive EBT study (presence of CAC) is nearly 100% specific for atheromatous coronary plaque.6,7⇓ Recent guidelines support the use of EBT in symptomatic persons, stating that EBT is “sufficiently accurate for predicting the presence of angiographic stenosis.”18 Previous reports tried to “merge’ multiple data sets with different entry criteria and different definitions of obstruction, demonstrating a very high sensitivity but lower specificity.18,19⇓ Because CAC is a continuous variable with increasing calcium scores leading to increasing specificity, there is much greater information content than use as a dichotomous variable.
The EBT-derived predictors of angiographic and multivessel disease can be useful as marker of underlying coronary atherosclerotic plaque burden, thus having a continuous association with atherosclerotic disease severity.10,20⇓ Calcium scores from EBT correlate with severity of CAD and are useful in predicting angiographic 3-vessel and/or left main disease.20 This study documents the clinical applicability of EBT-derived calcium scores for the noninvasive diagnosis of CAD in two broad areas: for diagnosis, by the correlation of posterior probability with angiographic prevalence; and for evaluation, by the relation of posterior probability to the anatomic severity of disease. As in other studies, the addition of EBT scores added independent and incremental information to predict obstructive CAD over age and sex (Figure 2), dramatically altering the post-test probability of disease across a wide range of patients (Table 3).
The identification of symptomatic patients with severe angiographic disease is becoming increasingly important.21 Our results demonstrate that in symptomatic patients, EBT in conjunction with a clinically derived pretest probability can noninvasively predict angiographic disease, with post-test probabilities for vessel-specific disease and multivessel disease. Not accounting for the increasing incidence of CAD with increasing age leads to a loss of information. By adding to the model an additional interaction term defined as (Total Score=0) · Age, the absence of calcification is weighted relatively more heavily in the model (proportional to age), whereas the overall calibration of the model remains essentially unchanged. For example, a 55-year-old man with a pretest probability of 53% (mean of our study population) and no calcification (calcium score=0) has a post-test probability of 9.9%. This is consistent with the raw negative predictive accuracy of 90% in this population: 315 of 350 patients with no CAC have normal coronary arteries. Similarly, the same man with a score of 380 (mean for the population studied) has a post-test probability of 87%. A variety of post-test probabilities are displayed in Table 3. Those patients who exhibited the greatest change from pretest to post-test probability were those patients with pretest probability ranging from 20% to 70%. This is consistent with many other cardiovascular tests, including exercise treadmill and thallium testing. Posterior probability also served as a predictor of the extent of angiographic disease, because the prevalence of multivessel disease increased as a monotonic curvilinear function of probability to a high of 92% (Figure 1C).
A graphical comparison of the pretest model with the combined model is illustrated in Figure 2. The comparison is statistically significant (P<0.0001), indicating that EBT adds an increment of diagnostic information to age and sex. The resultant prediction model estimated the posterior probability of significant CAD in each of these 932 patients (training sample) and the 919 patients (validation sample). This approach ensures the capability of integrating additional variables or adjusting for differences in baseline prevalence of disease.22 The ROC curves for test and adjusted models were essentially identical to that for the combined model.
The estimates of sensitivity and specificity are biased, however, because information regarding the ability of EBT to predict obstructive CAD was obtained from symptomatic patients who had CAD status verified with coronary angiography. As a result of this verification bias, sensitivity will be overestimated (positive EBT results are overrepresented) and specificity will be underestimated (negative EBT results are underrepresented). Bielak et al23 adjusted for verification bias, raising the specificity of EBT (for obstructive CAD) over the unadjusted specificity in the angiography group (72% versus 39%), with little change in the sensitivity (97% versus 99%). The verification bias in our study was 1.44, significantly greater than a value of 1 (no verification bias, P<0.0001) but lower than the value by Bielak et al.23 The patient enrollment in our study was mostly after the angiogram, so there is a higher likelihood that verification bias was reduced. Correction of the verification bias (Figure 1D) significantly raised the expected specificity for obstructive CAD (40% versus 66%, P<0.001), without significant change in sensitivity (96% versus 95%).
A large angiographic and EBT study (n=1764) recently reported similar findings, with a higher sensitivity and lower specificity,24 in part due to higher pretest probabilities of disease. When evaluated for scores >100, the two studies demonstrated nearly identical sensitivities and specificities for CAD as well as remarkably similar ROC values. Haberl et al24 demonstrated a sensitivity of 99% in men and 100% in women for detecting obstructive CAD, indicating that this test is an effective filter before angiography.
EBT greatly enhances the clinician’s ability to predict angiographic disease, both prevalence and severity, and can be of aid in determining the next test or treatment. These observations should “travel” well from laboratory to laboratory despite local differences in population, as EBT is standardized, quantitative,8,11⇓ and operator independent.11 These data will not be applicable to CAC scores obtained with multislice or spiral CT. Because this model is highly dependent on the absence of calcification (score=0), results of spiral CT should not be used. Calcium scores obtained with spiral CT are significantly different than EBT, with a lower sensitivity for calcification.25,26⇓
Continuous variables of cardiac risk factors, such as serum lipoprotein levels, were not available for analysis. The results of this study are applicable to persons with suspected CAD, not the general asymptomatic population.
We excluded severity of stress testing abnormality, since this must be used to estimate pretest probability. Similarly, the nature of chest pain would be used by the physician to derive a pretest probability of obstruction. A highly positive stress thallium examination in a patient with typical angina would greatly raise the pretest probability, not alter the age, sex, or EBT results, which could then be used to generate a post-test probability.
Clinical Relevance of the Study Design
We sought to determine the accuracy of EBT scanning in an environment that mirrored the realities of clinical practice. We gathered data from different geographic locales. The interpretation of each test, from risk factors to the angiogram, implied a variable amount of subjectivity and probable bias. This mirrors clinical reality, and is, in one sense, desirable. In summary, a practical, readily available program that integrates individual vessel calcium scores from EBT has been successfully used in the diagnosis and evaluation of CAD. EBT calcium scores are not a substitute for coronary angiography but can be used with other clinical and demographic markers to help predict angiographic stenosis severity in patients suspected of having obstructive CAD. As a noninvasive test for CAD, EBT has great appeal because it is fast (<10 minutes per scan), simple to use (no physiological or pharmacological stress required), and relatively inexpensive (estimated charge <$400),12 with negligible interreader and intrareader variability.12 The model described can assist clinicians to develop a post-test probability that will be useful for establishing a high (75% to 92%) as opposed to a low (<10%) probability of angiographically significant CAD. It may facilitate individual decision-making concerning the use of coronary catheterization in symptomatic patients. Exclusion of CAC defines a substantial subgroup of patients, albeit symptomatic, with a very low probability of significant stenosis. What differentiates this study is that it is the first to develop (1) a continuous model that is (2) adjusted for age and sex, (3) corrected for verification bias, and (4) independently validated in terms of its incremental diagnostic accuracy.
- ↵Kennedy J, Shavelle R, Wang S, et al. Coronary calcium and standard risk factors in symptomatic patients referred for coronary angiography. Am Heart J. 1998; 135: 696–702.
- ↵Detrano R, Tzung H, Wang S, et al. Prognostic value of coronary calcification and angiographic stenoses in patients undergoing coronary angiography. J Am Coll Cardiol. 1996; 27: 285–290.
- ↵Arad Y, Sparado LA, Goodman K, et al. Prediction of coronary events with electron beam computed tomography. J Am Coll Cardiol. 2000; 36: 1253–1260.
- ↵Raggi P, Callister TQ, Cooil B, et al. Identification of patients at increased risk of first unheralded acute myocardial infarction by electron beam computed tomography. Circulation. 2000; 101: 850–855.
- ↵Wong ND, Hsu JC, Detrano RC, et al. Coronary artery calcium evaluation by electron beam computed tomography and its relation to new cardiovascular events. Am J Cardiol. 2000; 86: 495–498.
- ↵Rumberger JA, Simons DB, Fitzpatrick LA, et al. Coronary artery calcium areas by electron beam computed tomography and coronary atherosclerotic plaque area: a histopathologic correlative study. Circulation. 1995; 92: 2157–2162.
- ↵Mautner GC, Mautner SL, Froelich J, et al. Coronary artery calcification: assessment with electron beam CT and histomorphometric correlation. Radiology. 1994; 192: 619–623.
- ↵Budoff MJ, Georgiou D, Brody A, et al. Ultrafast computed tomography as a diagnostic modality in the detection of coronary artery disease: a multicenter study. Circulation. 1996; 93: 898–904.
- ↵Wexler L, Brundage B, Crouse J, et al. Coronary artery calcification: pathophysiology, epidemiology, imaging methods, and clinical implications: a statement for health professionals from the American Heart Association. Circulation. 1996; 94: 1175–1192.
- ↵Guerci AD, Spadaro LA, Goodman KJ, et al. Comparison of electron beam computed tomography scanning and conventional risk factor assessment for the prediction of angiographic coronary artery disease. J Am Coll Cardiol. 1998; 32: 673–679.
- ↵Agatston AS, Janowitz WR, Hildner FJ, et al. Quantification of coronary artery calcium using ultrafast computed tomography. J Am Coll Cardiol. 1990; 15: 827–832.
- ↵Budoff MJ, Raggi P. Coronary artery disease progression assessed by electron-beam computed tomography. Am J Cardiol. 2001; 88: 46E–50E.
- ↵Becker C, Schatzl M, Feist H, et al. Assessment of the effective dose for routine protocols in conventional CT, electron beam CT and coronary angiography. Rofo Fortschr Geb Rontgenstr Neuen Bildgeb Verfahr. 1999; 170: 99–104.
- ↵Scanlon PJ, Faxon DP, Audet AM, et al. ACC/AHA guidelines for coronary angiography: a report of the American College of Cardiology/American Heart Association Task Force on practice guidelines (Committee on Coronary Angiography). J Am Coll Cardiol. 1999; 33: 1756–1824.
- ↵Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics. 1983; 39: 207–215.
- ↵Roger VL, Pellikka PA, Bell MR, et al. Sex and test verification bias: impact on the diagnostic value of exercise echocardiography. Circulation. 1997; 95: 405–410.
- ↵Vlietstra RE, Kronmal RA, Frye RL, et al. Factors affecting the extent and severity of coronary artery disease in patients enrolled in the Coronary Artery Surgery Study. Arteriosclerosis. 1982; 2: 208–215.
- ↵O’Rourke RA, Brundage BH, Froelicher VF, et al. American College of Cardiology/American Heart Association expert consensus document on electron-beam computed tomography for the diagnosis and prognosis of coronary artery disease. Circulation. 2000; 102: 126–140.
- ↵Nallamothu BK, Saint S, Bielak LF, et al. Electron-beam computed tomography in the diagnosis of coronary artery disease. Arch Intern Med. 2001; 161: 833–838.
- ↵Schmermund A, Bailey KR, Rumberger JA, et al. An algorithm for noninvasive identification of angiographic three-vessel and/or left main coronary artery disease in symptomatic patients on the basis of cardiac risk and electron-beam computed tomographic calcium scores. J Am Coll Cardiol. 1999; 33: 444–452.
- ↵Caracciolo EA, Davis KB, Sopko G, et al. Comparison of surgical and medical group survival in patients with left main coronary artery disease: long-term CASS experience. Circulation. 1995; 91: 2325–2334.
- ↵Morise AP, Diamond GA, Detrano R, et al. The effect of disease-prevalence adjustments on the accuracy of a logistic prediction model. Med Decis Making. 1996; 16: 133–142.
- ↵Bielak LF, Rumberger JA, Sheedy PF, et al. Probabilistic model for prediction of angiographically defined obstructive coronary artery disease using electron beam computed tomography calcium score strata. Circulation. 2000; 102: 380–385.
- ↵Haberl R, Becker A, Leber A, et al. Correlation of coronary calcification and angiographically documented stenoses in patients with suspected coronary artery disease: results of 1,764 patients. J Am Coll Cardiol. 2001; 37: 451–457.
- ↵Budoff MJ, Mao SS, Zalace CP, et al. Comparison of spiral and electron beam tomography in the evaluation of coronary calcification in asymptomatic persons. Int J Cardiol. 2001; 77: 181–188.
- ↵Goldin JG, Yoon HC, Greaser LEIII, et al. Spiral versus electron-beam CT for coronary artery calcium scoring. Radiology. 2001; 221: 213–221.