(Circulation. 2001;103:1225.)
© 2001 American Heart Association, Inc.
Clinical Investigation and Reports |
From the Swiss Cardiovascular Center, University Hospital, Bern, Switzerland (W.M., O.M., B.M.), and Center for Computing Technology, University of Bremen, Germany (J.A., M.B.W.). Dr W. Maier is currently affiliated with the Dept of Cardiology, University Hospital Zurich, Zurich, Switzerland.
Correspondence to Bernhard Meier, MD, Professor and Head of Cardiology, Swiss Cardiovascular Center Bern, CH-3010 Bern, Switzerland. E-mail bernhard.meier{at}insel.ch
| Abstract |
|---|
|
|
|---|
Methods and ResultsMC was attributed to the target lesions by consensus of 2 observers. The predictive value regarding procedural success (PS) and major adverse cardiac events (MACE) of MC was analyzed by conventional logistic regression analyses and by inductive machine learning models. The study was adequately powered for the methods applied with 325 target lesions of 250 cases. Overall, PS decreased and MACE increased from type A to type C lesions. Regression analysis identified no single factor as predictive. Logistic regression showed an error rate of 42%. Machine learning techniques achieved an individual predictive error of only 10%, which could be further reduced to 2% by addition of parameters. For PS, MC parameters showed a high ranking for building the model. For MACE, variables of the medical history showed more impact.
ConclusionsMC per se cannot individually predict PS or MACE. However, when all MC parameters are integrated together with additional lesion-specific and history variables, a high individual predictive value can be achieved. This technique may be clinically helpful for risk stratification in the catheterization laboratory and improvement of classification systems in interventional cardiology.
Key Words: angioplasty complications computers prognosis
| Introduction |
|---|
|
|
|---|
Additionally, the issues of quality control and outcomes assessment have emerged to provide justification for the steadily growing expenditure for coronary interventions. In this context, an international quality control study in interventional cardiology was designed in Europe.11 Given the interventions of the late 1990s, the purpose of the present study was to assess the individual predictive value of the ABC criteria by conventional logistic regression analyses and decision-tree algorithms (see Appendix).1 An extension of the data model (given by the availability of all study parameters to the decision-tree algorithms) might improve the predictive performance. We also analyzed whether specific parameters help predict the procedural result versus major adverse cardiac events (MACE) during the intervention-related hospital stay.
| Methods |
|---|
|
|
|---|
Stratified Sampling
The selection was performed according to the
necessities of a stratified sampling technique. The sample should
parallel the population with respect to certain key characteristics
that are important to the respective
investigation.12
Case Analysis
Case records and imaging material, including reports
and image documentation from preceding hospital stays, were collected.
A dedicated computer program (AQUA Intervent) was used to record
pertinent data.11 A total of
301 variables per case were available for electronic analysis. The
majority rating of an expert panel determined the end-point procedural
success (see below).
All imaging material was analyzed by 2 independent observers blinded to the clinical outcome. The ACC/AHA classification2 3 was attributed to the target lesions by a third consensus rating in case of disagreement. Each target lesion was systematically documented according to the classification table in the AQUA Intervent program. Caliper measurements were performed for angulation and length. As additional parameters, for each lesion, TIMI flow13 and the status of the lesion with regard to previous infarction were documented.
Definitions and End Points
The ACC/AHA ABC classification criteria were applied
according to Ryan et
al2 3 and Ellis et
al.4 Flow was determined
according to the TIMI study
group.13 The additional
parameter "vessel related to infarction" was attributed when the
target lesion was part of an infarcted vessel by any definition
(previous, subacute, acute, or nonQ-wave infarction).
For assessment of the predictive value of the ACC/AHA ABC classification criteria, the following end points were considered: (1) procedural outcome; (2) MACE during the intervention-related hospital stay (death, myocardial infarction [MI], need for urgent bypass surgery [CABG], or hemodynamic compromise); and (3) extended MACE during the intervention-related hospital stay (death, MI, CABG, or hemodynamic compromise, plus the following out-of-laboratory complications: stable angina [Canadian Cardiovascular Society class II-IV], unstable angina, bleeding requiring transfusion or surgical intervention, cardiac arrest, cardiovascular collapse, recatheterization, reintervention, stent thrombosis, or abrupt closure).
Procedural success was defined by the expert panel as a residual diameter stenosis <50% by visual estimate. Visual estimate was chosen because this was a clinical quality control trial, and angiograms had been made without a quantitative coronary angiography protocol. The rationale for expert analysis was that even if quantitative coronary angiography had been performed, a <50% diameter stenosis at the end of the procedure alone would not completely fulfill the criteria of procedural success in the intended sense of the ACC/AHA criteria concept. Therefore, clinical expert judgment was implemented by setting the majority vote of the panel as reference.
MI was defined according to established criteria by
clinical, ECG, or enzymatic
changes.14 NonQ-wave MIs
were included if creatine kinase elevations
3x the upper normal
limit with positive cardiac isoenzymes were present. The study was not
designed for assessment of minimal myocardial damage through serial
enzyme assessments.
Hemodynamic compromise was defined as prolonged hemodynamic instability (systolic pressure <80 mm Hg, including prolonged vasovagal reactions) and new pulmonary congestion or edema.
Machine Learning Techniques
Machine learning addresses the question of how to
construct computer programs that automatically improve with experience
(see
Figure 1
and Appendix). It has proved to be of great
practical value in a variety of application domains. Machine learning
techniques are especially useful in (1) data-mining problems for which
(large) databases may contain valuable regularities that can be
discovered automatically (ie, to analyze outcomes of medical treatments
from patient databases), (2) poorly understood domains where humans
might lack the knowledge to develop effective algorithms, and (3)
domains where the program must dynamically adapt to changing
conditions.
|
The techniques used in this study for improving the
predictive accuracy were
"boosting"15 and "cost
functions." The Pearson
2 or
likelihood-ratio tests were used for calculation of probability values.
Accuracy was measured in connection with n-fold cross-validation (see
Appendix).1
Software and Hardware
Conventional statistics were computed with SAS (SAS
Institute Inc). The decision-tree techniques were based on C4.5
(RuleQuest Research). The AQUA Intervent program was developed with
Delphi.
| Results |
|---|
|
|
|---|
|
Descriptive Statistics: Distribution of Target
Lesions and Complication Rates Versus Morphological Types
Figures 2
and 3
show the overall distribution of
morphological types relative to target lesions and unsuccessful
procedures or complications. The majority of target lesions (64%) were
classified as B1/B2 lesions
(Figure 2
). The overall percentage of unsuccessful procedures
and MACE increased from type A to type C lesions
(Figure 3
).
|
|
Morphological Classification and Device
Use
Morphological classification did not predict device
use. The overall stent rate per case was 54%. Other so-called new
devices accounted for a negligible 3%. The distribution of morphology
types among target lesions and stented lesions did not differ: A,
target lesions 12%, stented lesions 14%; B1, 27% and 25%; B2, 37%
and 36%; and C1/2, 19% and 19%, respectively.
Stenosis length was found to be relevant for stent placement only in connection with diffuse disease. Vessels related to infarction were found to be slightly preferred for stenting (target 35%, stented 39%). There was less stenting in bifurcation lesions (target 14%, stented 9%). Procedural success in bifurcation lesions was 100% with stenting and 89% with PTCA.
The only approved glycoprotein IIb/IIIa receptor antagonist (abciximab) represented an element of the extended database. It was used in 3% of patients.
Univariate and Multivariate Regression of
Procedural Success Versus Morphological Classification
For univariate regression, no single factor enabled the
satisfactory prediction of any end point. Multivariate regression
showed an error rate of 42% (percentage of misclassified, 95%
confidence).
Morphological Classification, Procedural
Success, and MACE: Assessment With Machine Learning
Techniques
Probability of Procedural Success Based on a
Maximally Extended Data Model
Starting with all parameters, the decision-tree
algorithm extracts the relevant ones. The training data are correctly
classified.
Table 2
shows the error rates of the 10 different models
generated in connection with boosting and the overall error rate when a
case is analyzed simultaneously by application of these 10 different
models (error rate of the various models: minimum 2%, maximum 8% on
trained data). A 10-fold cross-validation gives an overall error rate
of 10%; each time, a different partitioning of the data into a
training set (90% of the data) and a validation set (the remaining
10% of the data) is used.
Figure 1
shows a flow chart representing the machine
learning process with some exemplary rules and the interaction of the
subtasks in the algorithm.
|
Machine learning techniques generate a ranking of criteria
for procedural success, shown in
Table 3
(the subgroups represent parameters of the same
level of importance, whereas the subgroups themselves are ordered in
decreasing importance). For instance, vessel bending is less important
than calcification, and calcification is less important than several
stenoses per segment or total occlusion.
|
Procedural success is primarily defined by the classic ABC
criteria and the additional parameters of left ventricular (LV)
function, TIMI flow, vessel related to infarction, and the procedure
(PTCA or stenting) itself. The most important parameter is stenting
(yes or no). Depending on this decision, 2 different trees for
procedural success are obtained. For example, stented patients showing
the characteristic "several stenoses per segment" have no
significantly different procedural success than those without
(P=0.6983), in contrast to
patients with PTCA-only strategy
(P=0.0004). The parameters of
the medical history play a secondary role, in contrast to the results
for MACE (see below).
Table 4
summarizes relevant parameters for procedural
success derived from the boosting model.
|
For the "pure" ACC/AHA criteria, the following ranking in order of decreasing (or equivalent) importance for procedural success can be derived from the decision trees: total occlusion; irregular vessel contour; severe calcification; proximal segment; severe tortuosity; bifurcation lesions/double guidewires; and diffuse stenosis length.
By subsequently adding further parameters to the ACC/AHA criteria in the training model (which optimizes classification, in contrast to cross-validation, which analyzes the performance on unknown cases), the misclassification rate drops. With addition of the parameters LV function and vessel related to infarction, misclassification rates are only 6%. When the type of procedure is added, it further decreases to 5%, finally dropping to 2% with the addition of TIMI flow.
Probability of MACE Based on a Maximally
Extended Data Model
The relevant parameters for MACE are nearly the same as
for procedural success
(Table 4
), but the ranking is completely different.
For MACE, history variables are of greater importance than the ACC/AHA
criteria. Machine learning techniques generate the ranking of criteria
for MACE shown in
Table 5
.
|
From the 10 different models calculated, a host of
interesting rules ensue. The following
rule (Rule 2/5) is an example:
![]() |
Extended MACE
The results for extended MACE are similar to those for
MACE. The ABC classification is not significant
(P=0.4139). Taking
abrupt closure as the dependent parameter, the corresponding
P value is 0.8660. Only a
combination of 3 different groups of parameters (medical history,
morphological, and intervention-related criteria) characterize extended
MACE. The most important parameter is again acute MI
(P<0.0001). Other parameters
are total occlusion, collaterals, vessel related to infarction, vessel
bending, sex, concomitant valvular heart disease, and reduced LV
function (P<0.0001 for the
latter). Additional parameters for an adverse event of the type
"extended MACE" are diameter before intervention (>90%), diameter
after intervention (>50%), and length of stenosis (>2 cm and/or
diffuse). Procedural success as an individual parameter is not
significant
(P=0.2).
| Discussion |
|---|
|
|
|---|
The distribution of target lesions to morphological types shows the majority in the type B group and more type C than type A lesions, in accordance with a recently reported series from a single US center.7 There is an association of cumulatively slightly decreasing procedural success and increasing procedural risk with lesion staging from A to C. However, ACC/AHA criteria do not predict individual outcome even when used as a set. With machine learning techniques, sets of different combinations (equivalent to sets of rules) of ACC/AHA criteria predict individual outcome with a misclassification rate of 10% in a retrospective analysis. For procedural success, the additional parameters of TIMI flow, vessel related to infarction, LV function, and most importantly, stenting (yes or no) are of high importance (reduction of misclassification rate from 10% to 2%). For predicting MACE, variables related to the medical history are more important than the ACC/AHA criteria.
Glycoprotein IIb/IIIa receptor antagonists, which were infrequently used, were not identified as major additional determinants. The reiterative self-improvement capacities of the algorithm, however, would allow the detection of any new variable that influences the outcome to a significant degree.
The high predictive performance is only achieved by the
computer model as a whole and requires all techniques applied (eg,
boosting, pruning, cross-validation, and cost functions). No single
identified rule (as shown in
Figure 1
) is of individual predictive value. Because of
their self-improvement capabilities, the predictive potency of the
final algorithms could be expanded once more when applied to procedures
performed in the same laboratory and by the same operators. This
generates a "custom-tailored" set of rules for the respective
group.
The outcome of coronary angioplasty has been documented in
large-scale
registries,17 18 19
which in part served as the basis for establishing the ACC/AHA
criteria. An earlier retrospective study failed to predict the
individual risk of abrupt vessel
closure.8 The present study
specifically deals with the individual predictive value of clinical or
morphological conditions for outcome of PTCA using a detailed and
highly scrutinized data set of a randomly selected patient population.
TIMI flow, the relationship of the target vessel to MI, and stenting
were identified as important additional classifiers. The established
ACC/AHA criteria together with these newly identified classifiers can
predict procedural success of PTCA with a misclassification rate of
only
2%. Although many of the identified rules may appear familiar
to the interventional cardiologist, others are puzzling. It must be
emphasized that they are only applicable in the framework of the
complex computer model. The predictive results, however, are generated
by artificial intelligence independent of any interventional
experience.
Two recent studies underscore the prevailing importance of factors, which we intend to address by use of the ABC criteria. Whereas the present study analyzed the individual predictive value of the ABC criteria on the immediate procedural success and on outcome during the procedure-related hospital period, Kastrati et al20 assessed the influence of a modified ACC/AHA score on long-term angiographic and clinical outcome after coronary stenting. They found a significant prognostic value for long-term outcome, which again argues in favor of the fact that the ABC criteria contain important determinants of outcome in interventional cardiology, which perhaps only need to be adequately weighted by methods of modern computing technique. In a large study population selected for stent or glycoprotein IIb/IIIa use, Ellis et al21 again were able to extract risk factors for complications of coronary interventions derived from the original ABC criteria. Our approach aims at modeling an individual risk profile from as many lesion and history characteristics as possible, using current tools of information technologies.
Applicability of the Algorithms for Routine
Purposes
The software can be implemented into a catheterization
report program running on standard Windows operating systems and
standard hardware.
Limitations of the Study
A possible limitation of the study is the sample size.
It was chosen to enable a scrutinized analysis of a representative
cross section of European PTCA cases. Thus, a more detailed,
cross-validated data set could be assessed. For definition of
procedural success, the majority decision of expert interventional
cardiologists was available in contrast to merely the operators
documentation in case of retrospective analysis of large-scale
registries. Because of the stratified sampling for case selection
according to opinion
polls,12 the samples can be
considered representative.
For the conventional statistics, the traditional approach to sample-size estimation requires the smallest worthwhile effects to be statistically significant. We obtained a minimum sample size of 247, calculating the sample size over the parameter p of a binomial distribution with given type I errors and type II errors of 5% and 10%, respectively. The decision-tree techniques require a minimum sample size of 5 to 10 cases per relevant parameter, which is clearly met for the 26 AHA criteria by the current data set.
In summary, ACC/AHA class per se cannot predict individual outcome of angioplasty. However, inductive machine learning techniques are able to build models based on the ACC/AHA criteria that classify individual outcome with high accuracy. By implementation of additional (history, function, lesion-specific) parameters, accuracy can be improved stepwise. This technique provides an operator-independent risk stratification before coronary interventions. It is cost-effective and easily applicable in clinical routine provided that the necessary data are electronically documented and sufficiently scrutinized. With further progress of electronic data management in the medical environment, this technology might also be of value for building large-scale registries for future improvement of guidelines in interventional cardiology.
| Appendix 1 |
|---|
|
|
|---|
2. Cross-Validation
One way to get a more reliable estimate of predictive
accuracy is by n-fold
cross-validation.1 The cases
in the data file are divided into n blocks of roughly the same size and
class distribution (we used n=10). For each block in turn, a classifier
is constructed from the cases in the remaining blocks and tested on the
cases in the holdout block.
3. Cost Function
Misclassification of a patient with complications as
one without complications is more serious and "costs" more than the
converse. Misclassification costs, represented by cost functions, are
numeric penalties for classifying an item into one category when it
really belongs in another. We calculated decision trees for which the
total cost of misclassification was minimized.
4. Boosting
Boosting gives higher predictive
accuracy.15 As the first
step, a single decision tree or rule set is constructed as before from
the training data. This classifier will usually make mistakes on some
cases in the data, and the first decision tree possibly gives the wrong
class for some cases. When the second classifier is constructed, more
attention is paid to these cases. As a consequence, the second
classifier will generally be different from the first. It also will
make errors on some cases, and these will be focused on when the third
classifier is constructed. This process continues for a predetermined
number of iterations. When a new case is to be classified, each
classifier votes for its predicted class, and the votes are counted to
determine the final
class.
| Acknowledgments |
|---|
Received July 20, 2000; revision received November 2, 2000; accepted November 16, 2000.
| References |
|---|
|
|
|---|
2.
Ryan TJ, Faxon DP,
Gunnar RM, et al. Guidelines for percutaneous transluminal coronary
angioplasty. Circulation. 1988;78:486502.
3.
Ryan TJ, Bauman WB,
Kennedy JW, et al. Guidelines for percutaneous transluminal coronary
angioplasty. Circulation. 1993;88:29873007.
4.
Ellis SG,
Vandormael MG, Cowley MJ, et al. Coronary morphologic and clinical
determinants of procedural outcome with angioplasty for multivessel
coronary disease: implications for patient selection.
Circulation. 1990;82:11931202.
5. Myler RK, Shaw RE, Stertzer SH, et al. Lesion morphology and coronary angioplasty: current experience and analysis. J Am Coll Cardiol. 1992;19:16411652.[Abstract]
6. Moushmoush B, Kramer B, Hsieh AM, et al. Does the AHA/ACC task force grading system predict outcome in multivessel coronary angioplasty? Cathet Cardiovasc Diagn. 1992;27:97105.[Medline] [Order article via Infotrieve]
7. Zaacks SM, Allen JE, Calvin JE, et al. Value of the American College of Cardiology/American Heart Association stenosis morphology classification for coronary interventions in the late 1990s. Am J Cardiol. 1998;82:4349.[Medline] [Order article via Infotrieve]
8. Tenaglia AN, Fortin DF, Calvin JE, et al. Predicting the risk of abrupt vessel closure after angioplasty in an individual patient. J Am Coll Cardiol. 1994;24:10041011.[Abstract]
9. Tan K, Sulke N, Taub N, et al. Clinical and lesion morphologic determinants of coronary angioplasty success and complications: current experience. J Am Coll Cardiol. 1995;25:855865.[Abstract]
10. Kimmel SE, Berlin JA, Strom BL, et al. Development and validation of simplified predictive index for major complications in contemporary percutaneous transluminal coronary angioplasty practice: the Registry Committee of the Society for Cardiac Angiography and Interventions. J Am Coll Cardiol. 1995;26:931938.[Abstract]
11.
Maier W, Enderlin
MF, Bonzel T, et al. Audit and quality control in angioplasty in
Europe: procedural results of the AQUA Study 1997.
Eur Heart J. 1999;20:12611270.
12. Raj D. Sampling Theory. New York, NY: McGraw-Hill; 1968.
13. The Thrombolysis in Myocardial Infarction (TIMI) trial: phase I findings: TIMI Study Group. N Engl J Med. 1985;312:932936.[Medline] [Order article via Infotrieve]
14.
Ryan TJ, Anderson
JL, Antman EM, et al. ACC/AHA guidelines for the management of patients
with acute myocardial infarction: executive summary.
Circulation. 1996;94:23412350.
15. Schapire RE. Theoretical views of boosting. In: Computational Learning Theory. Heidelberg, Germany: Springer; 1999:110.
16.
Budde T, Haude M,
Hopp HW, et al. A prognostic computer model to predict individual
outcome in interventional cardiology: the INTERVENT Project.
Eur Heart J. 1997;18:16111619.
17. Detre K, Holubkov R, Kelsey S, et al. Percutaneous transluminal coronary angioplasty in 19851986 and 1977- 1981: the National Heart, Lung, and Blood Institute Registry. N Engl J Med. 1988;318:265270.[Abstract]
18.
Detre KM, Holmes
DR Jr, Holubkov R, et al. Incidence and consequences of periprocedural
occlusion: the 19851986 National Heart, Lung, and Blood Institute
Percutaneous Transluminal Coronary Angioplasty Registry.
Circulation.. 1990;82:739750.
19. Holmes DR Jr, Holubkov R, Vlietstra RE, et al. Comparison of complications during percutaneous transluminal coronary angioplasty from 1977 to 1981 and from 1985 to 1986: the National Heart, Lung, and Blood Institute Percutaneous Transluminal Coronary Angioplasty Registry. J Am Coll Cardiol. 1988;12:11491155.[Abstract]
20.
Kastrati A,
Schömig A, Elezi S, et al. Prognostic value of the modified American
College of Cardiology/American Heart Association stenosis morphology
classification for long-term angiographic and clinical outcome after
coronary stent placement.
Circulation. 1999;100:12851290.
21.
Ellis SG, Guetta
V, Miller D, et al. Relation between lesion characteristics and risk
with percutaneous intervention in the stent and glycoprotein IIb/IIIa
era: an analysis of results from 10,907 lesions and proposal for new
classification scheme.
Circulation. 1999;100:19711976.
This article has been cited by other articles:
![]() |
D. J. Drenth, J. B. Winter, N. J. G. M. Veeger, S. H. J. Monnink, A. J. van Boven, J. G. Grandjean, M. A. Mariani, and P. W. Boonstra Minimally invasive coronary artery bypass grafting versus percutaneous transluminal coronary angioplasty with stenting in isolated high-grade stenosis of the proximal left anterior descending coronary artery: Six months' angiographic and clinical follow-up of a prospective randomized study J. Thorac. Cardiovasc. Surg., July 1, 2002; 124(1): 130 - 135. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Circulation Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2001 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |