Relation of Operator Volume and Experience to Procedural Outcome of Percutaneous Coronary Revascularization at Hospitals With High Interventional Volumes
Background Although an inverse relation between physician caseload and complications has been conclusively demonstrated for several surgical procedures, such data are lacking for percutaneous coronary intervention, and the ACC/AHA guidelines requiring ≥75 cases per year for operator “competency” are considered by some physicians to be arbitrary.
Methods and Results From quality-controlled databases at five high-volume centers, models predictive of death and the composite outcome of death, Q-wave infarction, or emergency bypass surgery were developed from 12 985 consecutively treated patients during 1993 through 1994. Models had moderate to high discriminative capacity (area under ROC curves, 0.65 to 0.85), were well calibrated, and were not overfitted by standard tests. These models were used for risk adjustment, and the relations between both yearly caseload and years of interventional experience and the two adverse outcome measures were explored for all 38 physicians with ≥30 cases per year. The average physician performed a mean±SD of 163±24 cases per year and had been practicing angioplasty for 8±5 years. Risk-adjusted measures of both death and the composite adverse outcome were inversely related to the number of cases each operator performed annually but bore no relation to total years of experience. Both adverse outcomes were more closely related to the logarithm of caseload (for death, r=.37, P=.01; for death, Q-wave infarction, or bypass surgery, r=.58, P<.001) than to linear caseload.
Conclusions In this analysis, high-volume operators had a lower incidence of major complications than did lower-volume operators, but the difference was not consistent for all operators. If these data are validated, their implications for hospital, physician, and payer policy will require exploration.
An inverse relation has been described between the incidence of major complications of several procedures, including PTCA, and the number of these procedures performed at a given hospital.1 2 3 4 5 Differences in outcome between high- and low-volume hospitals may be due to greater experience at high-volume centers, to superior technique, or to treatment of sicker patients at low-volume centers.5 Although empirical data relating results to physician experience per se are scant,6 the data obtained from per-hospital analyses have been used to justify the current ACC/AHA recommendations that “a minimum of 75 PTCA procedures (be) performed per year as the ‘primary operator’ in order to be competent to perform PTCA.”7 These recommendations have been challenged.8
Even within a hospital with a large procedural volume, yearly physician volume and experience vary. At many such centers, formal teaching conferences, informal sharing of clinical experience, and experienced surgical teams might be expected to mitigate against differences in the procedural outcome of high- and low-volume operators, even if they might otherwise have existed. We sought to build on prior experience in analyzing differences in operator-related coronary interventional procedure outcome9 to ascertain whether yearly procedural outcome or overall experience influenced the procedural outcome of PTCR at five hospitals with considerable PTCA volume and quality-assured databases that would facilitate such an analysis.
Six hospitals with high-volume (>1000 procedures per year) programs in interventional cardiology and established databases were contacted to participate in this study. Each maintains credentialing standards for individual physician operators and an interventional database characterized by prospective entry of selected clinical and angiographic data, routine postprocedure analysis of ECGs for detection of periprocedural MI, analysis and coding of complications by trained personnel other than the interventional cardiologist performing the procedure, and internal audits and checks for data completeness and consistency.
Five hospital programs agreed to participate and submitted selected data on all interventional procedures performed at their center during calendar years 1993 and 1994 to the central data analysis center at the Cleveland Clinic. The sixth hospital declined, largely because, in contradistinction to the other hospitals, procedures were frequently performed by more than one staff-level physician, and hence, attribution of results to a single physician would be problematic. Because of recent improvements in the data auditing measures at one center, that center agreed to submit data only from July 1, 1994, to December 31, 1994. No patient with attempted treatment during the inclusive dates was excluded from the analysis for any reason.
Baseline Clinical and Angiographic Information
The variables acute MI (onset ≤24 hours), age, Canadian Cardiovascular Society angina class, cardiogenic shock, left ventricular ejection fraction, modified ACC/AHA lesion classification score,10 number of diseased vessels, prior bypass surgery, prior restenosis, sex, and unstable angina were recorded.
The treating interventionalist of record for each patient and the number of interventions and total years of performing interventions for each physician were also recorded.
The outcome variables in-hospital death, emergency bypass surgery, and Q-wave MI were tabulated for all patients.
Definitions and Conventions
The variables noted above have been described elsewhere.9 10 Although our earlier analyses revealed a nearly linear relation between the modified ACC/AHA lesion score and adverse outcome,9 our more recent analyses suggested that A and B1 lesions had a much better outcome than B2 and C lesions10 ; therefore, we analyzed lesion characteristic score as both a linear and a nonlinear risk variable. Emergency bypass surgery referred to that performed at any time during the index hospitalization because of a complication of the PTCA. Although definitions used at each hospital for this variable differed slightly, all stressed the urgency of the clinical indication for surgery and most specifically required ongoing ischemia, hemodynamic instability, or life-threatening anatomy not present before the intervention. Q-wave infarction was defined as significant Q waves present after the procedure that were not present before. All centers obtained postprocedure ECGs on all patients, which were interpreted by trained cardiologists. For the purposes of this analysis, a patient treated with PTCA for an acute MI who developed a Q-wave infarction on ECG was not considered to have had a complication of the procedure unless he or she died or had emergency bypass surgery.
Baseline characteristics of the treated patients were summarized in terms of frequencies and percentages for categorical variables and mean±SD for continuous variables. Multivariable logistic regression models were used to examine individual and interaction relations between baseline characteristics and death and also the composite of death, Q-wave infarction, or bypass surgery.
Models were fitted by the equation probability=ey/(1+ey), where y is a constant plus the sum of all variables’ (value×β-coefficient). The full study population was used in the model development process, and the predictive performance of the model was internally validated by cross-validation methodology.11 The model was fitted on a randomly selected subset of 80% of the study patients, and the resulting fit was tested on the remaining 20% of the study patients. This process was repeated 10 times to estimate the extent to which the predictive accuracy of the full model was overoptimistic. The measure of predictive discrimination used to characterize model performance was the area under the ROC curve (the c statistic).12 Calibration of the model predictions was assessed by comparison of the average model prediction with the observed event incidence across deciles of risk.
To evaluate the performance of individual operators, a variable coding for each operator was added one at a time to the models. The estimated logistic regression coefficient and standard error were then used to determine an adjusted odds ratio with corresponding 95% confidence limits, which were then converted back to adjusted risk by application of the adjusted odds ratio to the known risk of the event in the population. Only physician operators with at least 30 cases per year in the database were analyzed because of the concern that estimates of the risk of procedural complications for physicians with fewer cases would be highly unstable.9 The relations of operator volume and years of experience to raw and adjusted outcome were evaluated with least-squares linear regression techniques. Physician operators were also divided into quintiles by caseload, and the relationship between caseload quintile and risk-adjusted results in low-risk (ACC/AHA A or B1) and high-risk (ACC/AHA B2 or C) lesions was evaluated.
The baseline characteristics of the patients treated are enumerated in Table 1⇓. Most patients were middle-aged males; about half had unstable angina, 25% prior bypass surgery, 20% prior PTCR, 60% multivessel disease, and 70% complex ACC/AHA B2 or C lesions. Major adverse outcomes were seen in 4.5% of patients (death in 1.3%, Q-wave infarction in 3.5%, emergency bypass surgery in 2.1%).
Correlates of Adverse Outcome
Correlates of death and the composite adverse outcome (death, Q-wave infarction, or bypass surgery) are described in Tables 2⇓ and 3⇓. In-hospital death was independently correlated with the following variables: shock, treatment for acute MI, the logarithm of patient age in years, lesion morphology, female sex, and the number of diseased vessels. The model derived from these data showed good discriminatory power (area under the ROC curve, c statistic=0.846), was well calibrated to events (r=.96), and was not overfitted (cross-validation correction to the c statistic=0.002). The composite adverse outcome was independently associated with the variables shock, lesion morphology, presentation with acute MI, female sex, and no prior bypass surgery. The model obtained from these data had modest discriminatory power (c statistic=0.648), was very well calibrated to events (r=.98), and was not overfitted (cross-validation correction to the c statistic=0.013).
The use of these models to provide risk-adjusted outcomes made a considerable difference compared with the corresponding raw values for some physicians (see Figs 1⇓ and 2⇓). For the end point of death, half of the physicians had a relative change of >25%, and one quarter had a change of ≥40%. For the composite adverse outcome, half of the physicians had a relative change of >18%, and one quarter had a change of >25%.
Physician Experience and Outcome
The average physician evaluated in this analysis performed 163±24 (range, 30 to 628) cases per year during 1993 through 1994 and had been practicing angioplasty or related techniques (eg, atherectomy) for 8±5 years (range, 1 to 16 years). The results of five physicians were excluded because they did not perform the requisite number of cases per year. In aggregate, these five physicians performed 148 cases during the 2-year study. Their risk of death, Q-wave infarction, or emergency bypass surgery ranged from 0% to 7.9±8.6%. The relation between experience and patient outcomes for physicians with ≥30 cases per year is described in Table 4⇓. Both death and the composite adverse outcome were strongly and inversely related to the number of cases each operator performed annually, but they bore no relation to the total number of years of physician experience. The correlations for both death and the composite adverse outcome were higher for the logarithm of procedural volume than for volume itself, suggesting an inverse exponential relationship (Fig 3⇓ and Table 4⇓). The relation between operator volume and procedural results for all patients, as well as those with low- and high-risk lesions, is shown graphically in Fig 4⇓. For patients with only low-risk lesions, only the operator quintile with lowest volume (<70 cases per year) had results that were significantly worse than the remainder of the group. For patients with high-risk lesions, the lowest-volume quintile had significantly worse outcome, and the highest-volume quintile (>270 cases per year) had significantly better outcomes than the remainder of the group. The effect of caseload on outcome was consistent for all hospitals.
In this analysis of 12 985 patients relating PTCR outcome to physician experience, we found that risk adjustment made a considerable (≥30% relative change) impact on 10% to 40% of physician operators evaluated. Whether risk adjusted or not, however, both death (P≤.01) and the composite adverse outcome of death, Q-wave infarction, or need for emergency bypass surgery (P<.001) were significantly and inversely related to operator caseload (Figs 3⇑ and 4⇑). This occurred even though each hospital had a very large overall caseload and all are teaching hospitals to the extent that angioplasty outcome is routinely discussed at conferences and between colleagues. However, the correlation between caseload and procedural outcome was far from exact, implying considerable physician-to-physician variation, some “instability” of results even for providers with caseloads of 30 to 628 procedures per year, and limitations of the modeling process. Absolutely no relation between results and the number of years each physician had been performing angioplasty was seen.
The inverse relation between the caseload of many surgical procedures and complications is well supported by the results of several studies.1 2 3 To date, the relationship between angioplasty volume and results has been clearly defined only for hospital, but not for physician, volume. Ritchie et al4 analyzed the angioplasty experience in the State of California during 1989 using an administrative data set with ICD-9-CM codes13 and limited patient characteristic variables. These investigators found a strong inverse relation between hospital PTCA volume and the risk of emergency bypass surgery but not overall mortality. The presence of new Q-wave infarction could not be ascertained. Jollis et al5 used a Medicare Provider Analysis and Review (MEDPAR) data set from 1987 through 1990 and found that hospitals performing <200 angioplasties yearly had higher incidences of both death and emergency surgery than did hospitals with larger volumes. Because of the nature of the data set, the risk of MI could not be examined. In a small study, Hamad et al6 suggested that high-volume operators (>100 PTCAs per year) might have less frequent complications with “complex lesions” than “low-volume” operators.
On a largely empirical basis, the ACC/AHA suggested in 1993 that each physician perform a minimum of 75 angioplasties yearly to have sufficient experience to continue.7 Some insurance companies are reimbursing hospitals and physicians for angioplasty only if the physician has performed ≥75 cases per year. Not only insurance companies but also patients14 are becoming increasingly concerned about physician-specific patient outcome.
Most groups that perform such analyses recognize that the type of patients treated should be accounted for in judging results, but many administrative data sets have major limitations in data veracity and complexity, which limits their capacity to risk adjust.15 Sophisticated “scorecarding”16 of physician results may be beneficial in several ways,9 15 16 but application of incomplete data sets or those without key prognostic variables may lead to sanctioning of competent physicians from practicing or tempt physicians to avoid treating the higher-risk patients.16 17
Possible adverse end points to be used in “scorecarding” analysis such as this have been reviewed.9 Death and Q-wave infarction are inarguably detrimental to a patient’s well-being. One might argue that bypass surgery, as an alternative means of revascularization, should not be considered an adverse outcome. Emergency surgery, however, as defined in this analysis, is usually associated with a lessened likelihood of the patient’s receiving an arterial as opposed to a venous graft, and especially with ongoing ischemia, with a substantial risk of later death, infarction, or need for further revascularization.18 19
Data sets to be used for evaluating physician results should have complete and accurate data, have an outcome that is clearly linked to the responsible physician, and reflect an experience large enough that one can be confident that the overall outcome is unlikely to be due to chance alone.9 In addition, angioplasty databases that have access to clinical and angiographic variables such as hemodynamic instability preceding the procedure and lesion morphology nearly always find them to be independent correlates of adverse outcome; hence, despite recognized limitations,9 they should be a prerequisite for profiling angioplasty operators. The 12 985-patient data set used here meets all these requirements. Furthermore, these patients are typical of patients undergoing PTCR from broad-based series20 and tertiary referral centers.9 21 Finally, the models derived, validated, and used in this analysis have predictive capacity similar to the best reported results for angioplasty outcomes.9 21
The results of this analysis should be assessed in the context of certain study limitations. First, although the likelihood that the general relationship observed between volume and outcome was due to the play of chance was exceedingly small, the exact relation between volume and outcome for any single physician is not nearly so strong: many lower-volume operators had good results during the time period studied. Second, the results were derived from only five hospitals, and one should be conservative about generalizing the findings until they are confirmed. Third, as with all analyses that use regression analysis to adjust for differences between types of patients treated, unmeasured variables may have affected patient outcome. Nonetheless, the concordance of these results with others reported from the surgical literature1 2 3 strongly suggest that they have merit. Fourth, it is possible that some of the relationship between operator volume and results might be explained by the low-volume operator’s lack of immediate access to newer technologies, such as stents, for the “bailout” indication and not be experience per se. Only three physicians in this study were not credentialed for bailout stenting during 1993 through 1994 and would have required the assistance of another physician in this setting, thus limiting our ability to analyze this potential confounding factor. Finally, the technical and pharmacological strategies of PTCR continue to evolve quickly, and the relationship between operator experience and outcome will require reexamination.
The results of this study have three primary implications: (1) physician-to-physician differences in angioplasty outcome appear to be large enough so that it may be useful to develop individual physician profiles; (2) the magnitude of change in adjusted compared with nonadjusted outcomes is so large that non–risk adjusted data should not be used for physician evaluation if appropriate and validated risk-adjustment models are available; (3) if angioplasty procedures are performed by higher-volume operators, then outcomes, on average, will likely be better than if lower-volume operators perform the procedures.
Selected Abbreviations and Acronyms
|ACC/AHA||=||American College of Cardiology/ American Heart Association|
|PTCA||=||percutaneous transluminal coronary (balloon) angioplasty|
|PTCR||=||percutaneous transluminal coronary revascularization|
|ROC||=||receiver operating characteristic|
The authors appreciate the expert secretarial assistance of Patti Durnwald.
- Received November 19, 1996.
- Revision received February 12, 1997.
- Accepted February 28, 1997.
- Copyright © 1997 by American Heart Association
Ritchie JL, Phillips KA, Luft HS. Coronary angioplasty: statewide experience in California. Circulation. 1993;88:2735-2743.
Hamad N, Pichard AD, Lyle HRP, Lindsay J, and members of the Cardiac Catheterization Laboratory, Washington Hospital Center. Results of percutaneous transluminal coronary angioplasty by multiple, relatively low frequency operators: 1986-1987 experience. Am J Cardiol. 1988;61:1229-1231.
Ryan TJ, Bauman WB, Kennedy JW, Kereiakes DJ, King SB III, McCallister BD, Smith SC Jr, Ullyot DJ. Guidelines for percutaneous transluminal coronary angioplasty. Circulation. 1993;88:2987-3007.
O’Neill WW, Griffin JJ, Stone G, Brodie B, Jones DE, Sachs D, Esente P, Spain M, Ayres M, Grines CL. Operator and institutional volume do not affect the procedural outcome of primary angioplasty therapy. J Am Coll Cardiol. 1996;(special issue):13A. Abstract.
Ellis SG, Omoigui N, Bittl JA, Lincoff AM, Wolfe MW, Howell G, Topol EJ. Analysis and comparison of operator specific outcomes in interventional cardiology: from a multicenter database of 4860 quality-controlled procedures. Circulation. 1996;93:431-439.
Egan JP. Signal Detection Theory and ROC Analysis. New York, NY: Academic Press; 1975.
The International Classification of Diseases, 9th revision, clinical modification (ICD-9-CM). 2nd ed. Vol I. Diseases: Tabular List. Washington, DC: Government Printing Office; 1980. DHHS publication (PHS) 80-1260.
Zimman D. Heart surgeons rated: state reveals patient-mortality records. Newsday, Nassau and Suffolk edition. December 18, 1991:3.
Omoigui NA, Miller DP, Brown KJ, Annan K, Cosgrove D III, Lytle B, Loop F, Topol EJ. Outmigration for coronary bypass surgery in the era of public dissemination of clinical outcomes. Circulation. 1996;93:27-33.
Talley JD, Weintraub WS, Roubin GS, Douglas JS Jr, Anderson HV, Jones EL, Morris DC, Liberman HA, Craver JM, Guyton RA, King SB III. Failed elective percutaneous transluminal coronary angioplasty requiring coronary artery bypass surgery: in-hospital and late clinical outcome at 5 years. Circulation. 1990;82:1203-1213.
Hannan El, Arani DT, Johnson LW, Kemp HG, Lukacik G. Percutaneous transluminal coronary angioplasty in New York State: risk factors and outcomes. JAMA. 1993;268:3092-3097.