(Circulation. 1996;93:431-439.)
© 1996 American Heart Association, Inc.
Articles |
Analysis and Comparison of Operator-Specific Outcomes in Interventional Cardiology
From a Multicenter Database of 4860 Quality-Controlled Procedures
From the Department of Cardiology (S.G.E., N.O., M.L., G.H., E.J.T.), the Cleveland Clinic Foundation, Cleveland, Ohio; and Brigham and Women's Hospital (J.A.B., M.W.W.), Harvard Medical School, Boston, Mass.
Correspondence to Stephen G. Ellis, MD, the Cleveland Clinic Foundation, 9500 Euclid Ave, F-25, Cleveland, OH 44195.
Background Medical consumers are increasingly requesting methods to discriminate among the results of different providers. Standards for appropriate modeling, risk adjustment, and evaluation ("scorecarding") in this setting are not well developed, although such evaluation is being performed by the medical insurance industry and by several states in the United States. Our objectives were to develop and examine clinically meaningful methodology for assessing the operator-specific results for percutaneous coronary revascularization.
Methods and Results From a multicenter database of patients treated since January 1, 1990, we used training and validation samples (n=4860) to develop several models for risk adjustment and applied them to 38 providers performing 25 to 523 procedures in the database. Models were developed using multivariable logistic regression techniques for combinations of the end points of death, myocardial infarction, bypass surgery, and procedural success. Models were evaluated for predictive accuracy by using receiver operating characteristic (ROC) analysis, for the capacity to discriminate between superior and inferior provider outcomes, and for subjectivity and concordance. Major complications occurred in 3.6% of patients. The area under the ROC curve (with perfect discriminatory accuracy, area=1.0; with no apparent accuracy, area=0.5) in the validation sample, and frequency of identification of operators with outcomes outside the 95% CI for the outcome in question for the models were for death, 0.85 and 7.9%; for death, Q-wave infarction, and bypass surgery, 0.77 and 13.2%; for death, all infarction, and bypass surgery, 0.66 and 10.5%; and for procedural success, 0.76 and 23.7%. For the models as a group, identification of outliers was inversely related to provider volume (P=.05). Models evaluating non–Q-wave infarction or requiring measurement of percent diameter stenosis were identified as being most susceptible to provider manipulation.
Conclusions For percutaneous coronary revascularization, modeling to discriminate between provider outcomes is limited by the low incidence of major adverse events, subjectivity or susceptibility to manipulation of more frequently occurring adverse events, the generally modest predictive capacity of the models, and the low volume of individual provider treatments. Modeling will be most useful in the identification of providers with extremely poor outcomes and for discrimination between providers with very large procedural volume. Until improved understanding of the biological and mechanical correlates of major complications allows the development of more predictive models, interpretation of the results of scorecarding, particularly for low-volume providers, should be made with caution.
Key Words: angioplasty • myocardial infarction • statistics
This article has been cited by other articles:
 |
|
 |
|
  S. B. King III, T. Aversano, W. L. Ballard, R. H. Beekman III, M. J. Cowley, S. G. Ellis, D. P. Faxon, E. L. Hannan, J. W. Hirshfeld Jr, A. K. Jacobs, et al. ACCF/AHA/SCAI 2007 Update of the Clinical Competence Statement on Cardiac Interventional Procedures: A Report of the American College of Cardiology Foundation/American Heart Association/American College of Physicians Task Force on Clinical Competence and Training (Writing Committee to Update the 1998 Clinical Competence Statement on Recommendations for the Assessment and Maintenance of Proficiency in Coronary Interventional Procedures) J. Am. Coll. Cardiol., July 3, 2007; 50(1): 82 - 108. [Full Text] [PDF]
|
|
|
|
 |
|
 C. R. Narins, A. M. Dozier, F. S. Ling, and W. Zareba The Influence of Public Reporting of Outcome Data on Medical Decision Making by Physicians Arch Intern Med, January 10, 2005; 165(1): 83 - 87. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Singh, C. S. Rihal, F. Selzer, K. E. Kip, K. Detre, and D. R. Holmes Jr Validation of Mayo clinic risk adjustment model for in-hospital complications after percutaneous coronary interventions, using the National Heart, Lung, and Blood Institute dynamic registry J. Am. Coll. Cardiol., November 19, 2003; 42(10): 1722 - 1728. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. J. de Feyter and E. McFadden Risk score for percutaneous coronary intervention: forewarned is forearmed J. Am. Coll. Cardiol., November 19, 2003; 42(10): 1729 - 1730. [Full Text] [PDF]
|
|
|
|
|
|
R. E. Shaw, H. V. Anderson, R. G. Brindis, R. J. Krone, L. W. Klein, C. R. McKay, P. C. Block, L. J. Shaw, K. Hewitt, W. S. Weintraub, et al. Development of a risk adjustment mortality model using the American College of Cardiology-National Cardiovascular Data Registry (ACC-NCDR) experience: 1998-2000 J. Am. Coll. Cardiol., April 3, 2002; 39(7): 1104 - 1112. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. A. Cowper, E. D. Peterson, E. R. DeLong, M. B. Wightman, R. P. Wawrzynski, L. H. Muhlbaier, and M. H. Sketch Jr The impact of statistical adjustment on economic profiles of interventional cardiologists J. Am. Coll. Cardiol., November 1, 2001; 38(5): 1416 - 1423. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. A. Vakili, R. Kaplan, and D. L. Brown Volume-Outcome Relation for Physicians and Hospitals Performing Angioplasty for Acute Myocardial Infarction in New York State Circulation, October 30, 2001; 104(18): 2171 - 2176. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Moscucci, E. Kline-Rogers, D. Share, M. O'Donnell, A. Maxwell-Eward, W. L. Meengs, P. Kraft, A. C. DeFranco, J. L. Chambers, K. Patel, et al. Simple Bedside Additive Tool for Prediction of In-Hospital Mortality After Percutaneous Coronary Interventions Circulation, July 17, 2001; 104(3): 263 - 268. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. M. Bashore, E. R. Bates, P. B. Berger, D. A. Clark, J. T. Cusma, G. J. Dehmer, M. J. Kern, W. K. Laskey, M. P. O'Laughlin, S. Oesterle, et al. American College of Cardiology/Society for Cardiac Angiography and Interventions Clinical Expert Consensus Document on Cardiac Catheterization Laboratory Standards: A report of the American College of Cardiology Task Force on Clinical Expert Consensus Documents endorsed by the American Heart Association and the Diagnostic and Interventional Catheterization Committee of the Council on Clinical Cardiology of the AHA J. Am. Coll. Cardiol., June 15, 2001; 37(8): 2170 - 2214. [Full Text] [PDF]
|
|
|
|
|
|
D. R. Holmes Jr, P. B. Berger, K. N. Garratt, V. Mathew, M. R. Bell, G. W. Barsness, S. T. Higano, D. E. Grill, L. N. Hammes, and C. S. Rihal Application of the New York State PTCA Mortality Model in Patients Undergoing Stent Implantation Circulation, August 1, 2000; 102(5): 517 - 522. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Lindsay Jr., E. E. Pinnow, and A. D. Pichard Frequency of major adverse cardiac events within one month of coronary angioplasty: a useful measure of operator performance J. Am. Coll. Cardiol., December 1, 1999; 34(7): 1916 - 1923. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. G. Ellis, V. Guetta, D. Miller, P. L. Whitlow, and E. J. Topol Relation Between Lesion Characteristics and Risk With Percutaneous Intervention in the Stent and Glycoprotein IIb/IIIa Era : An Analysis of Results From 10 907 Lesions and Proposal for New Classification Scheme Circulation, November 9, 1999; 100(19): 1971 - 1976. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. D. McGrath, D. J. Malenka, D. E. Wennberg, S. J. Shubrooks Jr., W. A. Bradley, J. F. Robb, M. A. Kellett Jr., T. J. Ryan Jr., M. J. Hearne, B. Hettleman, et al. Changing outcomes in percutaneous coronary interventions: A study of 34,752 procedures in Northern New England, 1990 to 1997 J. Am. Coll. Cardiol., September 1, 1999; 34(3): 674 - 680. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Moscucci, G. T. O'Connor, S. G. Ellis, D. J. Malenka, J. Sievers, E. R. Bates, D. W. M. Muller, S. W. Werns, E. K. Rogers, D. Karavite, et al. Validation of risk adjustment models for in-hospital percutaneous transluminal coronary angioplasty mortality on an independent data set J. Am. Coll. Cardiol., September 1, 1999; 34(3): 692 - 697. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. G. Ellis, D. Miller, T. F. Keys, K. Brown, R. Ellert, G. Howell, A. M. Lincoff, and E. J. Topol Comparing physician-specific two-year patient outcomes after coronary angiography: Methodologic issues and results J. Am. Coll. Cardiol., April 1, 1999; 33(5): 1278 - 1285. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. Kastrati, F.-J. Neumann, and A. Schomig Operator volume and outcome of patients undergoing coronary stent placement J. Am. Coll. Cardiol., October 1, 1998; 32(4): 970 - 976. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. C. Block, E. C. Peterson, R. Krone, K. Kesler, E. Hannan, G. T. O'Connor, and K. Detre Identification of variables needed to risk adjust outcomes of coronary interventions: evidence-based guidelines for efficient data collection J. Am. Coll. Cardiol., July 1, 1998; 32(1): 275 - 282. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. G. Jollis and P. S. Romano Pennsylvania's Focus on Heart Attack -- Grading the Scorecard N. Engl. J. Med., April 2, 1998; 338(14): 983 - 987. [Full Text]
|
|
|
|
|
|
S. G. Ellis, W. Weintraub, D. Holmes, R. Shaw, P. C. Block, and S. B. King III Relation of Operator Volume and Experience to Procedural Outcome of Percutaneous Coronary Revascularization at Hospitals With High Interventional Volumes Circulation, June 3, 1997; 95(11): 2479 - 2484. [Abstract] [Full Text]
|
|
|
|
|
|
J. G. Jollis, E. D. Peterson, C. L. Nelson, J. A. Stafford, E. R. DeLong, L. H. Muhlbaier, and D. B. Mark Relationship Between Physician and Hospital Coronary Angioplasty Volume and Outcome in Elderly Patients Circulation, June 3, 1997; 95(11): 2485 - 2491. [Abstract] [Full Text]
|
|
|
|
|
|
J. A. Bittl Advances in Coronary Angioplasty N. Engl. J. Med., October 24, 1996; 335(17): 1290 - 1302. [Full Text] [PDF]
|
|
|
|
|
|
R. M. Califf, J. G. Jollis, and E. D. Peterson Operator-Specific Outcomes : A Call to Professional Responsibility Circulation, February 1, 1996; 93(3): 403 - 406. [Full Text]
|
|