Gap Between Clinical Trials and Clinical Practice
Lessons From the Bypass Angioplasty Revascularization Investigation (BARI)
McGuire and colleagues1 present interesting data and a careful analysis of how physicians respond to clinical trial results. Utilizing data from the National Cardiovascular Network and surveys of investigators at 13 sites, the authors considered the impact of the original Bypass Angioplasty Revascularization Investigation (BARI) randomized trial results,2 and the National Heart, Lung, and Blood Institute Clinical Alert3 on the use of percutaneous coronary interventions (PCI) and coronary artery bypass surgery (CABG) for patients with treated diabetes. Their conclusion is that the BARI results had no significant impact on revascularization treatment selection in the BARI-eligible patients with diabetes. The implications of this negative result are of real concern, but some caution in generalizing the conclusions is appropriate.
See p 1864
The study provokes the pertinent question, “Why do doctors do what they do?” A simple answer is not possible because numerous and diverse factors influence decision-making. However, the acceptance of various research outcomes depends at least to some extent on the quality and magnitude of the clinical evidence, the relevance of the research results to current practice, and the incentives for adopting changes in practice.
Clinical Trial Results
The BARI randomized clinical trial was designed to test the hypothesis that an initial strategy of percutaneous transluminal coronary angioplasty (PTCA) did not compromise 5-year survival compared with CABG for selected patients with multivessel coronary artery disease (CAD) requiring revascularization.4 The study concluded that there was no significant difference between CABG and PTCA with regard to 5-year survival in the overall population. However, in the subgroup of patients with treated diabetes, the study found a clinically and statistically significant survival benefit favoring CABG (81% CABG versus 65% PTCA, P=0.003).
The randomized clinical trial is the gold standard of evidence-based medicine, and the BARI study was properly designed and implemented. This National Institutes of Health-funded study included top academic institutions and researchers. The procedural outcomes were outstanding for their time, and the study patient follow-up was excellent. Treated diabetes was not a prespecified subgroup in the original protocol but was identified early in the trial by the Data and Safety Monitoring Board to be monitored for safety reasons, and rigorous criteria for statistical significance (P<0.005) were established for detecting a treatment difference in the patients with treated diabetes. The survey by McGuire et al found that 76% of clinicians thought that the BARI results were valid, perhaps indicating that the lack of specifying the subgroup of treated diabetes a priori did not influence the interpretation of the BARI results for patients with treated diabetes. Nonetheless, the conclusions from the BARI study results sent a mixed message to the cardiology community: PTCA was satisfactory for selected multivessel patients without diabetes, but CABG was superior for multivessel patients with diabetes. It is likely that a study with a uniform conclusion for the entire study population would have a stronger and more rapid impact on clinical practice.
Generalizability and Relevance to Clinical Practice
Fundamental to the design of BARI was the requirement that all patients entered into the randomization would be anatomically suitable for both CABG and PTCA. During the planning phase of the trial, the BARI investigators thoroughly discussed the expanding boundaries of PTCA use and concluded that more difficult multivessel patients were to be included in the trial consistent with good judgment and based on local experience of the participating sites. Even though two-thirds of the clinically eligible patients screened in BARI were excluded because of anatomic “unsuitability” for PTCA, it appears that those randomized to PTCA in BARI represented more complex PTCA than was usual practice in most laboratories at that time.
The BARI study also included a registry of 2010 patients who were determined to be eligible for the clinical trial but refused randomization. Among the BARI registry patients with treated diabetes, those with the most extensive disease were referred for CABG, whereas those selected for PTCA (61% of patients with diabetes received PTCA) had a lower risk angiographic profile; survival outcomes were comparable in these 2 non-randomized treatment groups.5 It seemed that physicians could identify those patients with diabetes who would do well with PTCA. Similar findings have been noted in other observational studies after publication of the BARI results. Thus, the results of the study by McGuire et al1 might be explained by the fact that revascularization practices for patients with diabetes were already consistent with the combined conclusions of the BARI randomized trial and registry. The patients with the most severe symptoms were being referred for CABG, and the most aggressive use of PTCA as practiced in the BARI randomized cohort was not usual practice in most laboratories at that time. Although some may be critical of the attempt to explore the boundary for more aggressive use of PTCA in BARI, it deserves emphasis that for patients without treated diabetes, this more aggressive use of PTCA as an initial strategy for a selected group of patients with multivessel disease was found to be safe. Survival curves at 5 and 10 years are superimposable for the PTCA and CABG treatment groups for patients without treated diabetes, despite less complete revascularization and more frequent repeat revascularization procedures. Unfortunately, those with treated diabetes did not share this tolerance for more aggressive use of PTCA, highlighting the unique cardiac consequences of the adverse metabolic milieu that complicates all interventions in patients with diabetes mellitus.
A more optimistic interpretation of the study by McGuire et al1 might be justified if one analyzes the use of PCI in patients with diabetes compared with patients without diabetes. Annual absolute numbers of PCI and CABG procedures in the United States and PCI as a percentage of all between the years 1990 and 2000 are presented in the Table.6 The dramatic increase in the proportion of PCI in the entire population of patients having coronary revascularization procedures is in striking contrast to the 28.6% versus 26.8% declining trend (P=0.06) presented by McGuire et al for PCI use before and after BARI in patients with diabetes. Thus, the fact that there was no increase in PCI use in the National Cardiovascular Network among patients with treated diabetes may have been influenced by the release of the BARI trial results and Clinical Alert.
Timeliness of Clinical Trials
When designing a study in a field of rapidly developing technology such as coronary revascularization, there is always a conflict between the desire to compare long-term clinical outcomes and to present these treatment comparisons in a “timely manner.” Clearly, if one of the protocol treatments is viewed as “obsolete” when the trial is completed, the conclusions are bound to have a diminished impact on clinical practice. On the other hand, the danger of using surrogate outcomes to promptly compare treatments has been observed repeatedly; as an example, antiarrhythmic drugs were approved and widely used because of their proven ability to suppress arrhythmias but were later shown in the Cardiac Arrhythmia Suppression Trial (CAST) to have a strong adverse effect on survival. PCI operators traditionally judge new procedures in terms of 30-day, 90-day, and 6-month endpoints. Studies that rely on short-term endpoints that are dominated by “traditional” PCI outcomes, such as target vessel revascularization (TVR), may miss important treatment effects on survival that may not be evident for several years. Given that the unique biology of CAD in patients with type 2 diabetes, which is characterized by more rapid progression, thrombophilia, and inflammation, the true impact of PCI requires longer duration of follow-up in this population. Although we agree that clinical trial results should be published as quickly as possible, it is also critical that studies that test long-term clinical efficacy not be abandoned because of the concern that technology is constantly evolving. We also believe there is an important role for serial registries such as the National Heart, Lung, and Blood Institute Dynamic Registry for PCI.
It takes time for clinicians to change their practice behavior based on new research results. The results must be released and disseminated, and the clinicians must discuss and digest the data. Moreover, unless the results are extraordinarily dramatic, clinicians generally require an accumulation of supportive evidence from other studies before they change their practice patterns. In the study by McGuire et al,1 91% of those surveyed were aware of the BARI results; thus, awareness of results does not seem to be an issue. Moreover, regardless of whether BARI influenced practice, its results have triggered appropriate a priori analyses of outcomes in patients with diabetes in the major PCI trials that have followed, such as Bypass Angioplasty Revascularization Investigation 2 Diabetes (BARI 2D), the Arterial Revascularization Therapies Study group (ARTS), and other studies.
The variation in the use of PCI among the clinical centers most pointedly provokes the question posed by this editorial: Why do doctors do what they do? Most physicians truly make every effort to provide what is in the best interest of their patients based on their understanding of current evidence. Patients’ or family members’ desires may also influence decisions regarding treatment selection. When data are discordant, arbitrary decisions may need to be made on the basis of best practice in a given environment. We agree with McGuire et al1 that this variation is the most worrisome issue raised by the current study, wherein rates of PCI in BARI eligible patients with treated diabetes ranged from 4.3% to 56.6%, with a surprisingly similar range among patients with triple vessel disease. The variation in stent use is of particular concern, as 80% of the clinicians felt that the BARI results were obsolete probably because of an absence of stents in the original BARI trial. It is indeed difficult to understand how one would justify balloon angioplasty (POBA) in patients with treated diabetes and triple vessel disease.
As McGuire et al point out, economic realities and potential conflicts of interest must be mentioned in trying to understand these findings. George Bernard Shaw in the Preface to the Doctors Dilemma7 expounds at length on the medical profession and in his own way characterized the conflict of interest for physicians, saying “[t]hat any sane nation, having observed that you could provide for the supply of bread by giving bakers a pecuniary interest in baking for you, should go on to give a surgeon a pecuniary interest in cutting off your leg is enough to make one despair of political humanity.” These issues of conflicts-of-interest go beyond individual physicians and include institutions in which the procedures are performed. In today’s reimbursement environment, high technology is the thing that pays. Dr Richard Gorlin in his Bishop Lecture frankly states, “There is an inherent conflict of interest for both practitioners and hospital in the performance of these interventional procedures. They are remunerative. Thus, we must be scrupulously conscientious in ensuring that we are not performing them to fill our hospital beds and to support our practices. Self-referral is an obvious ethical mistake.”8
Finally, pharmaceutical and medical device companies greatly affect clinical practice. Practitioners generally embrace technologies that promise to deliver better outcomes and expand the patient pool for PCI. The degree to which treatment recommendations are supported by CME activities undoubtedly impacts clinical practice, and these activities are generally supported by industry. We must ask what the driving forces influencing a change in behavior for revascularization are. Who has an interest in these treatment decisions? In particular, there really is no validation for choosing CABG over PCI or, even in appropriate settings, aggressive medical therapy as an initial strategy.
The cardiology community must fully support the establishment of practice guidelines by respected leaders in the profession and ensure their implementation. The American College of Cardiology/American Heart Association Guidelines started in 1981 is a model of these activities. It is also clear that the implementation of practice guidelines remains a challenge. In response, the American College of Cardiology has launched the Guidelines Applied in Practice (GAP) Program to address the gap between clinical recommendations and the care that is delivered in practice. Programs such as these are needed to effectively disseminate important study results and to assess how to best persuade physicians to change behavior in response to clinical evidence.
A complex combination of factors impacts clinician treatment decisions. The quality of the clinical evidence is just one of many factors, and frequently, it is not the most important factor. McGuire and colleagues1 are to be commended for an important contribution to our understanding of how clinical trial results influence medical practice. It is imperative that research results are promptly disseminated and evaluated and that appropriate changes in clinical practice follow. Although judgment and opinion will always be needed in care of individual patients, our profession must lead in the evolution of decisions based on evidence. To do otherwise would be to compromise our professional ethic.
The opinions expressed in this editorial are not necessarily those of the editors or of the American Heart Association.
McGuire DK, Anstrom KJ, Peterson ED. Influence of the Bypass Angioplasty Revascularization Investigation National Heart, Lung, and Blood Institute Diabetic Clinical Alert on Practice Patterns: results from the National Cardiovascular Network Database. Circulation. 2003; 107: 1864–1870.
National Library of Medicine, National Heart, Lung, and Blood Institute. Clinical alert: bypass over angioplasty for patients with diabetes. Available at: http://www.nlm.nih.gov/databases/alerts/bypass_diabetes.html. Accessed March 14, 2003.
The BARI Protocol for the Bypss Angioplasty Revascularization Investigation. Circulation. 1991; 84 (suppl V): V1–V27.
Feit F, Brooks MM, Sopko G, et al. Long term clinical outcome in the BARI Registry: comparison with the randomized trial. Circulation. 2000; 101: 2795–2802.
Kozak LJ, Hall MJ, Owings MF. National Hospital Discharge Survey: 2000 Annual Summary with detailed diagnosis and procedure data. National Center for Health Statistics. Vital Health Stat. 2002;13. Available at: http://www.cdc.gov/nchs/data/series/sr_13/sr13_153.pdf. Accessed January, 2003.
Shaw GB. Doctors Dilemna, With Preface on Doctors. New York, NY: Dodd, Mead, and Company; 1941.