Response to Letters Regarding Article, “Bayesian Methods Affirm the Use of Percutaneous Coronary Intervention to Improve Survival in Patients With Unprotected Left Main Coronary Artery Disease”
Dr Diamond explores the important issue of relating statistical inference to clinical significance in our recent report.1 He raises a specific concern about the width of the 95% Bayesian credible interval of the odds ratio, ranging from 0.68 to 1.45, and questions whether the approach that we used actually affirms that percutaneous coronary intervention (PCI) should be considered as an alternative to coronary artery bypass (CABG) surgery for selected patients with unprotected left main coronary artery disease, as suggested by the Class IIa recommendation from the American Heart Association Foundation/American Heart Association.2
The Bayesian credible intervals in our analysis were similar to the classical confidence intervals and included the value 1.0, suggesting that the 1-year mortality after PCI or CABG was similar. The seemingly wide Bayesian credible intervals were the consequences of sample size. To illustrate this, we note from the data that the 1-year mortality rate after CABG was 6.56%1 and estimate that a sample size of 90 000 per treatment arm would be required to detect a relative mortality difference of 5% in the PCI arm (alpha = 0.05, power = 0.80). If the relative difference in mortality between CABG and PCI were 10%, we would still need more >20 000 patients in each arm. Based on sample sizes for the PCI arm of 2059 and for the CABG arm of 2515,1 it is not surprising that the posterior probability satisfying Dr Diamond’s proposed threshold was low.
Our analysis did not include a formal test of equivalence, but it produced no evidence to suggest that the 1-year mortality rates were different. With that said, we agree with Dr Diamond that thresholds for clinical significance need to be considered in the evidence–synthesis process. Treating the threshold as a variable, we used Bayesian methods to summarize the posterior probability of the treatment effects that are greater or less than the threshold to generalize the example calculations from Dr Diamond and observed that the 1-year mortality differences between PCI and CABG of ±5%, ±10%, ±20%, and ±30% were associated with posterior probabilities of 0.26, 0.49, 0.79, and 0.93.
Regarding the network meta-analysis, Drs Ye and Zhang raise concerns about the exchangeability or consistency among the individual studies used in our analysis.1 We note that exchangeability and consistency are 2 separate and distinct statistical concepts. The exchangeability assumption implies that the study-specific effects (eg, the odds ratios) are not identical across studies but instead arise from a common distribution. This assumption is far less restrictive than the assumptions adopted in a fixed-effects meta-analysis and would be violated if (1) there were knowledge that the odds ratios from some studies were truly larger than others and (2) we know which specific studies have the larger odds ratios. We have no reason to believe this is the case. Inconsistency in a network meta-analysis refers to different estimates of an indirect treatment comparison obtained from different pairs of treatments. In an ideal network meta-analysis of PCI versus medical therapy (MT) trials and CABG versus MT trials, patients in all 3 treatment groups should be balanced for such characteristics as age, sex, diabetes mellitus, extent of non–left main coronary artery disease, and acuity of presentation, and they should have received similar concomitant therapy with aspirin, statins, β-blockers, and angiotensin-converting enzyme inhibitors. We attempted multiple sensitivity analyses but had no access to patient-level data or to studies that directly compared PCI with MT. Using study-level means and proportions for age and sex, however, we found that patients in the PCI arms were older and more likely to be female than those in the CABG and MT arms of the trials.1
Inconsistency could arise because the individual studies are underpowered, the treatment strategies have heterogeneous effects, or the estimates are truly different or incoherent.3 The last reason presents a real problem, because we have no direct comparison of PCI with MT and thus our findings are based entirely on the model assumptions.
The concerns raised in the letters confirm that the evidence used for developing clinical practice guidelines is imperfect because of inconsistent methodologies and varied patient cohorts. Developing an informed prior probability remains a challenge for Bayesian methods but the overall process is arguably improved, because approaches that incorporate prior information and establish inferences based on model-based probability functions may provide a quantitative approach in a decision framework.4 Going forward, guideline development shall benefit from an integrated approach using both classical and Bayesian methods to translate trial data into clinical practice.
John A. Bittl, MD
Munroe Regional Medical Center
Yulei He, PhD
Centers for Disease Control
Alice K. Jacobs, MD
Department of Medicine, Section of Cardiology
Boston Medical Center, Boston, MA
Clyde W. Yancy, MD, FAHA
Department of Medicine, Division of Cardiology
Northwestern Feinberg School of Medicine, Chicago, IL
Sharon-Lise T. Normand, PhD
Department of Health Care Policy, Harvard Medical School
Department of Biostatistics, Harvard School of Public Health
We are grateful to Drs Donald Malec and Abera Wouhib for their helpful suggestions.
- © 2014 American Heart Association, Inc.
- Bittl JA,
- He Y,
- Jacobs AK,
- Yancy CW,
- Normand S-LT
- Levine GN,
- Bates ER,
- Blankenship JC,
- Bailey SR,
- Bittl JA,
- Cercek B,
- Chambers CE,
- Ellis SG,
- Guyton RA,
- Hollenberg SM,
- Khot UN,
- Mauri L,
- Mehran R,
- Moussa ID,
- Mukherjee D,
- Nallamothu BK,
- Ting HH
- Berger JO