(Circulation. 2004;109:641-646.)
© 2004 American Heart Association, Inc.
Clinical Investigation and Reports |
From the Duke Clinical Research Institute and Departments of Medicine (K.S.P., J.G., R.M.C., R.A.H.) and Genomics and Biostatistics (V.H.), Duke University Medical Center, Durham, NC; Department of Statistics (A.A.T., M.D.), North Carolina State University, Raleigh, NC; Baylor College of Medicine/Texas Medical Center (N.S.K.), Houston, Tex; Erasmus Medical Center (E.B.), Rotterdam, the Netherlands; and the University of Alberta (W-C.C., P.W.A.), Edmonton, Alberta, Canada.
Correspondence to Karen S. Pieper, MS, Duke Clinical Research Institute, PO Box 17969, Durham, NC 27715. E-mail karen.pieper{at}duke.edu
Received June 23, 2003; revision received October 23, 2003; accepted October 31, 2003.
| Abstract |
|---|
|
|
|---|
Methods and Results We describe the problems with these analytical techniques and demonstrate that different approaches to the question can result in opposing answers.
Conclusions Clinical-practice decisions and practice guidelines should be based on overall trial results and not analyses of post-randomization subgroups.
Key Words: trials glycoproteins coronary disease
| Introduction |
|---|
|
|
|---|
The purpose of the present study is not to provide insight into the relative treatment effect in these PCI groups but rather to evaluate the techniques used in the literature in drawing conclusions about this treatment difference. More importantly, we will discuss problems with some of the analytical approaches described in the literature to address this question and how conclusions can widely vary, according to the approach.
| Improper Subgroups |
|---|
|
|
|---|
Several articles have been written about the pros and cons of subgroup analyses.1119 This discussion focuses on comparisons of the randomized treatment effect between subgroups. In the examination of randomized treatment differences, the improper subgroup represents a special case.11 The improper subgroup is based on a characteristic measured after randomization, and the randomized treatment may influence whether the patient enters the subgroup. In the cardiovascular literature, improper subgroups themselves can represent end points of 2 types: (1) clinical complications such as congestive heart failure, arrhythmias, stroke, or myocardial infarction (MI), and (2) end points based on physician discretion. These include procedures or therapies such as PCI, coronary artery bypass grafting (CABG), concomitant medications, or blood transfusions. Special care needs to be taken when interpreting results from this sort of subgroup.
Issues
For this analysis, we will define the PCI group as anyone who undergoes PCI during the time specified by the protocol in which the randomized treatment is to be administered. We will define the medical therapy group as anyone not undergoing PCI during the time of randomized therapy.
The randomized treatment undertaken in the trial may affect how assignments are made to study groups. For example, the use of a GP IIb/IIIa inhibitor may reduce early ischemic events, making PCI unnecessary. Conversely, ischemic events may trigger angiography and subsequent intervention (PCI or CABG); thus, the placebo patients might be more likely to undergo PCI than treated patients.
Many of these patients, particularly in the United States, would likely have received angiography routinely in any event. Angiographic appearances may be changed by the active treatment, and these findings can determine procedure decisions. The patients without coronary disease will not receive a PCI, potentially creating lower risk for the medical arm and diluting the absolute treatment effect.
The 2 treatment arms within the PCI subgroup may not be comparable (Table 1). Demographics of PCI and medically treated patients are rarely similar. The patients in the PCI group may be different from the medically treated patients for the same treatment arm. The PCI may have been more event driven in the placebo group. Comparing differences between treatment and placebo arms in the medically treated subgroup with differences between treatment and placebo arms in the PCI subgroup would be inappropriate.
|
In the above scenario, an additional problem is that of not giving consideration to the timing of events relative to PCI. If the end point for the trial is death or MI and the early ischemic event driving the PCI is an MI, then an end point is assigned to the PCI group before the patient has even undergone the procedure. The event actually occurred during the time of medical management.
The timing of events has been disregarded in many analyses, leading to the incorrect assumption that any patient who undergoes PCI during randomized therapy belongs in the PCI group and all others are medically treated. Randomized treatment comparisons are made within the PCI and medically treated groups separately. The absolute or relative treatment differences and associated P values also are compared between PCI and medically treated patients. This is the most problematic analytical method. Adding to this are the problems of selection bias, assignment of events to the wrong group, different timing of the PCI relative to the randomized treatment, inadequate statistical power, and multiple-comparison issues.
A slightly better approach is the same-assignment method but excluding any event occurring before PCI. This solves the problem of events during medical therapy being assigned to PCI patients, but it does not correctly assign these events to the medical group. To do that, one can apply survival-modeling techniques using all of the patients and censoring on PCI. Patients are followed up until a PCI is performed, an event occurs, or the end of the study, whichever occurs first. In this way, one estimates the treatment effect during the medical management period across all patients.
One also can make the PCI a time-dependent covariate to account for its timing relative to the start of randomized treatment.20 The time-dependent covariate considers each patient to be in the no-PCI group until PCI is performed. From then on, according to the model, the patient is considered part of the PCI group. The PCI patients are considered to be medically managed from randomization until the PCI is started, and they are compared with patients who have already undergone PCI during this time. Once a patient undergoes PCI, that patient is moved to the PCI group and then is compared with patients who have not yet undergone PCI.
These methods do not allow for direct comparisons of treatment effects between the 2 groups, nor do they correct for selection biases. For selection bias, one must adjust for differences in baseline characteristics, for the propensity to receive a PCI, or both.
A propensity score represents the likelihood that a patient will undergo PCI based on his or her baseline profile.21 If the score can fully explain this decision process, then the bias of patient selection for PCI is removed when the propensity score is added into the model. If the assumptions hold, then inferences actually can be made from the subgroups. Similarly, if one adjusts for all baseline factors associated with selection bias and outcomes, then valid inferences also can be made. If data needed to determine the appropriateness for PCI are unavailable, then the propensity score will not adequately control for selection bias. Similarly, if confounders are not available, adjusting only for baseline factors collected will not allow for valid inferences. It is likely that this problem will exist, because all factors that cause a physician to refer to PCI and factors that are associated with bad outcomes cannot be collected.
An interaction test of PCI with treatment can be used in these models to evaluate whether the treatment effect is statistically different for PCI patients than for medically treated patients. With this, one can make direct comparisons of the treatment effects between the 2 groups, but the test has only limited statistical power.
Example
The Platelet IIb/IIIa in Unstable angina: Receptor Suppression Using Integrilin Therapy (PURSUIT) trial was an international study of patients with ACS but without persistent ST-segment elevation on hospital arrival.22 Its primary results showed that eptifibatide was more effective than placebo in reducing death or MI in these patients at 30 days (P=0.042). This study involved a heterogeneous population representing 726 hospitals in 28 countries. Many of these patients did not undergo PCI during the treatment period. The study was designed to answer the question of whether eptifibatide was beneficial for all patients with ACS, but the results did not convince many cardiologists that eptifibatide should be used outside of PCI.
Kleiman et al3 evaluated the question of differential treatment effect between patients who underwent PCI <72 hours after randomization (the expected duration of randomized treatment) and those who did not. Disregarding timing, treatment outcomes were more favorable for patients who underwent PCI (Table 2). In patients with PCI, the incidence of the primary end point (death or MI at 30 days) was 11.6% for patients receiving eptifibatide versus 16.7% for placebo-treated patients (P=0.010). In patients not undergoing early PCI, the treatment effect was minimal, 14.6% versus 15.6% (P=0.232).
|
Table 3 provides insight into this difference.22 Far more patients in the placebo arm of the trial had an early MI and then underwent PCI (5.5% in the placebo group versus 1.7% in the eptifibatide group; P<0.001). Although these events were attributed to the PCI arm in Table 1, they actually occurred during medical management. The treatment effect considering events that occurred only after PCI was no longer statistically significant (12.4% for placebo versus 10.2% for eptifibatide; P=0.235).
|
The censored analysis results were that eptifibatide was again effective during medical management (P=0.035 censoring on PCI alone; P=0.058 censoring on PCI or CABG).3 A Cox proportional hazard model with PCI as a time-dependent covariate was also considered.3 The interaction term for a different treatment effect with PCI versus medical management was not statistically significant (P=0.635), even after also including a propensity-score adjustment (P=0.634), indicating no statistical basis for concluding that treatment effects differed between patients who underwent early PCI and those who did not.
Results from this one study show a wide variationfrom a strong treatment effect only in the early-PCI patients to no statistically significant difference in treatment effect between PCI and non-PCI patients. Addressing selection biases and properly assigning outcomes to subgroup categories made a significant difference in the results.
An Alternative Approach
No clinical trial has randomized patients in a factorial design to (1) receive a GP IIb/IIIa inhibitor versus placebo and (2) undergo early PCI versus medical management. A placebono PCI arm would be unethical, given the knowledge that PCI with a GP IIb/IIIa inhibitor is more effective than a PCI with placebo. Trials of applying an early invasive strategy conferred better clinical outcomes than those applying a conservative strategy.2325 These trials all used a 1-drug treatment strategy, however, so the drug effects between the 2 strategies could not be evaluated. The C7E3 Fab AntiPlatelet Therapy in Unstable REfractory angina (CAPTURE) study was the one study that randomly assigned patients to placebo versus a GP IIb/IIIa inhibitor and studied the treatment effect in the 24 hours before a procedure, as well as in the period after the PCI.2
Given the literature to date, when should a GP IIb/IIIa inhibitor be given? Clinicians who believe that such treatment has little effect without PCI might choose to wait until a procedure has been planned to consider using the treatment. Those who believe that a treatment benefit exists even without PCI might choose to give it on the patients arrival to the hospital. With this approach, the patient would already be receiving a GP IIb/IIIa inhibitor if PCI becomes necessary but still would benefit if PCI were not performed.
Figure 1 illustrates the design of a clinical trial testing this question. Patients with ACS would be randomized to receive either a GP IIb/IIIa inhibitor as soon as possible after hospital arrival (early-treatment arm) or receive placebo and be switched to a GP IIb/IIIa inhibitor once PCI is planned (delayed-treatment arm). Treatment with angiography or a procedure would occur at the discretion of the physician for both treatment arms. In the delayed-treatment arm, a GP IIb/IIIa inhibitor would be started if and when the physician decided to perform PCI. Otherwise, the patient would continue placebo treatment until the end of the treatment period. Note that some patients will have an event before the PCI can occur.
|
Although such a trial has not been performed, other existing trial results provide insight. The patients who were randomly assigned to receive eptifibatide in PURSUIT would be similar to patients in the arm of the proposed study for whom active treatment would be administered on hospital arrival. The patients who were randomized to receive placebo would be similar to those in the delayed arm of the proposed study, except when a PCI was given. For these PCI patients, we can estimate how many events would have been avoided had a GP IIb/IIIa inhibitor been used.
Figure 2 shows the results of the proposed study using data from the PURSUIT trial. The overall event rate in the delayed-treatment arm was 15.7% versus 14.2% in the early-treatment arm. Most of the patients in the delayed-treatment arm did not undergo an early PCIonly 561 of the 4739 placebo-treated patients. An additional 61 patients experienced an event before PCI. Boersma et al26 found a 23% reduction in odds in the PCI arm across all of the GP IIb/IIIa inhibitor trials. Assuming 25% of the 43 post-PCI events would have been prevented with GP IIb/IIIa inhibition before the PCI, the overall event rate for the delayed-treatment arm would barely change (15.5%). Although the assumption is somewhat unrealistic, had all 43 events been prevented, the event rate in this arm would still have been slightly higher (14.8%) than that of the early-treatment group.
|
Figure 3 illustrates the proposed study results using data from a recent meta-analysis of all of the trials.26 Again, even if 25% of the events had been prevented by giving a GP IIb/IIIa inhibitor before PCI, the early-treatment arm would still have had a lower event rate than the delayed-treatment arm (10.8% versus 11.6%).
|
| Discussion |
|---|
|
|
|---|
Antman,28 in an editorial response to this study, concurred with the authors: "If a patient with an ACS is treated only medically using a GP IIb/IIIa inhibitor, the number of events prevented per 1000 patients treated is less than a quarter of that attained when the GP IIb/IIIa inhibitor is used in association with PCI" (p 1409). Both Roffi et al27 and Antman28 overlook the fact that all events occurring before the PCI were attributed to the PCI arm. Therefore, a large treatment benefit seen during the early hours of treatment was erroneously attributed to the PCI group and not the medical therapy group to which they actually belonged.
Antmans editorial additionally acknowledges the results of a previous meta-analysis of these same trials (Figure 3).26 The authors of this meta-analysis had noted important differences between patients chosen to undergo early PCI and those not chosen. First, they showed that, compared with the placebo patients, significantly fewer patients assigned to the active treatment arms had undergone early PCI (14.9% versus 13.2%; P=0.049). Among patients who did undergo early PCI, those treated with placebo (compared with active treatment) experienced significantly more MIs before the procedure occurred (OR, 0.70; 95% CI, 0.55 to 0.89). For the subgroup of patients who did not undergo an early PCI, the treatment difference was not statistically significant (OR, 0.95; 95% CI, 0.87 to 1.02). However, if all patients in the study were included, using the information during the medical management of each patient, then the treatment difference was statistically significant (OR, 0.92; 95% CI, 086 to 0.99).
Boersma et al1 had previously performed more extensive evaluations of this question by combining 2 of the trials from the meta-analysis, PURSUIT and the Platelet Receptor Inhibition in ischemia Syndrome Management in Patients Limited by Unstable Signs and symptoms (PRISM-PLUS) trial with the CAPTURE study. They found a 34% reduction in events during the medical-therapy phase (2.5% for active treatment versus 3.8% for placebo) and a 41% reduction during the first 48 hours after PCI (4.9% versus 8.0%).
| Conclusions |
|---|
|
|
|---|
There should always be heavy reliance on the overall results of a trial. After careful evaluation of the data, including consideration of timing and bias issues, any other results should be discussed in terms of being hypothesis-generating only. Clinical-practice decisions and practice guidelines should not be based on analyses of postrandomization subgroups.
| Acknowledgments |
|---|
| Footnotes |
|---|
Dr Armstrong has received research grants from Hoffman-LeRoche Laboratories, Eli Lilly & Co, and Centocor, Inc. Dr Califf has received research funding from Millennium Pharmaceuticals, Inc, Schering-Plough Research Institute, Merck & Co, Inc, Centocor Inc, and Eli Lilly & Co. Dr Harrington has received research grants from Millennium Pharmaceuticals, Inc, Schering-Plough Research Institute, Eli-Lilly & Co, Centocor Inc, and Hoffman-LeRoche Laboratories.
| References |
|---|
|
|
|---|
2. The CAPTURE Investigators. Randomized placebo-controlled trial of abciximab before and during coronary intervention in refractory unstable angina: the CAPTURE study. Lancet. 1997; 349: 14291435.[CrossRef][Medline] [Order article via Infotrieve]
3. Kleiman NS, Lincoff AM, Flaker GC, et al. Early percutaneous coronary intervention, platelet inhibition with eptifibatide, and clinical outcomes in patients with acute coronary syndromes. Circulation. 2000; 101: 751757.
4. Simoons ML, for the GUSTO IV-ACS Investigators. Effect of glycoprotein IIb/IIIa receptor blocker abciximab on outcome in patients with acute coronary syndromes without early coronary revascularisation: the GUSTO IV-ACS randomised trial. Lancet. 2001; 357: 19151924.[CrossRef][Medline] [Order article via Infotrieve]
5. Theroux P, Alexander J Jr, Dupuis J, et al. Upstream use of tirofiban in patients admitted for an acute coronary syndrome in hospitals with or without facilities for invasive management: PRISM-PLUS Investigators. Am J Cardiol. 2001; 87: 375380.[CrossRef][Medline] [Order article via Infotrieve]
6. The IMPACT II Investigators. Randomized placebo-controlled trial of effect of eptifibatide on complications of percutaneous coronary intervention. Lancet. 1997; 349: 14221428.[CrossRef][Medline] [Order article via Infotrieve]
7. The EPIC Investigators. Use of a monoclonal antibody directed against the platelet glycoprotein IIb/IIIa receptor in high-risk coronary angioplasty. N Engl J Med. 1994; 330: 956961.
8. The ESPRIT Investigators. Novel dosing regimen of eptifibatide in planned coronary stent implantation (ESPRIT): a randomised, placebo-controlled trial. Lancet. 2000; 356: 20372044.[CrossRef][Medline] [Order article via Infotrieve]
9. The EPISTENT Investigators. Randomised placebo-controlled trial and balloon-angioplastycontrolled trial to assess safety of coronary stenting with use of platelet glycoprotein-IIb/IIIa blockade. Lancet. 1998; 352: 8792.[Medline] [Order article via Infotrieve]
10. The RESTORE Investigators. Effects of platelet glycoprotein IIb/IIIa blockade with tirofiban on adverse cardiac events in patients with unstable angina or acute myocardial infarction undergoing coronary angioplasty. Circulation. 1997; 96: 14451453.
11. Yusuf S, Wittes J, Probstfield J, et al. Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. JAMA. 1991; 266: 9398.
12. Assmann SF, Pocock SJ, Enos LE, et al. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet. 2000; 355: 10641069.[CrossRef][Medline] [Order article via Infotrieve]
13. Bullpitt CJ. Subgroup analysis. Lancet. 1988; 2: 3134.[Medline] [Order article via Infotrieve]
14. Buyse ME. Analysis of clinical trial outcomes: some comments on subgroup analyses. Control Clin Trials. 1989; 10: 187S194S.[CrossRef][Medline] [Order article via Infotrieve]
15. Adams KF Jr. Post hoc subgroup analysis and the truth of a clinical trial. Am Heart J. 1998; 136: 753758.[CrossRef][Medline] [Order article via Infotrieve]
16. Stallones RA. The use and abuse of subgroup analysis in epidemiological research. Prev Med. 1987; 16: 183194.[CrossRef][Medline] [Order article via Infotrieve]
17. Schneider B. Analysis of clinical trial outcomes: alternative approaches to subgroup analysis. Control Clin Trials. 1989; 10: 17651865.
18. Rockette HE, Caplan RJ. Strategies for subgroup analysis in clinical trials. Cancer Res. 1988; 111: 4954.
19. Parker AB, Naylor CD. Subgroups, treatment effects, and baseline risks: some lessons from major cardiovascular trials. Am Heart J. 2000; 139: 952961.[CrossRef][Medline] [Order article via Infotrieve]
20. Kalbfleisch JD, Prentice RL. The Statistical Analysis of Failure Time Data. New York: John Wiley & Sons; 1980: 122127.
21. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983; 70: 4155.
22. The PURSUIT Investigators. Inhibition of platelet glycoprotein IIb/IIIa with eptifibatide in patients with acute coronary syndromes without persistent ST-segment elevation: a randomized, placebo-controlled, clinical trial. N Engl J Med. 1998; 339: 436443.
23. The FRISC-II Investigators. Long-term low-molecular-mass heparin in unstable coronary-artery disease: FRISC II prospective randomised multicentre study. Lancet. 1999; 354: 701707.[CrossRef][Medline] [Order article via Infotrieve]
24. Cannon CP, Weintraub WS, Demopoulos LA, et al. Comparison of early invasive and conservative strategies in patients with unstable coronary syndromes treated with the glycoprotein IIb/IIIa inhibitor tirofiban. N Engl J Med. 2001; 344: 18791887.
25. The RITA Investigators. Interventional versus conservative treatment for patients with unstable angina or nonST-elevation myocardial infarction: the British Heart Foundation RITA 3 randomised trial. Lancet. 2002; 360: 743751.[CrossRef][Medline] [Order article via Infotrieve]
26. Boersma E, Harrington RA, Moliterno DJ, et al. Platelet glycoprotein IIb/IIIa inhibitors in acute coronary syndromes: a meta-analysis of all major randomised clinical trials. Lancet. 2002; 359: 189198.[CrossRef][Medline] [Order article via Infotrieve]
27. Roffi M, Chew DP, Mukherjee D, et al. Platelet glycoprotein IIb/IIIa inhibition in acute coronary syndromes: gradient of benefit related to the revascularization strategy. Eur Heart J. 2002; 23: 14411448.
28. Antman EM. "I can see clearly now": a new view on the use of IV GP IIb/IIIa inhibitors in acute coronary syndromes. Eur Heart J. 2002; 23: 14081411.
This article has been cited by other articles:
![]() |
R. P. Giugliano, J. A. White, C. Bode, P. W. Armstrong, G. Montalescot, B. S. Lewis, A. van `t Hof, L. G. Berdan, K. L. Lee, J. T. Strony, et al. Early versus Delayed, Provisional Eptifibatide in Acute Coronary Syndromes N. Engl. J. Med., May 21, 2009; 360(21): 2176 - 2190. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. W. Hamm, H. Möllmann, J.-P. Bassand, and F. van de Werf CHAPTER 16 Acute Coronary Syndromes ESC Textbook of Cardiovascular Medicine, January 1, 2009; 2(1): med-9780199566990-chapter - med-9780199566990-chapter. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. V Hernandez, C. M Westerhout, E. W Steyerberg, J. P A Ioannidis, H. Bueno, H. White, P. Theroux, D. J Moliterno, P. W Armstrong, R. M Califf, et al. The authors' reply: Heart, October 1, 2007; 93(10): 1293 - 1293. [Full Text] [PDF] |
||||
![]() |
Authors/Task Force Members, J.-P. Bassand, C. W. Hamm, D. Ardissino, E. Boersma, A. Budaj, F. Fernandez-Aviles, K. A.A. Fox, D. Hasdai, E. M. Ohman, et al. Guidelines for the diagnosis and treatment of non-ST-segment elevation acute coronary syndromes: The Task Force for the Diagnosis and Treatment of Non-ST-Segment Elevation Acute Coronary Syndromes of the European Society of Cardiology Eur. Heart J., July 1, 2007; 28(13): 1598 - 1660. [Full Text] [PDF] |
||||
![]() |
K. W. Mahaffey and R. A. Harrington Optimal Timing for Use of Glycoprotein IIb/IIIa Inhibitors in Acute Coronary Syndromes: Questions, Answers, and More Questions JAMA, February 14, 2007; 297(6): 636 - 639. [Full Text] [PDF] |
||||
![]() |
P. W. Armstrong, W.-C. Chang, L. Wallentin, P. Goldstein, C. B. Granger, K. Bogaerts, T. Danays, F. Van de Werf, and for the ASSENT-3 and ASSENT-3 PLUS Investigators Efficacy and safety of unfractionated heparin versus enoxaparin: a pooled analysis of ASSENT-3 and -3 PLUS data. Can. Med. Assoc. J., May 9, 2006; 174(10): 1421 - 1426. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Glaser, H. A. Glick, H. C. Herrmann, and S. E. Kimmel The Role of Risk Stratification in the Decision to Provide Upstream Versus Selective Glycoprotein IIb/IIIa Inhibitors for Acute Coronary Syndromes: A Cost-Effectiveness Analysis J. Am. Coll. Cardiol., February 7, 2006; 47(3): 529 - 537. [Abstract] [Full Text] [PDF] |
||||
![]() |
SYNERGY Trial Investigators Enoxaparin vs Unfractionated Heparin in High-Risk Patients With Non-ST-Segment Elevation Acute Coronary Syndromes Managed With an Intended Early Invasive Strategy: Primary Results of the SYNERGY Randomized Trial JAMA, July 7, 2004; 292(1): 45 - 54. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. L. Petersen, K. W. Mahaffey, V. Hasselblad, E. M. Antman, M. Cohen, S. G. Goodman, A. Langer, M. A. Blazing, A. Le-Moigne-Amrani, J. A. de Lemos, et al. Efficacy and Bleeding Complications Among Patients Randomized to Enoxaparin or Unfractionated Heparin for Antithrombin Therapy in Non-ST-Segment Elevation Acute Coronary Syndromes: A Systematic Overview JAMA, July 7, 2004; 292(1): 89 - 96. [Abstract] [Full Text] [PDF] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Circulation Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2004 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |