Use and Misuse of Surrogate Outcomes in Arrhythmia Trials
Rigorous design and careful execution of clinical trials have led to major advances in cardiovascular medicine over the past few decades. A pivotal design feature of any trial is the choice of outcomes. We place great value on reduction of mortality and somewhat less on outcomes, such as myocardial infarction, that lead to major morbidity and death. Changes in quality of life and functional capacity are harder to measure but are clinically meaningful in many situations. On the other hand, surrogate outcomes have very low status in this hierarchy. These are outcomes that “stand in” for more clinically relevant outcomes. The key feature of a surrogate outcome is that unproven assumptions are necessary to connect a change in the surrogate to a change in the accepted clinical outcome. The problem of using surrogate outcomes has been highlighted by situations in which the assumptions have been subsequently discredited. In the early 1980s, on the basis of numerous studies that demonstrated an association between ventricular premature depolarization (VPD) frequency and mortality in post-myocardial infarction (MI) patients, VPD suppression became an accepted method for antiarrhythmic drug selection in clinical practice. Even though VPDs were usually asymptomatic, VPD suppression served as a primary outcome measure in clinical trials that resulted in regulatory approval of antiarrhythmic drugs.1 The underlying assumption that VPD suppression was associated with a reduction in mortality was totally upended when the Cardiac Arrhythmia Suppression Trial (CAST) demonstrated that several drugs, which were highly effective at VPD suppression, actually increased mortality after MI.2
Article p 776
It is not always possible or desirable to perform a full-scale trial with the most highly clinically relevant outcomes. Surrogate outcomes are particularly useful in the early stages of development of a new drug or device when the main goal is to prove a concept or to determine the dose that is most likely to be effective and safe. Demonstrating that cardiac resynchronization therapy improved cardiac performance and then showing reduced symptoms of heart failure provided a justification to perform large, pivotal trials evaluating the efficacy of this therapy to reduce mortality.3 Moderately sized, multidose trials of new oral anticoagulants in patients at risk of postoperative venous thromboembolism provide useful information about which doses provide the best balance of safety and efficacy against one type of thrombosis. Information provided relatively efficiently in this clinical setting can provide some guidance for the design of the megatrials required to demonstrate efficacy and safety for stroke prevention in atrial fibrillation.4,5 Thus, surrogate outcomes are useful in numerous settings, but we must be wary of our assumptions and test them when possible.
Implantable cardioverter-defibrillator (ICD) therapy uses high-voltage shocks to terminate life-threatening episodes of ventricular tachycardia (VT) and ventricular fibrillation (VF). Although the goal of therapy has always been primarily to reduce mortality, there has been interest in using ICD shocks as a surrogate outcome for mortality. Before the publication of the results of randomized mortality trials, this approach gained some traction, and numerous articles reported “improved” outcomes based on the assumption that patients who received shocks would have died if the ICD had not been implanted.6 This approach was generally recognized as a problem because early-generation devices could not distinguish between shocks delivered for potentially lethal VT and VF and those delivered, inappropriately, for supraventricular tachycardias (SVT). ICDs are designed to err on the side of delivering therapy in the presence of tachycardia, and therefore inappropriate shocks for SVT are very common. A major advance in ICD technology has addressed this particular problem. Modern ICDs are now able to record the actual ECG activity occurring during shock episodes, which greatly increases our ability to differentiate between shocks delivered for VT and VF from inappropriate shocks, especially when both atrial and ventricular electrograms are available. In light of this advance in technology, can we use electrogram-documented shocks delivered for VT or VF as a surrogate outcome for arrhythmic death?
The article in this issue of Circulation by Ellenbogen et al,7 on behalf of the Defibrillators in Nonischemic Cardiomyopathy Treatment Evaluation (DEFINITE) investigators, indicates once again that our assumptions need to be carefully questioned. Their report is based on the results of the DEFINITE trial, in which patients with advanced nonischemic cardiomyopathy were randomized to receive or not receive an ICD. Deaths were categorized as to whether or not they were due to a ventricular arrhythmia (arrhythmic death). ICD shocks were collected and categorized by an adjudication committee as appropriate or not on the basis of stored cardiac electrograms. If randomization in DEFINITE was successful, then we expect that the patients in the 2 treatment groups (ICD and standard) will be similar and will have the same risk of having an arrhythmic event during follow-up. ICD therapy does not prevent episodes of potentially fatal arrhythmic events; rather, it terminates them if they occur. Thus, one would expect that VT/VF episodes should occur at similar rates in the 2 groups. There were 15 fatal arrhythmic events (6.6%) in the standard therapy group compared with 1 (0.4%) in the ICD group. If randomization was successful, the excess in fatal arrhythmic events in the standard treatment group should be balanced by a similar number of appropriate shocks for VT and VF occurring in the ICD group. However, the key observation of this article is that the number of appropriate shocks occurring in the ICD group far exceeds the expected number. If all appropriate shocks were considered equivalent to arrhythmic deaths, then the rate of arrhythmic death in the ICD group would be more than double the rate in the standard group (P=0.013). This observation challenges the idea that episodes of VT and VF that are treated with an ICD shock would have resulted in death if the ICD had not been present. It indicates that many shocks for VT and VF are not preventing death and that even appropriate ICD shocks are not a good surrogate for sudden death.
Next, the authors extend their analysis by evaluating the rates of syncope between the 2 groups. Although there are many causes of syncope, in patients with advanced cardiomyopathy, at least some of the episodes are due to VT and VF. If the excess of appropriate shocks in the ICD patients is due to episodes of VT/VF that would have terminated spontaneously, one would expect an excess in syncope in the standard therapy group that would counterbalance the excess in appropriate shocks in the ICD group. Indeed, there are more syncopal episodes in the standard group (8.7%) than in the ICD group (5.2%). Thus, the total number of arrhythmic episodes, when one includes arrhythmic deaths, appropriate shocks, and syncopal episodes, is similar between the 2 groups, satisfying our starting assumption that randomization produces 2 comparable groups of patients who have similar rates of arrhythmic episodes.
There are several assumptions implicit in this clever and interesting analysis that we should examine critically. These are as follows: (1) Randomization has produced 2 groups of patients that have comparable rates of VT/VF; (2) the intervention tested (ICD therapy) does not affect the rates of ventricular arrhythmia; (3) it is possible to reliably categorize the cause of death as arrhythmic or nonarrhythmic; (4) it is possible to reliably adjudicate whether the shocks were delivered for VT/VF or not; and (5) nonsustained VT episodes typically cause syncope. Randomization, when done properly, is highly likely to yield groups of patients that are comparable, especially when numbers are large. In this study baseline patient characteristics are similar, and there is no reason to suspect that randomization was not successful. Additionally, there is no reason to think that ICD therapy will affect the intrinsic rate of spontaneous arrhythmic events, although differences in cointerventions, such as antiarrhythmic drugs, could occur in this unblinded study. Categorization of cause of death into arrhythmic or nonarrhythmic mechanisms, based on the circumstances of death, cannot be readily validated. However, categorization of deaths in this way has yielded results that are consistent with our understanding of the manner in which antiarrhythmic therapies reduce mortality. For example, each of the 3 secondary prevention trials of ICD therapy categorized deaths as arrhythmic or nonarrhythmic. The results of meta-analysis of the 3 trials were that the ICD reduced arrhythmic death by 50% but had no effect on nonarrhythmic death.8 The fact that this is exactly how we expect the ICD to reduce mortality supports the validity of this method of clinical categorization of death. Differentiation of VT/VF from SVT by means of ICD stored electrograms can be challenging, especially when only ventricular electrograms are available, as was the case with the single-chamber devices used in DEFINITE. Eighteen of 174 arrhythmia episodes receiving shocks remained undetermined in this study, and perhaps the committee incorrectly classified some episodes. Finally, at least some syncopal episodes in this population are likely due to nonsustained VT, and syncopal episodes due to nonarrhythmic causes should be evenly distributed between the 2 groups. Because the assumptions used in the analysis are reasonable, we should consider what we have learned from it.
Many episodes of rapid VT and VF that lead to shock therapy from ICDs would not have resulted in death, presumably because they would have terminated spontaneously. The authors correctly note that this conclusion applies primarily to patients with nonischemic cardiomyopathy and that it may not be true in patients with ischemic heart disease. Clearly, this should be examined in existing databases of patients with ischemic cardiomyopathy. For now, one must be cautious in using an ICD therapy as a surrogate for a bad outcome. On the other hand, survival without a shock or antitachycardia pacing therapy for many years leads to a reasonable conclusion that the device was not useful. Because there is value in being able to use ICD telemetry and shock information as a surrogate for arrhythmic deaths, in future studies it would be useful to determine whether there are rates of tachycardia or morphologies of VT/VF that are a more suitable surrogate for arrhythmic death. Finally, although the ICD shock is not a useful surrogate for arrhythmic death, shocks are very distressing to most patients and are thus a valid outcome in trials that evaluate treatments to reduce ICD shock frequency to improve quality of life. Recent randomized clinical trials have demonstrated that antiarrhythmic drug therapies9 and improved methods of device programming10 significantly reduce ICD shock frequency, which should lead to improved patient outcomes.
The opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.
Abraham WT, Hayes DL. Cardiac resynchronization therapy for heart failure. Circulation. 2003; 108: 2596–2603.
Albers GW, Diener HC, Frison L, Grind M, Nevinson M, Partridge S, Halperin JL, Horrow J, Olsson SB, Petersen P, Vahanian A, for the SPORTIF Executive Steering Committee for the SPORTIF V Investigators. Ximelagatran vs warfarin for stroke prevention in patients with nonvalvular atrial fibrillation: a randomized trial. JAMA. 2005; 293: 690–698.
Grimm W, Flores BT, Marchlinski FE. Shock occurrence and survival in 241 patients with implantable cardioverter-defibrillator therapy. Circulation. 1993; 87: 1880–1888.
Ellenbogen KA, Levine JH, Berger RD, Daubert JP, Winters SL, Greenstein E, Shalaby A, Schaechter A, Subacius H, Kadish A. Are implantable cardioverter defibrillator shocks a surrogate for sudden cardiac death in patients with nonischemic cardiomyopathy? Circulation. 2006; 113: 776–782.
Connolly SJ, Hallstrom AP, Cappato R, Schron EB, Kuck KH, Zipes DP, Greene HL, Boszor S, Domanski M, Follmann D, Gent M, Roberts RS. Overview of the implantable cardioverter secondary prevention trials. Eur Heart J. 2000; 21: 2071–2078.
Connolly SJ, Dorian P, Roberts RS, Gent M, Bailin S, Fain ES, Thorpe K, Champagne J, Talajic M, Coutu B, Gronefeld GC, Hohnloser SH. Comparison of β-blockers, amiodarone plus β-blockers, or sotalol for prevention of shocks from implantable cardioverter defibrillators: the OPTIC Study: a randomized trial. JAMA. 2006; 295: 165–171.
Wathen MS, DeGroot PJ, Sweeney MO, Stark AJ, Oterness MF, Adkisson WO, Canby RC, Khalighi K, Machada C, Rubenstein DS, Volosin KJ. Prospective randomized multicenter trial of empirical antitachycardia pacing versus shocks for spontaneous rapid ventricular tachycardia in patients with implantable cardioverter defibrillators. Circulation. 2004; 110: 2591–2596.