Abstract 195: Challenges of Standardization in Evidence-Based Medicine With Particular Emphasis on Out-of-Hospital Research
Introduction: Standardization of in-hospital treatment (IH) has been suggested as a necessary component of out-of-hospital (OOH) clinical trials. The primary reason advanced is that standardization increases the power of OOH trials to detect a treatment effect. This study examines this assertion, with particular application to OOH clinical trials carried out by the Resuscitation Outcomes Consortium (ROC). We investigate its implications for both validity and reliability.
Methods: We consider examples from OOH cardiac arrest and trauma research for which hospital standardization of care was or could be considered. We scrutinize conceptual as well as statistical aspects of in-hospital standardization. In addition, we simulate data for a pre-hospital trial of hypothermia to illustrate the impact of standardization on validity and reliability of the estimation of a treatment effect.
Results: 1. Inappropriate standardization of IH care can lead to invalid treatment effect estimates. Suppose that OOH hypothermia is only effective if followed by at least 24 hours of IH hypothermia but not by 12 hours. Standardization on 12 hours IH would fail to demonstrate an OOH effect. 2. Under inappropriate standardization, but a realistic scenario, a treatment effect of 5% increase in survival might be estimated as zero even with a sample size of 4,000. 3. We present an example where standardization would have prevented detection of an adverse event. 4. Critical considerations include existence or lack of a valid standard, lack of equipoise, effectiveness versus efficacy, blinding, stratification, and randomization.
Conclusions: Whether standardization of in-hospital care is friend or foe of a particular OOH clinical trial depends largely on the impact of standardization on validity. Sample size calculations are typically of secondary importance. Clinical trials should be concerned first of all with validity, then reliability. In addition, in the absence of high-level evidence, beyond minimal standard care (2005 AHA Resuscitation Guidelines: “there is little evidence to support specific therapies”) it will be challenging to obtain consensus on what constitutes an optimal post-resuscitation treatment plan and to apply it uniformly across the US and Canada.
- © 2010 by American Heart Association, Inc.