doi: 10.1161/CIRCULATIONAHA.107.721357
(Circulation. 2008;118:1675-1684.)
© 2008 American Heart Association, Inc.
Key Issues in Outcomes Research |
Evaluating the Evidence
Is There a Rigid Hierarchy?
From the Denver Veterans Affairs Medical Center (P.M.H.), University of Colorado Denver (P.M.H., P.N.P., F.A.M.), and Denver Health Medical Center (P.N.P., F.A.M.), Denver, Colo.
Correspondence to Frederick A. Masoudi, MD, MSPH, Cardiology, MC 0960, 777 Bannock St, Denver, CO 80204. E-mail fred.masoudi{at}uchsc.edu
Key Words: cardiovascular diseases • epidemiology • research design
 |
Introduction |
|---|
 Top Introduction ConclusionsReferences |
|---|
Health outcomes research applies a wide range of methods to identify optimal approaches to determine optimal approaches to determine the effects of healthcare interventions and policies. Critical evaluation skills are necessary to navigate the evidence and identify studies that should influence clinical decision making and policy. A hierarchical "pyramid of evidence" that emphasizes randomized controlled clinical trials (RCTs) has been promulgated as the approach to judging study design and quality. Whereas this hierarchy may be suitable for the evaluation of efficacy, it may be inappropriate for many health outcomes research questions. This article examines the relevance of this construct to questions beyond those of therapeutic efficacy and discusses the essential role of study designs beyond RCTs. The strengths and weaknesses of study designs commonly encountered in the medical literature are reviewed. The article concludes with 2 case studies that apply the concepts reviewed and illustrate the need to match the study design with the research question. Of note, the concepts of internal validity, external validity, confounding, and bias are central to the discussion of the strengths and weaknesses of study designs in the medical literature. Readers not familiar with these concepts should refer to the Appendix in the online-only Data Supplement for clarification.
The Pyramid of Evidence: A Useful Construct?
Traditionally, the design of a study has been considered a principal barometer of the validity of its findings. In this construct, different study designs are considered in the context of a pyramid of evidence, in which studies most susceptible to threats to internal validity reside at the bottom and those least prone reside at the top (Figure). This hierarchy is widely used in discussions about the quality of medical studies and is integrated into the grading of evidence in practice guidelines.1 However, although the pyramid of evidence is undoubtedly well suited for questions of therapeutic efficacy, it is of limited to no value for many other questions addressed in the medical literature, such as the effectiveness of therapies (ie, how well a treatment works in practice); the association between exposures that cannot be controlled and outcomes; or the safety or harm of a therapy or intervention. For some questions, RCTs are not feasible or are simply impossible. When RCTs are not available to answer a given question in a specific population, which is frequently the case, readers of the medical literature must determine the extent to which the study design implemented was appropriate for the question asked rather than implementing rigid constructs of study quality.
|
Observational effectiveness studies provide insights into how an intervention or treatment works in a population representative of clinical practice. Effectiveness studies assess the extent to which trial findings can be extended to patients seen in clinical practice by evaluating the intervention in populations not included in the efficacy studies. For example, the initial clinical trials of β-blockers for acute myocardial infarction (AMI) were conducted in narrowly defined patient populations with AMI.2,3 However, cohort studies suggest similar benefits of β-blockers in populations not resembling those in the clinical trials, including older patients4 and those with potential relative contraindications.5 Reliance on RCTs alone would limit our understanding of the manner in which evidence from efficacy trials translates into clinical practice. Thus, effectiveness studies are an important complement to RCTs in addressing questions about the benefits of therapy.
RCTs do not provide information on the manner in which results are implemented in clinical care. Specifically, although RCTs can establish what should be done, other study designs are necessary to identify underuse, overuse, and misuse, all of which have the capacity to erode the potential benefits of evidence-based therapy. An example of the power of observational designs is illustrated in a study of the use of antiplatelet and antithrombotic agents for acute coronary syndromes. In an observational analysis of patients with non–ST-segment elevation myocardial infarction (non-STEMI), Alexander and colleagues6 found that excessive medication doses were administered to 33% of patients treated with heparin, 14% of those treated with low–molecular-weight heparin, and 27% of those treated with glycoprotein IIb/IIIa inhibitors. Among patients older than 75 years treated with glycoprotein IIb/IIIa inhibitors, excessive dosing was the rule (64.5% of patients). Perhaps not surprisingly, excessive dosing was associated with significantly higher risks of major bleeding and was estimated to account for 15% of major bleeding episodes. Because dosing protocols are strictly specified in RCTs, this harmful practice could not have been identified without a subsequent observational study focused on practice patterns in "real-world" settings.
For questions evaluating exposures or risk factors that cannot be controlled, nonrandomized designs are again the study design of choice. For example, a recent analysis from the INTERHEART study found an independent graded relationship between waist-to-hip ratio and risk of myocardial infarction even after adjustment for body mass index using a case-control study design.7 In addition, several studies have demonstrated a graded inverse relationship between renal function and patient outcomes after hospitalization for acute coronary syndrome.8 In these scenarios, it is not possible to randomize patients to waist-to-hip ratios or degrees of renal dysfunction.
In general, RCTs are conducted to assess the efficacy of treatment. Although safety end points are invariably included, the sample sizes of RCTs are calculated on the basis of estimates of the impact of therapy on efficacy end points. Because adverse events generally occur less frequently than efficacy end points, clinical trials typically have limited power to detect important safety signals. Furthermore, an intervention may result in an unforeseen event that is not included in the data collection of a RCT and thus is not detected.
Several study designs have been employed to identify important safety concerns. For example, case-control studies were instrumental to understanding the relationship between appetite-suppressant drugs and pulmonary hypertension9 and between aspirin and Reye syndrome in children.10 Meta-analyses of clinical trials, by increasing the numbers of patients exposed to therapy, can also play a role. For example, a recent meta-analysis of 42 clinical trials enrolling 27 847 patients with diabetes mellitus identified an increased risk of myocardial infarction and a trend toward a greater risk of cardiovascular death in those treated with the antihyperglycemic agent rosiglitazone.11 Although a subsequent analysis using alternative meta-analytic approaches and including a larger number of studies did not confirm these results,12 the original study illustrates how meta-analysis can be used to assemble markedly larger sample sizes, thus increasing the possibility of identifying adverse events of therapy.
Although RCTs are a critical component of the evidence base for contemporary medical practice, this design is not feasible or appropriate for all questions. Depending on the question posed, especially in situations in which an RCT is either not feasible or impractical, other study designs may be superior. Therefore, the appropriate use and the strengths and limitations of the wide range of available designs must be understood.
Overview of Study Designs
Research studies can be broadly categorized as experimental or observational.13 In experimental studies, the investigator controls the allocation of participants to particular study groups, which is often but not always performed randomly. In observational studies, the investigator "observes" phenomena or outcomes without specifically assigning a treatment.
Experimental Studies
Randomized Controlled Trials
RCTs are considered the criterion standard for the assessment of whether a treatment or intervention is efficacious—in other words, whether the treatment "works" under ideal conditions (Table). In RCTs, in which subjects are randomly allocated to receive active treatment or a control (placebo, no treatment, or even an active control), each patient has an equal chance of assignment to intervention or control group. Randomization produces comparable study groups in terms of measured and unmeasured variables apart from the intervention itself. Thus, confounding is not a threat to the internal validity of a properly conducted RCT, and any differences in outcome that occur between groups can be attributed to the intervention.
|
Despite this significant strength, RCTs also have important limitations. They are typically expensive, time consuming, and designed to answer a single question or small number of questions about treatment efficacy that are usually narrow in scope (eg, whether aspirin reduces the risk of second events in middle-aged patients after myocardial infarction). Thus, the RCT design may be impractical or even inappropriate to address questions beyond therapeutic efficacy in a well-circumscribed population.
Importantly, RCTs usually employ strict inclusion or exclusion criteria that are intended to maximize the internal validity of the study. However, these criteria intrinsically limit the extent to which the findings of an RCT apply to patients seen in clinical practice (ie, the extent to which the findings are "externally" valid).14,15 In fact, an inverse relationship often exists between internal and external validity in clinical trials.16 The investigator wants to demonstrate that the intervention/treatment works, which frequently requires that the study be performed in a well-defined population that has few competing risks and a low likelihood of dropping out of the study. Additionally, the intensive follow-up necessary for most study protocols often bears little resemblance to follow-up patterns in usual clinical practice. Thus, although rigorous RCTs establish the efficacy of a specific therapy in a limited patient population receiving close follow-up, they often leave remaining questions about the impact of therapy in populations not resembling those enrolled in the trial or of variations of the intervention that were not specifically studied. The need for more practical clinical trials—those that select clinically relevant alternative interventions applied to a diverse patient population in heterogeneous care settings and measuring a broad range of health outcomes—has been recognized.17
The use of RCTs is not confined strictly to drugs or devices but can be useful to assess strategies of care. For example, several trials have compared strategies of care (early invasive versus early conservative) after acute coronary syndromes.18,19 In cases of nontherapeutic strategies (eg, patient education), trials may implement cluster randomization, in which the intervention is randomized according to units such as hospitals, clinics, or physicians rather than on the patient level.20 This approach is often used when researchers are concerned about "contamination" or when it is likely that patients within clusters are all likely to be influenced by the availability of the strategy regardless of whether it is intended to be applied to that patient. However, cluster randomization provides less precise effect estimates than individual-level randomization because of a loss of statistical efficiency with the former approach.20
Noninferiority Trials
Noninferiority trials are increasingly common variants of the RCT in cardiology. The objective of such trials is to demonstrate that a new treatment is no worse than an established treatment by a predefined margin.21 It is used in clinical scenarios when an efficacious therapy exists and a new treatment or intervention has important potential advantages over current standard therapy with respect to tolerability, safety, convenience, or cost. The existence of an established efficacious therapy precludes a placebo control on an ethical basis; thus, the established therapy must be used as a comparator.
The Valsartan in Acute Myocardial Infarction Trial (VALIANT) provides an example of this design.22 VALIANT was conducted to test the primary hypothesis that an angiotensin receptor blocker would result in better survival than treatment with an angiotensin-converting enzyme (ACE) inhibitor in patients with myocardial infarction complicated by heart failure or left ventricular systolic dysfunction. However, it also included an explicit assessment for the noninferiority of the angiotensin receptor blocker compared with ACE inhibitors, which at the time represented the established form of renin-angiotensin blockade in this patient population. After a mean follow-up of slightly >1 year, no difference was found in total mortality between valsartan and captopril treatment (hazard ratio=1.00; 97.5% confidence intervals, 0.90 to 1.11). The upper limit of the confidence interval for this comparison was within the prespecified noninferiority margin of a hazard ratio of 1.13 (P=0.004 for the null hypothesis of inferiority), suggesting a very small likelihood that valsartan was >13% less effective than captopril on a relative basis. Furthermore, valsartan was generally better tolerated than captopril.
In the appraisal of noninferiority trials, one of the most important considerations is the prespecified margin of noninferiority.23,24 Although based on the impact of the established therapy on outcomes and statistical factors, the noninferiority margin is largely arbitrary. Greater margins result in smaller sample size requirements and increase the probability that the new treatment will be deemed noninferior to the established treatment, at the expense of the precision of the findings.
An additional consideration in the assessment of noninferiority studies is the choice of comparators. Some noninferiority trials have been criticized for using weak comparators, placing the new intervention being evaluated at an unfair advantage.25 Ultimately, if it is presumed that the noninferiority margin selected is reasonable and the comparator is fair, the demonstration of noninferiority permits the conclusion that "the new intervention is not worse than the established therapy by more than a pre-specified amount."23
The absence of a placebo arm in noninferiority trials has other important implications. First, an explicit comparison between the new therapy and placebo is not possible and thus must be assumed. Furthermore, the appropriateness of exposing patients to an experimental treatment that under the best circumstances will be found to be "no worse" than the standard of care has been questioned.26
A brief mention of industry sponsorship of trials is warranted. Such sponsorship does not by any means invalidate the findings of a trial. On the contrary, many important clinical trials would not have been possible without manufacturer support. However, even contemporary manufacturer-sponsored cardiovascular trials are more likely to report positive results than those funded by nonprofit organizations.27 Furthermore, the choice of treatment, comparators, and end points in industry-sponsored studies is designated with the primary objective of achieving an indication for the use of the drug or device. The extent to which these important parameters are aligned with those dictated by larger societal interests may vary substantially.
Quasi-Experimental Studies
In some situations, an experimental design is not feasible because of ethical or logistical constraints.28,29 For example, it may be impractical to assess the impact of policy interventions or the publication of new study findings on outcomes because the exposure cannot be controlled. Similarly, a control group may not be available. In such circumstances, nonrandomized quasi-experimental studies are often the most appropriate, if not the only, design possible to assess causality. Among the many quasi-experimental designs, those commonly employed include (1) the pretest/posttest study without controls, (2) the interrupted time series design, and (3) the pretest/posttest study with a contemporaneous control.28
In the pretest/posttest study design, single observations are made before and after the intervention. An example is the study evaluating the Guidelines Applied in Practice quality improvement initiative in Michigan for AMI care.30 In this study, AMI quality-of-care measures (eg, evidence-based medications and counseling) were assessed at baseline among eligible patients and then remeasured after introduction of clinical care tools designed to improve performance on these quality-of-care measures. Between baseline and remeasurement, significant improvements were found in quality of care during hospitalization and at hospital discharge, potentially attributable to the intervention. Because of the absence of an adequate control group, natural temporal trends or concurrent interventions could not be excluded as potential explanations for the findings.
A time series study design uses multiple observations before and after an event or intervention, allowing for an assessment of temporal trends. Stability of the outcome before and a marked change in the outcome after the introduction of the exposure under study provides greater confidence that the observed changes are related to the intervention itself. For example, a study performed in Canada evaluated rates of hospitalization for hyperkalemia before and after publication of the Randomized Aldactone Efficacy Study (RALES), which had established the efficacy of spironolactone therapy for patients with severe symptomatic systolic heart failure.31 Although rates of hospitalization for hyperkalemia were stable at several points in time in the 5 years before publication of RALES, hyperkalemia hospitalizations increased dramatically after publication of RALES and persisted at several points in time thereafter. One possibility to explain this finding, the indiscriminate use of spironolactone in patients at high risk for hyperkalemia, has been confirmed in other studies.32
Similarly, the question of the impact of smoking bans on smoking-related morbidity and mortality is poorly suited to a randomized trial. However, quasi-experimental design has been implemented to assess the effects of these policy interventions on public health. A pretest/posttest design with a contemporaneous control was implemented to assess rates of hospital admissions for AMI before and after a public smoking ban in Helena, Montana, compared with the surrounding area.33 The investigators found that during the 6 months in which the ban was implemented, AMI admissions in Helena fell significantly from an average of 40 to 24. In the surrounding areas, such admissions increased from 12.4 to 18, although this change was not statistically significant. These findings have been subsequently replicated in other communities, lending strength to the hypothesis that smoking bans have a meaningful positive impact on population health.34
As noted in the previous examples, a control group strengthens the conclusions of quasi-experimental studies. A control provides some evidence that changes occurring over time were not the result of natural temporal trends or of unmeasured events that occurred contemporaneously with the exposure under study. However, because such control groups are not selected randomly, associations between intervention and outcomes are at some risk for confounding or bias. Another potential limitation of quasi-experimental study designs is regression to the mean, a common statistical phenomenon in which extreme values on any measure at a point in time will probably be less extreme the next time it is measured.29 This is particularly a problem if the groups selected for study are identified by their initial "outlier" status with respect to the outcome.
Nonexperimental/Observational Study Designs
In nonexperimental studies, the investigator plays no role in the assignment of subjects to their exposure status and observes the population either prospectively or retrospectively. A wide range of observational study designs are implemented in the medical literature, ranging from cohort studies to case series. As with quasi-experimental studies, because the assignment to the exposure in observational studies is not randomly allocated, confounding, especially unmeasured confounding, is a potential threat to internal validity.
Cohort Studies
Cohort studies follow groups of subjects over time and are often used to identify potential risk factors for outcomes or the impact of treatment in real-world populations. In prospective cohort studies, the investigator defines the sample population and measures exposure variables before the occurrence of outcomes. The Framingham Heart Study is a well-known example of the prospective cohort design that was originally initiated to identify patient characteristics that are risk factors for the development of cardiovascular disease.35 The investigators recruited 5209 men and women without cardiovascular disease, performed examinations and interviews to document postulated risk factors, and followed participants over time for the occurrence of cardiovascular disease events.
In retrospective cohort studies, the investigator defines the sample population and collects data about both exposures and outcomes that have occurred in the past. An example is a study evaluating the effectiveness of ACE inhibitors among older patients hospitalized with heart failure and left ventricular systolic dysfunction.36 The baseline characteristics of the patient population and the outcome had already been collected when the analysis was initiated. The investigators found that, after employing statistical techniques to account for measured differences between the treated and untreated patients, the prescription of ACE inhibitors was associated with a significantly lower risk of mortality after 1 year. Although previous RCTs had established the efficacy of ACE inhibitors in selected patients with left ventricular systolic dysfunction, this study provided evidence that the benefits of these drugs may extend to an older population with multiple comorbidities not resembling the subjects of the clinical trials.
In prospective cohort studies, the investigator has control over what exposures and outcomes are measured, whereas this flexibility is not possible in retrospective cohort studies because the measurements have already been performed. For the same reason, however, retrospective cohort studies are usually less expensive to perform, require less time to assemble the cohort, and can realistically study extremely large samples (eg, >17 000 patients were included in the study of ACE inhibitors in older patients with heart failure). A further disadvantage of prospective cohort studies is that patient consent is generally required, which can lead to selection bias or a bias commonly known as the Hawthorne effect. The Hawthorne effect occurs when people who know that they are being observed (such as during a research study) temporarily change their behavior or performance. Neither type of cohort study is ideal when the outcome is rare unless a large sample size is available.
In some cases, a question will be asked within the context of a prospective cohort that the study was not designed a priori to answer. For example, numerous studies have used the Framingham cohort data to address questions that extend beyond the initial intent of the prospective cohort. Such studies share the strengths and limitations of the retrospective cohort design.
Cohort studies have specific strengths and limitations (Table). Both prospective and retrospective designs establish the temporal relationship between exposures and outcomes, thus ensuring that the measurement of the exposure is not biased by the outcome and reducing the likelihood that an association is "effect-cause." In both types of cohort studies, confounding must be considered as a possible threat to validity. When the study of ACE inhibitors cited above is considered, for example, it is possible that unmeasured factors that motivated the decision not to prescribe ACE inhibitors were predictive of mortality ("confounding by indication"). Statistical methods such as multivariable regression are typically employed to "control" for the influence of potentially confounding variables.37 Propensity scores represent the likelihood that the subject would receive the treatment, given the subject’s characteristics. They are used as a means of addressing confounding by balancing measured covariates between treated and untreated groups.38 However, they cannot account for unmeasured confounders.
Case-Control Studies
In case-control studies, the sample is defined by the presence or absence of the outcome of interest, in contrast with cohort studies, in which the sample is defined by the presence or absence of an exposure of interest. After identifying a group of cases and controls, the investigator looks back in time to identify potential exposures of interest, which may be environmental factors, behavior factors, or a treatment/intervention. The odds of a prior exposure are then compared between cases and controls.
A recent study by Go and colleagues39 implemented a case-control design to study the extent to which the use of preventive cardiovascular medications influence the presentation of coronary artery disease. The investigators assembled groups of patients within a large health system whose first clinical presentation was either AMI (cases) or stable angina (controls). The use of statins and β-blockers (the exposures) was assessed in all subjects with the use of retrospective pharmacy data. After controlling for differences in cardiovascular risk factors, recent use of statins (odds ratio=0.45) and β-blockers (odds ratio=0.26) was associated with lower likelihoods of presenting with an AMI. These findings suggest that these medications may protect patients with underlying coronary artery disease from higher-risk clinical presentations.
The selection of an appropriate control group is a particularly important challenge in the conduct of case-control studies. Spurious associations may be found if the control group has an exaggerated exposure status (either too high or low) that does not represent the level of the exposure in the general population without the disease.16 Ideally, the control group is selected from a population similar to cases in all respects except for the presence of the outcome or from a representative population of all persons without the disease.16 In the case of the study of drug therapy on initial coronary presentation, both the case and control subjects were individuals without prior coronary artery disease enrolled in the same health system.
Case-control studies are characterized by specific strengths and limitations (Table). This design is particularly well suited for studying rare events. Furthermore, they are typically inexpensive and can generate results in a timely manner. Case-control studies have several limitations. First, such studies are particularly susceptible to unmeasured confounding. Second, if ascertainment of exposure is based on a subject’s recollection of past events, exposures may be misclassified if recollection is inaccurate (so-called "recall" bias), although the use of objective sources of information to assess exposure can ameliorate this concern. Finally, because the study groups are determined on the basis of the outcome, the prevalence of the outcome cannot be determined in a case-control study. For these reasons, case-control studies are typically considered hypotheses generating, although such studies have occasionally changed practice.10
A variant of the case-control design can be conducted within randomized trials or cohort studies ("nested case-control" studies). An example is a recently published study evaluating the association between valvular regurgitation and past exposure to dopamine agonists in a population-based cohort of patients in the United Kingdom prescribed antiparkinsonian drugs.40 Cases with new valvular regurgitation and controls without valvular regurgitation matched by age, sex, and year of entry into the cohort were identified, and the investigators retrospectively ascertained exposure to antiparkinsonian drugs. The rate of valvular regurgitation was increased with current use of ergot-derived dopamine agonists but not other dopamine agonists.
The results of nested case-control studies are less prone to bias than case-control studies because the data are typically collected prospectively, retaining some of the advantages of cohort studies. In addition, the control group is obtained from the same cohort, minimizing the potential for selection bias.16
Cross-Sectional Studies
In cross-sectional studies, both exposures and outcomes are measured at a single point in time, and the prevalence of the outcome is compared among those with and without exposure. For example, Hanna et al41 assessed the association between exposures of hyperlipidemia and lipid-lowering drug therapy and the outcome of atrial fibrillation among patients with left ventricular systolic dysfunction from a multicenter registry. They found that the prevalence of atrial fibrillation was much lower among patients with hyperlipidemia taking statin medications compared with patients with hyperlipidemia but not taking statins or patients without hyperlipidemia. The investigators concluded that lipid-lowering drugs were associated with lower prevalence of atrial fibrillation after accounting for multiple other factors.
Advantages of cross-sectional studies are that multiple exposures and outcomes can be measured simultaneously, disease prevalence can be estimated, and studies can be performed often with relatively few resources in a short period of time. Conversely, cross-sectional studies have important limitations in addition to the potential threats to internal validity posed by confounding or bias. Because exposure and outcome are measured simultaneously, a temporal sequence between exposure and outcome cannot be established. In the aforementioned study, for example, it is possible that because patients with atrial fibrillation were more likely to see a physician, they were more likely to have received treatment for hyperlipidemia. This is not a particular issue if the exposure is a fixed characteristic (eg, patient sex). Because of their limitations, cross-sectional studies typically form the foundation for more definitive studies.
Case Series
A case series is a descriptive study of a relatively small number of people with a disease or condition, all of whom experienced the same exposure (ie, no control group without the condition or not receiving the treatment of interest is available). An example is a study evaluating the benefit of percutaneous alcohol septal ablation for reducing left ventricular outflow tract gradients in patients with hypertrophic obstructive cardiomyopathy.42 The authors found that this procedure significantly reduced outflow tract gradients and exercise capacity in a series of 50 patients undergoing the procedure.
Case series generally provide weak evidence of causality because they are particularly prone to bias and confounding. Furthermore, the absence of a control group is a critical limitation. However, case series are useful in several situations, including the discovery of new diseases or rare manifestations of disease and the detection of unexpected benefits or risks of a treatment.43 The findings from a case series are used to generate hypotheses for testing with the use of more rigorous study designs that include a comparison group; however, in occasional circumstances in which the relationships found are extremely compelling, or when identifying a treatment that appears to provide benefits for rapidly fatal conditions, case series may have an important impact on medical practice.
Other Study Designs
Meta-Analyses
Meta-analyses are used to aggregate data from multiple studies, most often when the results of individual studies have been contradictory or inconclusive or when an outcome of interest was too rare in individual studies to generate firm conclusions.44–46 Combining findings from multiple studies may provide enough end points or outcomes to answer the question of interest. Although meta-analyses may include observational studies, they are frequently used to summarize data from experimental studies. Systematic reviews, like meta-analyses, use standardized methods to search for and appraise the available studies to reduce bias. However, systematic reviews do not employ quantitative methods to summarize the results and thus may not provide the same strength of evidence as meta-analyses.47
Meta-analyses can be employed to assess questions of efficacy, safety, or both. A recent meta-analysis evaluated the efficacy and potential harm of antifibrinolytic agents among patients undergoing cardiac surgery.48 Despite the number of trials on antifibrinolytic agents, each of the individual studies had small sample sizes, and questions remained about their efficacy as well as the potential for harm. By combining 138 studies, the authors had enough end points and found that these agents reduced blood loss as well as transfusions without an increase in adverse outcomes.
Several issues related to the selection of the studies included are relevant to the evaluation of meta-analyses. The first pertains to the methodological quality of the studies included. Because the quality of the output can be no better than the inputs (ie, "garbage in" produces "garbage out"), poor-quality primary studies fundamentally undermine a meta-analysis. Separately from study quality, heterogeneity across individual studies or the extent to which results differ among the component studies should also be considered. Some authors argue that meta-analyses pooling significantly heterogeneous studies have limited validity.47 Another issue is publication bias, which is a form of selection bias whereby results that are more interesting, significant, of higher methodological quality, or from prominent groups are more likely to be included. Typically, because positive studies tend to be reported more frequently than negative studies, meta-analyses confined to published results may result in biases in favor of treatment. Publication bias is known to have an important impact on the results of meta-analyses.49
Cost-Effectiveness Studies
Although identifying the incremental efficacy or effectiveness of a treatment or strategy is useful, it does not provide a perspective on cost. The interest in understanding the incremental costs of achieving particular benefits in formal cost-effectiveness studies has grown proportionate to the dramatic growth in healthcare expenditures. Cost-effectiveness studies, which apply decision-analytic methods to compare the costs per unit of effectiveness of different treatment options, provide perspectives on the comparative value of these options.46
Cost-effectiveness is particularly relevant to expensive interventions such as implantable cardioverter-defibrillator therapy. In an analysis using data from 6 clinical trials, investigators estimated the costs associated with primary prevention implantable cardioverter-defibrillator therapy.50 Using base-case assumptions, the cost-effectiveness of implantable cardioverter-defibrillator therapy compared with control ranged from $34 000 to $70 200 per quality-adjusted life year gained. In sensitivity analyses, in which the parameters of the base-case scenario were altered over a range of assumptions, the cost-effectiveness ratio remained below $100 000 per quality-adjusted life year presuming that implantable cardioverter-defibrillator therapy resulted in lower mortality for at least 7 years.
Because cost-effectiveness analyses are by their nature hypothetical, concerns have been raised about the arbitrariness of the underlying assumptions. Readers of such studies should consider a number of attributes, including the perspective used, the comparators employed, the time horizon of the analysis, the sources of costs, the use of discounting, and the use of sensitivity analyses to assess the robustness of the findings across a wide range of reasonable assumptions.46 Expert recommendations on guidelines for conducting cost-effectiveness analysis,51 the calculation of the cost-effectiveness ratio,52 and appropriate assessment of quality-adjusted life years53 are available to facilitate the assessment of cost-effectiveness studies.
Qualitative Studies
Although frequently overlooked in discussions of the taxonomy of study designs, qualitative studies generate narrative stories, explanations, typologies of phenomena, and conceptual frameworks.54 They are often used in areas in which data or knowledge is inadequate or in which conventional theories may not be applicable. Qualitative studies can generate theories and identify relevant variables to be studied subsequently in quantitative studies, or they can be used in a complementary fashion to yield findings that are broader in scope and richer in meaning.
In a novel application of qualitative methods, Bradley and colleagues55 conducted in-depth interviews in 8 hospitals that varied widely in terms of changes in the use of β-blockers for their patients hospitalized with AMI in the 3 years preceding the study. Hospitals with greater improvements in β-blocker use uniquely demonstrated several characteristics, including shared goals for improvement, substantial administrative support, strong physician advocates for β-blocker use, and the implementation of credible data feedback. Although this study could not conclusively prove that these face-valid characteristics were the cause of improvements in β-blocker use, it identified potential elements of successful programs to improve AMI care.
Case Studies
The following 2 case studies are intended to illustrate the strengths and weaknesses of the traditional hierarchy of evidence and of the various study designs. The first, which involves hormone replacement therapy (HRT), illustrates the relevance of the hierarchy of evidence for a question focused on treatment efficacy. The second, which involves improving door-to-balloon times for STEMI, demonstrates the failings of this traditional hierarchy of evidence and highlights the role of other study methods when the question of interest does not focus on treatment efficacy.
Example 1: HRT
In the 1990s, use of HRT was widely believed to confer protection from cardiovascular events on the basis of consistent findings of benefits in prospective cohort studies.56–59 For example, in the Nurses’ Health Study, a prospective cohort study of 59 337 women with 16 years of follow-up, HRT was associated with a significantly lower risk of major coronary heart disease events.57 As a result of this body of evidence, HRT was widely prescribed for this indication.
However, long-awaited RCTs completely changed the understanding of the relationship between HRT and cardiovascular outcomes. The Women’s Health Initiative, a RCT, found an excess coronary heart disease risk with estrogen plus progestin in healthy postmenopausal women (hazard ratio, 1.29; 95% confidence interval, 1.03 to 1.69) compared with placebo.60 In fact, excess risks were found per 10 000 person-years that were attributable to estrogen plus progestin, including more coronary heart disease events, strokes, pulmonary emboli, and invasive breast cancers. Subsequent RCTs corroborated the finding of no cardiovascular benefit with HRT.61,62 These RCTs have widely reversed the clinical practice of prescribing HRT for cardiovascular protection.63
It is likely that, at least in part, confounding and bias in the observational studies explain the marked differences in conclusions between the early studies that suggested a benefit of HRT and subsequent RCTs. In the observational studies, patients were not randomly allocated to treatment. In general, women who were willing to take HRT likely had healthier lifestyles, and women with healthier lifestyles have reduced risk of cardiovascular disease. Therefore, a healthier lifestyle may have confounded the association between current HRT use and lower cardiovascular events seen in the observational studies.64 Furthermore, women taking HRT in the observational studies may have been more likely to take medications in general and comply with other healthcare recommendations; in addition, this healthy adherer effect has been associated with improved outcomes even among placebo-treated patients.65 Therefore, adherent patients may have been more likely to enroll in the observational studies of HRT, and this selection bias could have accounted for the association between current HRT use and cardiovascular outcomes.66 These 2 threats to internal validity, confounding and bias, highlight the fact that for questions on efficacy of therapy, randomized studies remain the ideal study design. The case of HRT demonstrates that adhering to the traditional pyramid of evidence is very important to overcome the limitations inherent in observational studies when efficacy is assessed.
The ongoing debate notwithstanding, the HRT story may be perceived as signaling the demise of observational studies.67 As a generalization, this conclusion is unfortunate; with respect to questions of therapeutic efficacy, RCTs are understandably considered superior study designs. However, as discussed previously, nonexperimental study designs are indispensible in answering many questions beyond efficacy.
Example 2: Door-to-Balloon Times for STEMI
The evolution of the understanding of how to improve the delivery of primary percutaneous coronary intervention to patients with STEMI illustrates the importance of study designs other than the RCT. Despite data that faster door-to-balloon times improve outcomes for patients presenting with STEMI,68 substantial variation exists in door-to-balloon times among hospitals, and many do not reach performance targets recommended in guidelines.69 Although it is clear that improvement is needed, little has been known about how to achieve improvement.
To address this gap in knowledge, Bradley and colleagues70 conducted interviews among top-performing hospitals in an attempt to understand what high performers were doing to achieve their results. Subsequently, these investigators assessed the relationship between these strategies and door-to-balloon times in a survey of 365 hospitals nationwide.71 They identified several strategies associated with significantly lower door-to-balloon times, including (1) having emergency medicine physicians activate the catheterization laboratory, (2) having a single call to a central page operator to activate the laboratory, (3) activating the laboratory before hospital arrival, (4) expecting staff to arrive in the catheterization laboratory within 20 minutes after being paged, (5) having an attending cardiologist always on site, and (6) using real-time data feedback. None of these strategies was employed by a majority of hospitals, suggesting that instituting these approaches might result in important improvements in care.
Other studies using various study designs have provided complementary data on improving reperfusion therapy for STEMI. In a before/after quasi-experimental design, Khot and colleagues72 found that door-to-balloon times dropped by 38 minutes after the institution of emergency department activation of the catheterization laboratory. In another quasi-experimental study, Carstensen and colleagues73 found that field triage and emergency department bypass were associated with shorter symptom onset to balloon times compared with a concurrent control group. A recent large-scale before-and-after quasi-experimental study suggested the effectiveness of a statewide system for providing reperfusion therapy in North Carolina.74 Finally, a cross-sectional study of patients presenting with STEMI suggested that suboptimal ECG interpretation and suboptimal understanding of evidence-based indications and contraindications to reperfusion therapy were associated with failure to provide reperfusion therapy, thus identifying possible targets for improvement initiatives.75
Although the understanding of how to improve reperfusion therapy for patients with STEMI is not by any means complete, these examples demonstrate the power of many different study designs to address critical questions. Although RCTs were instrumental in establishing the knowledge that reperfusion therapy should be provided to patients with STEMI, quasi-experimental, cross-sectional, and qualitative studies have generated important insights into how this therapy can be delivered in a timely manner to as many eligible patients as possible.
|
Conclusions |
|---|
|
TopIntroductionConclusionsReferences |
|---|
A hierarchy of study designs has been proposed on the basis of strength of evidence that these studies provide for causality, and experimental studies, especially RCTs, reside at the pinnacle. This hierarchy is most appropriate in evaluating the efficacy of a treatment or intervention. However, experimental studies are not always feasible or appropriate and are often not well suited to answer important questions, such as those on the safety and effectiveness of therapies in real-world populations, the impact of risk factors on outcomes, or the effects of policy interventions. Ultimately, the interpretation of the medical literature requires not only the understanding of the strengths and limitations of different study designs but also an appreciation for the circumstances in which the traditional hierarchy does not apply and integration of complementary information derived from various study designs is needed.
|
Acknowledgments |
|---|
Disclosures
Dr Masoudi has served on advisory boards for Takeda, NA; Amgen; and United Healthcare. Dr Masoudi has also received research support from Amgen, has contracts with the Oklahoma Foundation for Medical Quality and the Colorado Foundation for Medical Care, and is an Associate Editor for Journal Watch Cardiology of the Massachusetts Medical Society. The other authors report no conflicts.
|
Footnotes |
|---|
The online-only Data Supplement is available with this article at http://circ.ahajournals.org/cgi/content/full/118/16/1675/DC1.
|
References |
|---|
|
TopIntroductionConclusionsReferences |
|---|
1. Antman EM, Anbe DT, Armstrong PW, Bates ER, Green LA, Hand M, Hochman JS, Krumholz HM, Kushner FG, Lamas GA, Mullany CJ, Ornato JP, Pearle DL, Sloan MA, Smith SC Jr, Alpert JS, Anderson JL, Faxon DP, Fuster V, Gibbons RJ, Gregoratos G, Halperin JL, Hiratzka LF, Hunt SA, Jacobs AK. ACC/AHA guidelines for the management of patients with ST-elevation myocardial infarction: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 1999 Guidelines for the Management of Patients With Acute Myocardial Infarction). Circulation. 2004; 110: 588–636.
2. Randomised trial of intravenous atenolol among 16,027 cases of suspected acute myocardial infarction: ISIS-1. Lancet. 1986; 328: 57–66.[CrossRef]
3. Roberts R, Rogers WJ, Mueller HS, Lambrew CT, Diver DJ, Smith HC, Willerson JT, Knatterud GL, Forman S, Passamani E. Immediate versus deferred beta-blockade following thrombolytic therapy in patients with acute myocardial infarction: results of the Thrombolysis in Myocardial Infarction (TIMI) II-B Study. Circulation. 1991; 83: 422–437.
4. Krumholz HM, Radford MJ, Wang Y, Chen J, Heiat A, Marciniak TA. National use and effectiveness of beta-blockers for the treatment of elderly patients after acute myocardial infarction: National Cooperative Cardiovascular Project. JAMA. 1998; 280: 623–629.
5. Chen J, Radford MJ, Wang Y, Marciniak TA, Krumholz HM. Effectiveness of beta-blocker therapy after acute myocardial infarction in elderly patients with chronic obstructive pulmonary disease or asthma. J Am Coll Cardiol. 2001; 37: 1950–1956.
6. Alexander KP, Chen AY, Roe MT, Newby LK, Gibson CM, len-LaPointe NM, Pollack C, Gibler WB, Ohman EM, Peterson ED. Excess dosing of antiplatelet and antithrombin agents in the treatment of non-ST-segment elevation acute coronary syndromes. JAMA. 2005; 294: 3108–3116.
7. Yusuf S, Hawken S, Ounpuu S, Dans T, Avezum A, Lanas F, McQueen M, Budaj A, Pais P, Varigos J, Lisheng L. Effect of potentially modifiable risk factors associated with myocardial infarction in 52 countries (the INTERHEART study): case-control study. Lancet. 2004; 364: 937–952.[CrossRef][Medline] [Order article via Infotrieve]
8. Masoudi FA, Plomondon ME, Magid DJ, Sales A, Rumsfeld JS. Renal insufficiency and mortality from acute coronary syndromes. Am Heart J. 2004; 147: 623–629.[CrossRef][Medline] [Order article via Infotrieve]
9. Abenhaim L, Moride Y, Brenot F, Rich S, Benichou J, Kurz X, Higenbottam T, Oakley C, Wouters E, Aubier M, Simonneau G, Begaud B; International Primary Pulmonary Hypertension Study Group. Appetite-suppressant drugs and the risk of primary pulmonary hypertension. N Engl J Med. 1996; 335: 609–616.
10. Hurwitz ES, Barrett MJ, Bregman D, Gunn WJ, Pinsky P, Schonberger LB, Drage JS, Kaslow RA, Burlington DB, Quinnan GV. Public Health Service study of Reye’s syndrome and medications: report of the main study. JAMA. 1987; 257: 1905–1911.
11. Nissen SE, Wolski K. Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med. 2007; 356: 2457–2471.
12. Diamond GA, Bax L, Kaul S. Uncertain effects of rosiglitazone on the risk for myocardial infarction and cardiovascular death. Ann Intern Med. 2007; 147: 578–581.
13. Grimes DA, Schulz KF. An overview of clinical research: the lay of the land. Lancet. 2002; 359: 57–61.[CrossRef][Medline] [Order article via Infotrieve]
14. Masoudi FA, Havranek EP, Wolfe P, Gross CP, Rathore SS, Steiner JF, Ordin DL, Krumholz HM. Most hospitalized older persons do not meet the enrollment criteria for clinical trials in heart failure. Am Heart J. 2003; 146: 250–257.[CrossRef][Medline] [Order article via Infotrieve]
15. LaPointe NM, Chen AY, Alexander KP, Roe MT, Pollack CV Jr, Lytle BL, Ohman ME, Gibler BW, Peterson ED. Enoxaparin dosing and associated risk of in-hospital bleeding and death in patients with non ST-segment elevation acute coronary syndromes. Arch Intern Med. 2007; 167: 1539–1544.
16. Hulley SB, Cummings SR, Browner WS, Grady D, Hearst N, Newman TB. Designing Clinical Research. 2nd ed. Philadelphia, Pa: Lippincott Williams & Wilkins; 2001.
17. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003; 290: 1624–1632.
18. Cannon CP, Weintraub WS, Demopoulos LA, Vicari R, Frey MJ, Lakkis N, Neumann FJ, Robertson DH, DeLucca PT, DiBattiste PM, Gibson CM, Braunwald E. Comparison of early invasive and conservative strategies in patients with unstable coronary syndromes treated with the glycoprotein IIb/IIIa inhibitor tirofiban. N Engl J Med. 2001; 344: 1879–1887.
19. de Winter RJ, Windhausen F, Cornel JH, Dunselman PH, Janus CL, Bendermacher PE, Michels HR, Sanders GT, Tijssen JG, Verheugt FW. Early invasive versus selectively invasive management for acute coronary syndromes. N Engl J Med. 2005; 353: 1095–1104.
20. Donner A, Klar J. Design and Analysis of Cluster Randomized Trials in Health Research. London, UK: Arnold; 2000.
21. Kaul S, Diamond GA. Good enough: a primer on the analysis and interpretation of noninferiority trials. Ann Intern Med. 2006; 145: 62–69.
22. Pfeffer MA, McMurray JJ, Velazquez EJ, Rouleau JL, Kober L, Maggioni AP, Solomon SD, Swedberg K, Van de WF, White H, Leimberger JD, Henis M, Edwards S, Zelenkofske S, Sellers MA, Califf RM. Valsartan, captopril, or both in myocardial infarction complicated by heart failure, left ventricular dysfunction, or both. N Engl J Med. 2003; 349: 1893–1906.
23. Gotzsche PC. Lessons from and cautions about noninferiority and equivalence randomized trials. JAMA. 2006; 295: 1172–1174.
24. Le HA, Giraudeau B, Baron G, Ravaud P. Quality of reporting of noninferiority and equivalence randomized trials. JAMA. 2006; 295: 1147–1151.
25. Friedman LM, Furberg CD, DeMets DL. Fundamentals of Clinical Trials. 3rd ed. New York, NY: Springer; 1998.
26. Garattini S, Bertele' V. Non-inferiority trials are unethical because they disregard patients’ interests. Lancet. 2007; 370: 1875–1877.[CrossRef][Medline] [Order article via Infotrieve]
27. Ridker PM, Torres J. Reported outcomes in major cardiovascular clinical trials funded by for-profit and not-for-profit organizations: 2000–2005. JAMA. 2006; 295: 2270–2274.
28. Campbell DT, Stanley JC. Experimental and Quasi-experimental Designs for Research. Dallas, Tex: Houghton Mifflin Company; 1963.
29. Harris AD, Bradham DD, Baumgarten M, Zuckerman IH, Fink JC, Perencevich EN. The use and interpretation of quasi-experimental studies in infectious diseases. Clin Infect Dis. 2004; 38: 1586–1591.[CrossRef][Medline] [Order article via Infotrieve]
30. Eagle KA, Mehta RH, Riba AL, DeFranco AC, Montoye CK. Taking the ACC/AHA guidelines for care of acute myocardial infarction to the bedside: the GAP projects in southeastern Michigan. Am Heart J. 2004; 148: S49–S51.[CrossRef][Medline] [Order article via Infotrieve]
31. Juurlink DN, Mamdani MM, Lee DS, Kopp A, Austin PC, Laupacis A, Redelmeier DA. Rates of hyperkalemia after publication of the Randomized Aldactone Evaluation Study. N Engl J Med. 2004; 351: 543–551.
32. Masoudi FA, Gross CP, Wang Y, Havranek EP, Rathore SS, Foody JM, Krumholz HM. Adoption of spironolactone therapy for older patients with heart failure and left ventricular systolic dysfunction in the US: 1998–2001. Circulation. 2005; 112: 39–47.
33. Sargent RP, Shepard RM, Glantz SA. Reduced incidence of admissions for myocardial infarction associated with public smoking ban: before and after study. BMJ. 2004; 328: 977–980.
34. Bartecchi C, Alsever RN, Nevin-Woods C, Thomas WM, Estacio RO, Bartelson BB, Krantz MJ. Reduction in the incidence of acute myocardial infarction associated with a citywide smoking ordinance. Circulation. 2006; 114: 1490–1496.
35. The Framingham Study. Available at: http://www.nhlbi.nih.gov/about/framingham/index.html. Accessed December 13, 2007.
36. Masoudi FA, Rathore SS, Wang Y, Havranek EP, Curtis JP, Foody JM, Krumholz HM. National patterns of use and effectiveness of angiotensin-converting enzyme inhibitors in older patients with heart failure and left ventricular systolic dysfunction. Circulation. 2004; 110: 724–731.
37. Harrell FE Jr. Regression Modeling Strategies with Applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY: Springer-Verlag New York Inc; 2001.
38. Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann Intern Med. 1997; 127: 757–763.
39. Go AS, Iribarren C, Chandra M, Lathon PV, Fortmann SP, Quertermous T, Hlatky MA. Statin and beta-blocker therapy and the initial presentation of coronary heart disease. Ann Intern Med. 2006; 144: 229–238.
40. Schade R, Andersohn F, Suissa S, Haverkamp W, Garbe E. Dopamine agonists and the risk of cardiac-valve regurgitation. N Engl J Med. 2007; 356: 29–38.
41. Hanna IR, Heeke B, Bush H, Brosius L, King-Hageman D, Dudley SC Jr, Beshai JF, Langberg JJ. Lipid-lowering drug use is associated with reduced prevalence of atrial fibrillation in patients with left ventricular systolic dysfunction. Heart Rhythm. 2006; 3: 881–886.[CrossRef][Medline] [Order article via Infotrieve]
42. Boekstegers P, Steinbigler P, Molnar A, Schwaiblmair M, Becker A, Knez A, Haberl R, Steinbeck G. Pressure-guided nonsurgical myocardial reduction induced by small septal infarctions in hypertrophic obstructive cardiomyopathy. J Am Coll Cardiol. 2001; 38: 846–853.
43. Vandenbroucke JP. In defense of case reports and case series. Ann Intern Med. 2001; 134: 330–334.
44. Egger M, Smith GD. Meta-analysis: potentials and promise. BMJ. 1997; 315: 1371–1374.
45. Egger M, Smith GD, Phillips AN. Meta-analysis: principles and procedures. BMJ. 1997; 315: 1533–1537.
46. Petitti DB. Meta-analysis, Decision Analysis, and Cost-effectiveness Analysis: Methods for Quantitative Synthesis in Medicine. 2nd ed. New York, NY: Oxford University Press; 2000.
47. The Evidence-Based Medicine Working Group. User’s Guide to the Medical Literature: A Manual for Evidence-Based Practice. Chicago, Ill: American Medical Association Press; 2002.
48. Brown JR, Birkmeyer NJ, O'Connor GT. Meta-analysis comparing the effectiveness and adverse outcomes of antifibrinolytic agents in cardiac surgery. Circulation. 2007; 115: 2801–2813.
49. Sutton AJ, Duval SJ, Tweedie RL, Abrams KR, Jones DR. Empirical assessment of effect of publication bias on meta-analyses. BMJ. 2000; 320: 1574–1577.
50. Sanders GD, Hlatky MA, Owens DK. Cost-effectiveness of implantable cardioverter-defibrillators. N Engl J Med. 2005; 353: 1471–1480.
51. Genduso LA, Kotsanos JG. Review of health economic guidelines in the form of regulations, principles, policies, and positions. Drug Inform J. 1996; 30: 1016.
52. Weinstein MC, Siegel JE, Gold MR, Kamlet MS, Russell LB. Recommendations of the Panel on Cost-effectiveness in Health and Medicine. JAMA. 1996; 276: 1253–1258.
53. Nord E. Toward quality assurance in QALY calculations. Int J Technol Assess Health Care. 1993; 9: 37–45.[Medline] [Order article via Infotrieve]
54. Giacomini MK, Cook DJ; Evidence-Based Medicine Working Group. Users’ guides to the medical literature, XXIII: qualitative research in health care A: are the results of the study valid? JAMA. 2000; 284: 357–362.
55. Bradley EH, Holmboe ES, Mattera JA, Roumanis SA, Radford MJ, Krumholz HM. A qualitative study of increasing beta-blocker use after myocardial infarction: why do some hospitals succeed? JAMA. 2001; 285: 2604–2611.
56. Stampfer MJ, Colditz GA, Willett WC, Manson JE, Rosner B, Speizer FE, Hennekens CH. Postmenopausal estrogen therapy and cardiovascular disease: ten-year follow-up from the Nurses’ Health Study. N Engl J Med. 1991; 325: 756–762.[Abstract]
57. Grodstein F, Stampfer MJ, Colditz GA, Willett WC, Manson JE, Joffe M, Rosner B, Fuchs C, Hankinson SE, Hunter DJ, Hennekens CH, Speizer FE. Postmenopausal hormone therapy and mortality. N Engl J Med. 1997; 336: 1769–1775.
58. Grodstein F, Stampfer MJ, Manson JE, Colditz GA, Willett WC, Rosner B, Speizer FE, Hennekens CH. Postmenopausal estrogen and progestin use and the risk of cardiovascular disease. N Engl J Med. 1996; 335: 453–461.
59. Stampfer MJ, Colditz GA. Estrogen replacement therapy and coronary heart disease: a quantitative assessment of the epidemiologic evidence. Prev Med. 1991; 20: 47–63.[CrossRef][Medline] [Order article via Infotrieve]
60. Rossouw JE, Anderson GL, Prentice RL, LaCroix AZ, Kooperberg C, Stefanick ML, Jackson RD, Beresford SA, Howard BV, Johnson KC, Kotchen JM, Ockene J. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women’s Health Initiative randomized controlled trial. JAMA. 2002; 288: 321–333.
61. Hulley S, Grady D, Bush T, Furberg C, Herrington D, Riggs B, Vittinghoff E; Heart and Estrogen/Progestin Replacement Study (HERS) Research Group. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. JAMA. 1998; 280: 605–613.
62. Gabriel SR, Carmona L, Roque M, Sánchez GL, Bonfill X. Hormone replacement therapy for preventing cardiovascular disease in post-menopausal women. Cochrane Database Syst Rev. 2005; (2): CD002229.
63. US Preventive Services Task Force. Hormone replacement therapy for the prevention of chronic conditions in postmenopausal women. Available at: http://www.ahrq.gov/clinic/uspstf/uspspmho.htm. Accessed December 14, 2007.
64. Lawlor DA, Smith GD, Ebrahim S. Socioeconomic position and hormone replacement therapy use: explaining the discrepancy in evidence from observational and randomized controlled trials. Am J Public Health. 2004; 94: 2149–2154.
65. Simpson SH, Eurich DT, Majumdar SR, Padwal RS, Tsuyuki RT, Varney J, Johnson JA. A meta-analysis of the association between adherence to drug therapy and mortality. BMJ. 2006; 333: 15.
66. Petitti DB. Coronary heart disease and estrogen replacement therapy: can compliance bias explain the results of observational studies? Ann Epidemiol. 1994; 4: 115–118.[Medline] [Order article via Infotrieve]
67. Taubes G. Do we really know what makes us healthy? New York Times. September 16, 2007.
68. Cannon CP, Gibson CM, Lambrew CT, Shoultz DA, Levy D, French WJ, Gore JM, Weaver WD, Rogers WJ, Tiefenbrunn AJ. Relationship of symptom-onset-to-balloon time and door-to-balloon time with mortality in patients undergoing angioplasty for acute myocardial infarction. JAMA. 2000; 283: 2941–2947.
69. Williams SC, Schmaltz SP, Morton DJ, Koss RG, Loeb JM. Quality of care in U.S. hospitals as reflected by standardized measures, 2002–2004. N Engl J Med. 2005; 353: 255–264.
70. Bradley EH, Roumanis SA, Radford MJ, Webster TR, McNamara RL, Mattera JA, Barton BA, Berg DN, Portnay EL, Moscovitz H, Parkosewich J, Holmboe ES, Blaney M, Krumholz HM. Achieving door-to-balloon times that meet quality guidelines: how do successful hospitals do it? J Am Coll Cardiol. 2005; 46: 1236–1241.
71. Bradley EH, Herrin J, Wang Y, Barton BA, Webster TR, Mattera JA, Roumanis SA, Curtis JP, Nallamothu BK, Magid DJ, McNamara RL, Parkosewich J, Loeb JM, Krumholz HM. Strategies for reducing the door-to-balloon time in acute myocardial infarction. N Engl J Med. 2006; 355: 2308–2320.
72. Khot UN, Johnson ML, Ramsey C, Khot MB, Todd R, Shaikh SR, Berg WJ. Emergency department physician activation of the catheterization laboratory and immediate transfer to an immediately available catheterization laboratory reduce door-to-balloon time in ST-elevation myocardial infarction. Circulation. 2007; 116: 67–76.
73. Carstensen S, Nelson GC, Hansen PS, Macken L, Irons S, Flynn M, Kovoor P, Soo Hoo SY, Ward MR, Rasmussen HH. Field triage to primary angioplasty combined with emergency department bypass reduces treatment delays and is associated with improved outcome. Eur Heart J. 2007; 28: 2313–2319.
74. Jollis JG, Roettig ML, Aluko AO, Anstrom KJ, Applegate RJ, Babb JD, Berger PB, Bohle DJ, Fletcher SM, Garvey JL, Hathaway WR, Hoekstra JW, Kelly RV, Maddox WT Jr, Shiber JR, Valeri FS, Watling BA, Wilson BH, Granger CB; Reperfusion of Acute Myocardial Infarction in North Carolina Emergency Departments (RACE) Investigators. Implementation of a statewide system for coronary reperfusion for ST-segment elevation myocardial infarction. JAMA. 2007; 298: 2371–2380.
75. Tricomi AJ, Magid DJ, Rumsfeld JS, Vinson DR, Lyons EE, Crounse L, Ho PM, Peterson PN, Masoudi FA. Missed opportunities for reperfusion therapy for ST-segment elevation myocardial infarction: results of the Emergency Department Quality in Myocardial Infarction (EDQMI) study. Am Heart J. 2008; 155: 471–477.[CrossRef][Medline] [Order article via Infotrieve]
This article has been cited by other articles:
 |
|
 |
|
  H. M. Krumholz Outcomes Research: Myths and Realities Circ Cardiovasc Qual Outcomes, January 1, 2009; 2(1): 1 - 3. [Full Text] [PDF]
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||