This Article Free upon publication Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Zaritsky, A. Articles by Morley, P. T. Search for Related Content PubMed Articles by Zaritsky, A. Articles by Morley, P. T.

(Circulation. 2005;112:III-128 – III-130.)
© 2005 American Heart Association, Inc.

Editorial

The Evidence Evaluation Process for the 2005 International Consensus Conference on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations

Arno Zaritsky, MD; Peter T. Morley, MD

From the 2005 International Consensus Conference on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations, hosted by the American Heart Association in Dallas, Texas, January 23–30, 2005.

	Introduction

Top Introduction Steps for Evidence Evaluation Controversies Encountered Summary References

 

"In God we trust. All others must bring data."

— —Robert Hayden, Plymouth State College

Evidence-based medicine is described as "the conscientious, explicit and judicious use of current best evidence in making decisions about individual patients."¹ The evidence evaluation process summarized in this supplement was designed to ensure the review of all available evidence pertaining to resuscitation. Many aspects of the resuscitation process create unique challenges for the design of experimental protocols and data analysis and have not been evaluated by randomized controlled human studies. Exclusion of studies other than controlled human studies would eliminate a wealth of information that could help guide resuscitation management; for this reason, lower levels of evidence, including nonhuman studies, were included in the review.

To begin the review process, international experts (worksheet reviewers) were assigned questions to evaluate. The questions were selected from a survey of each of the International Liaison Committee on Resuscitation (ILCOR) specialty task forces (eg, basic life support, advanced life support, pediatrics) and from the ILCOR member resuscitation councils and their training networks. The evaluation of each question was completed on a structured evidence evaluation worksheet developed for the 2005 Consensus Conference. Because many of the worksheet reviewers had never conducted a structured evidence-based review, instructional sessions were held at the twice-yearly ILCOR meetings and an instructional CD-ROM was created, demonstrating how to conduct an efficient search for evidence, complete the worksheet, and use citation management software. Two worksheet experts (Arno Zaritsky and Peter Morley) were appointed to provide further quality assurance; they reviewed all submitted worksheets. Comments, emendations, and queries were provided to the worksheet reviewers in an iterative process until the worksheets were deemed complete by the worksheet experts.

The worksheets completed for the 2005 Consensus Conference are linked from the electronic version of this document as online data supplements. Most superscript worksheet numbers are located adjacent to headings and begin with the letter W to distinguish them from other reference citations. Readers of the electronic version of this supplement can access a cited worksheet by clicking on the linked worksheet callout. This will link the reader to the Worksheets Cited list at the end of each section. This list contains active links to the individual worksheets cited. Readers of the printed publication can identify the complete title and author of a cited worksheet by referring to the numbered worksheet list at the end of this supplement (See Appendix 1) and then accessing that worksheet on the conference website at www.C2005.org. In the discussion below, a blank worksheet is cited and can be accessed for reference.

	Steps for Evidence Evaluation

Top Introduction Steps for Evidence Evaluation Controversies Encountered Summary References

 
The following steps correspond with the major steps listed in the evidence evaluation worksheets.

Step 1. State the Proposal (1A) and Gather and Select the Evidence (1B)^W277
All reviewers were instructed to search their allocated questions broadly. Reviewers documented their search strategies to ensure reproducibility of the search. The minimum electronic databases to be searched included the Cochrane database for systematic reviews and the Central Register of Controlled Trials [http://www.cochrane.org/], MEDLINE [http://www.ncbi.nlm.nih.gov/PubMed/], EMBASE (www.embase.com), and the master reference library collated by the American Heart Association (AHA). To identify the largest possible number of relevant articles, reviewers were also encouraged to perform hand searches of journals, review articles, and books as appropriate.

The reviewers documented the mechanism by which studies relevant to the hypothesis were selected. Specific study inclusion and exclusion criteria and study limitations were documented. Inclusion of all relevant evidence (from animal and manikin/model studies as well as human studies) was encouraged.

Step 2. Assess the Quality of Evidence^W277
In this step reviewers were asked to determine the level of evidence of relevant studies (Step 2A), assess the quality of study research design and methods (Step 2B), determine the direction of results (Step 2C), and cross-tabulate assessed studies (Step 2D).

The levels of evidence used for the 2005 consensus process (see Part 1 of this supplement)² were modified from those used in 2000.^3,4 In many situations summary conclusions were based on lower levels of evidence because human clinical trial data was not available.

The reviewers assessed the quality of research design and methods and allocated each study to 1 of 5 categories: excellent, good, fair, poor, or unsatisfactory. Studies graded as poor or unsatisfactory were excluded from further analysis.

Reviewers evaluated the direction of the study results as supportive, neutral, or opposed and then depicted the data in 1 of 2 grids. The grids were 2-dimensional, showing quality and levels of evidence. The reviewers completed a Supporting Evidence grid and a Neutral or Opposing Level of Evidence grid.

Step 3. Recommendation for Class of Recommendation
The 2005 AHA Guidelines for CPR and ECC⁵ use a class of recommendation system to indicate the overall strength of recommendations. These classes of recommendation were not used in the ILCOR 2005 CPR Consensus document contained in this supplement.⁶

In this step reviewers were invited to offer an opinion on the overall strength of a specific treatment recommendation for the AHA or other council-specific guidelines. Statements contained in this section of the worksheet reflect the reviewer’s opinion and may or may not be consistent with consensus conclusions from the 2005 Consensus Conference or the final recommendations in the 2005 AHA Guidelines for CPR and ECC or guidelines from other resuscitation councils.

Step 4. Reviewer’s Perspective and Potential Conflict of Interest^W277
All reviewers completed a conflict-of-interest disclosure form and also listed potential conflicts of interest on the worksheets. This ensured transparency of the review process. More details of the conflict-of-interest disclosure process are described in another editorial in this supplement.⁷

Step 5. Summary of the Science
^W277

Worksheet reviewers created a summary of the science. In the summary format reviewers were encouraged to provide a detailed discussion of the evidence, including the outcomes evaluated and the strengths and limitations of the data.

The final step in the science summary process was the creation of draft consensus on science statements and treatment recommendations. Statement templates were provided to standardize the comprehensive summary of information. Elements of the consensus on science statement template included the specific intervention or assessment tool, number of studies, levels of evidence, clinical outcome, population studied, and the study setting. Elements of the treatment recommendation template included specific intervention or assessment tool, population and setting, and strength of recommendation.

The statements drafted by the reviewers in the worksheets reflect the recommendations of the reviewers and may or may not be consistent with the conclusions of the 2005 Consensus Conference.

Step 6. References
Worksheet reviewers were asked to provide a database file containing the references that were used. The submitted references were added to the master reference library collated by the AHA.

Step 7. Posting on the Internet
Completed worksheets were posted on the Internet for further review. The initial process involved posting the worksheet to a password-protected area of the AHA Intranet (accessible to worksheet reviewers). In December 2004 the completed worksheets were posted on an Internet site that could be accessed by the public for further review and feedback before the 2005 Consensus Conference in Dallas (www.C2005.org).

	Controversies Encountered

Top Introduction Steps for Evidence Evaluation Controversies Encountered Summary References

 
Studies on Related Topics (LOE 7)
Many reviewers identified studies that answered related questions but did not specifically address the reviewer’s initial hypothesis. Examples include the extrapolation of adult data for pediatric worksheets and extrapolation of the results of glucose control in critically ill patients to the postresuscitation setting. Worksheet reviewers were instructed to clearly designate evidence that represented extrapolations. Reviewers could designate such studies as LOE 7, or they could assign a level of evidence based on the study design but include terms such as "extrapolated from" with specific relevant details in the draft consensus on science statements to indicate clearly that these were extrapolations from data collected for other purposes.

Animal Studies and Mechanical Models
Animal studies can be performed under highly controlled experimental conditions using extremely sophisticated methodology. Irrespective of methodology, all animal studies and all studies involving mechanical models (eg, manikin studies) were classified as LOE 6. Specific details about these studies (including methodology) are included in the summary of science where appropriate.

Studies Evaluating Diagnosis or Prognosis
The default levels of evidence used for the 2005 consensus process were not designed for the review of studies that evaluate diagnosis or prognosis. For these studies other methods of assigning levels of evidence were considered (such as those proposed by the Oxford Centre for Evidence-Based Medicine [http://www.cebm.net/]). Worksheet reviewers planning to include alternative levels of evidence were asked to define such levels clearly and to retain the default levels of evidence.

	Summary

Top Introduction Steps for Evidence Evaluation Controversies Encountered Summary References

 
The 2005 consensus process provided a large number of detailed literature reviews published on the Internet and summarized in this supplement. This review suggests that the evidence evaluation process for resuscitation literature will continue to evolve, providing a comprehensive process for collating data, summarizing the science, and facilitating its translation into treatment recommendations.

	Footnotes

 
The opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.

This article has been copublished in Resuscitation.

	References

Top Introduction Steps for Evidence Evaluation Controversies Encountered Summary References

 

1. Sackett D, Richardson W, Rosenberg W, Haynes R. Evidence-based Medicine. How to Practice and Teach EBM. London, England: Churchill Livingstone; 1997.
   
2. International Liaison Committee on Resuscitation. 2005 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care With Treatment Recommendations, Part 1: Introduction. Circulation. 2005; 112: III-1–III-4.
   
3. American Heart Association in collaboration with International Liaison Committee on Resuscitation. Guidelines 2000 for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation. 2000; 102 (suppl): I-1–I-384.[Abstract/Free Full Text]
   
4. Guidelines 2000 for Cardiopulmonary resuscitation and emergency cardiovascular care—an international consensus on science. Resuscitation. 2000; 46: 1–447.[CrossRef][Medline] [Order article via Infotrieve]
   
5. ECC Committee and Subcommittees, American Heart Association. 2005 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation. 2005; 112: IV-1–IV-211.
   
6. International Liaison Committee on Resuscitation. 2005 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations. Circulation. 2005; 112: III-1–III-136.
   
7. Billi JE, Zideman DA, Eigel B, Nolan J, Montgomery WH, Nadkarni VM, from the International Liaison Committee on Resuscitation and American Heart Association. Conflict of interest management before, during, and after the 2005 International Consensus Conference on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations [editorial]. Circulation. 2005; 112: III-131–III-132.
   
8. W277. http://circ.ahajournals.org/cgi/content/full/CIRCULATIONAHA.105.170522/DC450
   

This Article Free upon publication Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Zaritsky, A. Articles by Morley, P. T. Search for Related Content PubMed Articles by Zaritsky, A. Articles by Morley, P. T.