doi: 10.1161/CIRCULATIONAHA.105.594929
(Circulation. 2007;115:654-657.)
© 2007 American Heart Association, Inc.
Statistical Primer for Cardiovascular Research |
Receiver-Operating Characteristic Analysis for Evaluating Diagnostic Tests and Predictive Models
From Children’s Hospital Boston (K.H.Z.), Harvard Medical School (K.H.Z., A.J.O., L.M.), Brigham and Women’s Hospital (L.M.), and Harvard Clinical Research Institute (L.M.), Boston, Mass.
Correspondence to Kelly H. Zou, PhD, Department of Health Care Policy, Harvard Medical School, 180 Longwood Ave, Boston, MA 02115. E-mail kelly.zou@childrens.harvard.edu
Key Words: diagnosis • ROC curve • sensitivity and specificity • statistics • tests
An extract of the first 250 words of the full text is provided, because this article has no abstract. |
 |
Introduction |
|---|
Receiver-operating characteristic (ROC) analysis was originally developed during World War II to analyze classification accuracy in differentiating signal from noise in radar detection.1 Recently, the methodology has been adapted to several clinical areas heavily dependent on screening and diagnostic tests,2–4 in particular, laboratory testing,5 epidemiology,6 radiology,7–9 and bioinformatics.10
ROC analysis is a useful tool for evaluating the performance of diagnostic tests and more generally for evaluating the accuracy of a statistical model (eg, logistic regression, linear discriminant analysis) that classifies subjects into 1 of 2 categories, diseased or nondiseased. Its function as a simple graphical tool for displaying the accuracy of a medical diagnostic test is one of the most well-known applications of ROC curve analysis. In Circulation from January 1, 1995, through December 5, 2005, 309 articles were published with the key phrase "receiver operating characteristic." In cardiology, diagnostic testing plays a fundamental role in clinical practice (eg, serum markers of myocardial necrosis, cardiac imaging tests). Predictive modeling to estimate expected outcomes such as mortality or adverse cardiac events based on patient risk characteristics also is common in cardiovascular research. ROC analysis is a useful tool in both of these situations.
In this article, we begin by reviewing the measures of accuracy—sensitivity, specificity, and area under the curve (AUC)—that use the ROC curve. We also illustrate how these measures can be applied using the evaluation of a hypothetical new diagnostic test as an example.
|
Diagnostic Test and Predictive Model |
|---|
A diagnostic classification test typically yields binary, ordinal, or continuous outcomes. The simplest type, binary
This article has been cited by other articles:
 |
|
 |
|
  H Adams, A Tzankov, A Lugli, and I Zlobec New time-dependent approach to analyse the prognostic significance of immunohistochemical biomarkers in colon cancer and diffuse large B-cell lymphoma J. Clin. Pathol., November 1, 2009; 62(11): 986 - 997. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. J. S. Windle, G. A. Rose, R. Devillers, and M.-J. Fortin Exploring spatial non-stationarity of fisheries survey data using geographically weighted regression (GWR): an example from the Northwest Atlantic ICES J. Mar. Sci., September 4, 2009; (2009) fsp224v1. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. J. Price Bedside Evaluation of Thienopyridine Antiplatelet Therapy Circulation, May 19, 2009; 119(19): 2625 - 2632. [Full Text] [PDF]
|
|
|
|
 |
|
 F. Drago, M. S. Russo, M. S. Silvetti, A. De Santis, and M. T. Naso Onofrio 'Time to effect' during cryomapping: a parameter related to the long-term success of accessory pathways cryoablation in children Europace, May 1, 2009; 11(5): 630 - 634. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 V. Nambi, R. C. Hoogeveen, L. Chambless, Y. Hu, H. Bang, J. Coresh, H. Ni, E. Boerwinkle, T. Mosley, R. Sharrett, et al. Lipoprotein-Associated Phospholipase A2 and High-Sensitivity C-Reactive Protein Improve the Stratification of Ischemic Stroke Risk in the Atherosclerosis Risk in Communities (ARIC) Study Stroke, February 1, 2009; 40(2): 376 - 381. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. L. Benowitz, J. T. Bernert, R. S. Caraballo, D. B. Holiday, and J. Wang Optimal Serum Cotinine Levels for Distinguishing Cigarette Smokers and Nonsmokers Within Different Racial/Ethnic Groups in the United States Between 1999 and 2004 Am. J. Epidemiol., January 15, 2009; 169(2): 236 - 248. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Soreide Receiver-operating characteristic curve analysis in diagnostic, prognostic and predictive biomarker research J. Clin. Pathol., January 1, 2009; 62(1): 1 - 5. [Full Text] [PDF]
|
|
|
|
 |
|
 C. Velik-Salchner, S. Maier, P. Innerhofer, W. Streif, A. Klingler, C. Kolbitsch, and D. Fries Point-of-Care Whole Blood Impedance Aggregometry Versus Classical Light Transmission Aggregometry for Detecting Aspirin and Clopidogrel: The Results of a Pilot Study Anesth. Analg., December 1, 2008; 107(6): 1798 - 1806. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. M. Miller, C. E. Rochitte, M. Dewey, A. Arbab-Zadeh, H. Niinuma, I. Gottlieb, N. Paul, M. E. Clouse, E. P. Shapiro, J. Hoe, et al. Diagnostic Performance of Coronary Angiography by 64-Row CT N. Engl. J. Med., November 27, 2008; 359(22): 2324 - 2336. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. C. Saumarez, M. Pytkowski, M. Sterlinski, J. P. Bourke, J. R. Clague, S. M. Cobbe, D. T. Connelly, M. J. Griffith, P. P. McKeown, K. McLeod, et al. Paced ventricular electrogram fractionation predicts sudden cardiac death in hypertrophic cardiomyopathy Eur. Heart J., July 1, 2008; 29(13): 1653 - 1661. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K. Rossing, H. Mischak, M. Dakna, P. Zurbig, J. Novak, B. A. Julian, D. M. Good, J. J. Coon, L. Tarnow, P. Rossing, et al. Urinary Proteomics in Diabetes and CKD J. Am. Soc. Nephrol., July 1, 2008; 19(7): 1283 - 1290. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Levrat, A. Gros, L. Rugeri, K. Inaba, B. Floccard, C. Negrier, and J.-S. David Evaluation of rotation thrombelastography for the diagnosis of hyperfibrinolysis in trauma patients Br. J. Anaesth., June 1, 2008; 100(6): 792 - 797. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. Hachamovitch and M. F. Di Carli Methods and Limitations of Assessing New Noninvasive Tests: Part I: Anatomy-Based Validation of Noninvasive Testing Circulation, May 20, 2008; 117(20): 2684 - 2690. [Full Text] [PDF]
|
|
|
|
|
|
J. Schwitter, C. M. Wacker, A. C. van Rossum, M. Lombardi, N. Al-Saadi, H. Ahlstrom, T. Dill, H. B.W. Larsson, S. D. Flamm, M. Marquardt, et al. MR-IMPACT: comparison of perfusion-cardiac magnetic resonance with single-photon emission computed tomography for the detection of coronary artery disease in a multicentre, multivendor, randomized trial Eur. Heart J., February 2, 2008; 29(4): 480 - 489. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. L. del Rio, P. I. McConnell, M. Kukielka, R. Dzwonczyk, B. D. Clymer, M. B. Howie, and G. E. Billman Electrotonic remodeling following myocardial infarction in dogs susceptible and resistant to sudden cardiac death J Appl Physiol, February 1, 2008; 104(2): 386 - 393. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. R. Hunsaker, K. H. Zou, A. C. Poh, B. Trotman-Dickenson, F. L. Jacobson, R. R. Gill, and S. Z. Goldhaber Routine Pelvic and Lower Extremity CT Venography in Patients Undergoing Pulmonary CT Angiography Am. J. Roentgenol., February 1, 2008; 190(2): 322 - 326. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. R. Cook Statistical Evaluation of Prognostic versus Diagnostic Models: Beyond the ROC Curve Clin. Chem., January 1, 2008; 54(1): 17 - 23. [Abstract] [Full Text] [PDF]
|
|