This Article Extract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Janket, S.-J. Articles by Meurman, J. H. Search for Related Content PubMed PubMed Citation Articles by Janket, S.-J. Articles by Meurman, J. H. Related Collections Risk Factors Other diagnostic testing Epidemiology

(Circulation. 2007;116:e133.)
© 2007 American Heart Association, Inc.

Correspondence

Letter by Janket et al Regarding Article, "Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction"

Sok-Ja Janket, DMD, MPH

Department of General Dentistry, Boston University School of Dental Medicine, Boston, Mass

Ye Shen, BS, MS

Division of Biostatistics, Yale School of Public Health, New Haven, Conn

Jukka H. Meurman, DOdont, MD, PhD

University of Helsinki Central Hospital, The Institute of Dentistry, Helsinki, Finland

To the Editor:

We read the article by Dr Nancy Cook with great interest.¹ Although we agree with Dr Cook that the likelihood ratio test may be more precise, the C-statistic is the only statistical moiety that assesses potential clinical utility of a regression model via juxtaposition of predicted probability of true positives and false-positives.

Relative risk is calculated by comparison of disease probability between those who have high levels of a biomarker and those who have low levels or none at all.² This reference (usually the first quartile) is chosen without clinical consideration and may generate a large number of false-positives. If regression results were to be applied to clinical practice directly, this could lead to inappropriate clinical decisions (eg, vaccination or prescription of medications with adverse effects to everyone with a biomarker level above the first quartile). Therefore, prior to clinical application of the research results, it is imperative to consider not only true-positives but also accompanying false-positives. The C-statistic provides simultaneous assessment of true- and false-positives at different levels of risk factors.

"No change in C-statistic," as Cook stated, indicates that other factors already in the model have explained what this marker contributes, and/or this marker may generate high false-positives and provides no additional information. When we tested these properties with a simpler version of The Women’s Health Study,³ C-reactive protein levels above the first quartile yielded a relative risk of 2.74 and identified 86% of true-positives correctly. However, 69% of those who do not have heart disease were incorrectly categorized as heart disease patients. These results are consistent with the clinical observation that not all persons with high C-reactive protein develop cardiovascular events⁴ and that C-reactive protein has low specificity.⁵ Would clinicians prescribe statins to patients if they knew the false-positive rate is 69%?

No matter how robust a biomarker appears to be in a regression analysis, it only provides the faade of a clinical picture. A sophisticated stochastical model remains theoretical, whereas the C-statistic, in spite of its limitations, gives us a glimpse of potential clinical relevance.⁶

As a side note, the inherent properties of sensitivity/specificity of inanimate objects such as an x-ray machine do not vary. The sensitivity/specificity derived from observational studies, however, should be considered cohort-specific because data collection methods, study design, or cohort characteristics engender variability in these test properties.

	Acknowledgments

 
Sources of Funding

Dr Janket is supported by the National Scientist Development Grant from The American Heart Association. Dr Meurman is supported by Grant TYH 3245 from the Helsinki University Central Hospital, Finland; the Ulf Nilsonne Foundation (SalusAnsvar Prize), Stockholm, Sweden; and the Paivikki and Sakari Sohlberg Foundation.

Disclosures

None.

	References

Top References

 

1. Cook N. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007; 115: 928–935.[Abstract/Free Full Text]
   
2. Wald NJ, Hackshaw AK, Frost CD. When can a risk factor be used as a worthwhile screening test? BMJ. 1999; 319: 1562–1565.[Free Full Text]
   
3. Levinson S, Elin R. What is C-reactive protein telling us about coronary artery disease? Arch Intern Med. 2002; 162: 389–392.[Free Full Text]
   
4. Libby P, Ridker, Maseri A. Inflammation and atherosclerosis. Circulation. 2002; 105: 1135–43.[Abstract/Free Full Text]
   
5. Kushner I, Rzewnicki D, Samols D. What does minor elevation of C-reactive protein signify? Am J Med. 2006; 119: 166.e17–166.e28.
   
6. Greenland P, O’Malley PG. When is a new prediction marker useful? A consideration of lipoprotein-associated phospholipase A2 and C-reactive protein for stroke risk [comment]. Arch Intern Med. 2005; 165: 2454–2456.[Free Full Text]
   

This article has been cited by other articles:

				S.-J. Janket, Y. Shen, and A. E. Baird Why must new cardiovascular risk factors be carefully re-assessed prior to clinical application? Eur. Heart J., May 2, 2008; 29(10): 1336 - 1337. [Full Text] [PDF]

This Article Extract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Janket, S.-J. Articles by Meurman, J. H. Search for Related Content PubMed PubMed Citation Articles by Janket, S.-J. Articles by Meurman, J. H. Related Collections Risk Factors Other diagnostic testing Epidemiology