Abstract 13773: A Bias Corrected Net Reclassification Improvement for Clinical Subgroups
Comparing prediction models using reclassification within subgroups at intermediate risk is often of clinical interest and has been used extensively in the cardiovascular literature. We found the current method to be substantially biased towards false positive results. Consequently, we propose a method for obtaining an unbiased estimate for the Net Reclassification Improvement (NRI) evaluated only on a subset, or clinical NRI. To do this we derived the expected value of the clinical NRI under the null hypothesis of no improvement in prediction using the same principles as the overall NRI. Our bias-corrected estimate is shown, unlike the naïve estimate commonly used in the literature, to be unbiased with a reasonable type 1 error. We applied our method to data from the Women's Health Study, a prospective cohort of 24,171 female health professionals, comparing models for 10-year cardiovascular risk with and without high-density lipoprotein cholesterol using cut points at 5%, 10% and 20%, with risk between 5% and 20% as the subset of interest. The naïve clinical NRI was 17.0% (95% CI 10.4 to 23.7) and the naïve p-value was less than 0.001. However, the bias-corrected clinical NRI was 4.0%, with a p-value of 0.24 (95% CI -2.7 to 10.7). Our proposed method is an improvement over currently used methods of calculating the clinical NRI and is recommended to reduce overly optimistic results.
- © 2011 by American Heart Association, Inc.