Abstract 16399: Use and Customization of Risk Scores for Predicting Cardiovascular Events Using Electronic Health Record Data
Background: The Framingham Risk Score (FRS) and the ACC/AHA Pooled Cohort Score (PCS) are widely used in clinical practice to guide individual patient care decisions. However, these risk scores have been estimated and validated mostly using data from longitudinal cohort studies and their performance when applied to patient data extracted from electronic health records is less well-established.
Methods: Risk factor and outcomes data were obtained from the electronic medical records and insurance claims of 84,130 adults aged 40-79 receiving care at a large health care delivery and insurance organization from 2001-2011. We assessed calibration and discrimination for four risk scores: the published versions FRS and PCS, and versions obtained by re-fitting the FRS and PCS Cox regression models using a subset of the available data. Population subgroups where the various models gave highly divergent risk predictions were identified using recursive partitioning techniques.
Results: The original FRS was well-calibrated (Calibration statistic K < 5.7, p = 0.13), but the original PCS was not (K = 30, p < 0.001). The original PCS yielded somewhat better discrimination than the original FRS (C-statistic C = 0.757 vs. 0.738). The refitted FRS (K = 1.4, p = 0.71 and C = 0.754) and refitted PCS (K = 8, p = 0.046 and C = 0.756) were both well-calibrated and showed good discrimination. The original and refitted FRS disagreed substantially on the risks of individuals within a number of population subgroups, most of which encompassed individuals under the age of 60; the original and refitted PCS differed substantially in their risk predictions for African-American males; and the refitted FRS and PCS produced similar risk predictions across subgroups.
Conclusions: Both the FRS and PCS are appropriate for use in clinical decision support systems which rely on electronic health data, though it may be advisable to refit the statistical models they are based on using available data from the target population to ensure acceptable calibration and discrimination performance.
Author Disclosures: J. Wolfson: Research Grant; Modest; Supported by NIH grant HL102144. D.M. Vock: None. S. Bandyopadhyay: Research Grant; Significant; Supported by NIH grant HL102144. G. Adomavicius: None. P.J. O'Connor: Employment; Significant; Employed full-time at HealthPartners Institute for Education and Research. Research Grant; Significant; Current research funding from NIH on multiple projects including: HL102144, DK092924, HL090965, HS019859, HL115082, DK092317, MH092201, CMS-1B1-11-001.. Honoraria; Modest; Paid travel for consulting or grand rounds at: Mayo Clinic, Peking University, and University of Vienna in last several years. Other; Significant; Patent No. US 8355348 B2. G.X. Vazquez-Benitez: Employment; Significant; Employed full-time at HealthPartners Institute for Education and Research.
- © 2015 by American Heart Association, Inc.