Abstract 4017: Automated Chart Abstraction Can Provide Highly Accurate Data Extraction For Clinical Quality Measures: Assessment of REMIND™ for CMS Heart Failure Measures
Background: There is growing need to measure clinical performance in order to assess quality. However, much of the information is stored in unstructured form (e.g. free text), or found in disparate data sources (e.g. pharmacy, labs). Additionally, clinical information in medical records may be contradictory and difficult to interpret. Therefore, trained nurses perform chart abstraction to extract these data. We assessed the hypothesis that a system for automated chart abstraction could reliably and efficiently extract CMS heart failure measures.
Methods: We analyzed 325 patients who were hospitalized with a primary diagnosis of heart failure. A total of 12 clinical measures, including LVF assessment, indication of LV systolic dysfunction, smoking history, smoking cessation counseling, discharge instructions, and whether the patient was prescribed ACE inhibitors or ARBs or had documented contra-indications, were extracted for each patient by a trained nurse. These patients were then analyzed by a novel applications called REMIND (Reliable Extraction and Meaningful Inference from Nonstructured Data) to automatically extract these same measures. REMIND uses natural language processing and Bayesian reasoning to combine information from multiple sources and automatically extract information from patient charts. REMIND was blinded to the results of manual extraction, and comparisons were done on each measure for each patient. Not all measures applied to all patients; for example, if a patient was a non-smoker, then smoking counseling was not considered. Specific definitions for each of these criteria were provided by the Specifications Manual for National Hospital Quality Measures.
Results: A total of 3335 answers were compared from the 325 patients. REMIND and the nurse abstractor agreed 3197 out of 3335 times, a 96% agreement. Many of the discrepancies were due to handwritten records that were not accessible by REMIND. Each patient chart abstraction took 15 minutes for the trained nurse. REMIND took approximately 1 second to process each patient chart (900 times faster).
Conclusions: REMIND can automatically extract clinical quality measures from patient records with high degree of accuracy and efficiency.