| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Circulation. 2008;117:371-378.)
© 2008 American Heart Association, Inc.
Health Services and Outcomes Research |
From the Departments of Medical Informatics (J.T.v.W., M.A.M.v.W., M.C.J.M.S., M.M., P.W.M., J.v.d.L.) and Epidemiology and Biostatistics (M.C.J.M.S.), ErasmusMC University Medical Center Rotterdam, Rotterdam, the Netherlands.
Correspondence to Johan van der Lei, MD, PhD, Department of Medical Informatics, ErasmusMC University Medical Centre Rotterdam, PO Box 2040, 3000 CA Rotterdam, the Netherlands. E-mail j.vanderlei{at}erasmusmc.nl
Received March 16, 2007; accepted October 5, 2007.
| Abstract |
|---|
|
|
|---|
Methods and Results— In a clustered randomized trial design, 38 Dutch general practices (77 physicians) and 87 886 of their patients (39 433 men 18 to 70 years of age and 48 453 women 18 to 75 years of age) who used the ELIAS electronic health record participated. Each practice was assigned to receive alerts, on-demand support, or no intervention. We measured the percentage of patients screened and treated after 12 months of follow-up. In the alerting group, 65% of the patients requiring screening were screened (relative risk versus control=1.76; 95% confidence interval, 1.41 to 2.20) compared with 35% of patients in the on-demand group (relative risk versus control=1.28; 95% confidence interval, 0.98 to 1.68) and 25% of patients in the control group. In the alerting group, 66% of patients requiring treatment were treated (relative risk versus control=1.40; 95% confidence interval, 1.15 to 1.70) compared with 40% of patients (relative risk versus control=1.19; 95% confidence interval, 0.94 to 1.50) in the on-demand group and 36% of patients in the control group.
Conclusion— The alerting version of the clinical decision support systems significantly improved screening and treatment performance for dyslipidemia by general practitioners.
Key Words: behavior clinical decision support systems computers prevention statins lipids
| Introduction |
|---|
|
|
|---|
Editorial p 336
Clinical Perspective p 378
Cardiovascular disease (CVD) is the leading cause of mortality in industrialized countries.5 Researchers report low primary and secondary prevention performance of physicians.6–11 To improve both primary and secondary prevention of CVD, we developed the noncommercial CDSS, named CholGate, on the basis of recommendations for lipid management from the guidelines of the Dutch College of General Practitioners (DCGP).12–16 For more than a decade, >90% of Dutch primary care physicians have used electronic health records (EHR) for all primary care record activities.17 Taking advantage of this widespread use of EHR by Dutch general practitioners, the CDSS is integrated within the EHR to provide decision support as part of the clinicians workflow. We conducted a randomized trial to compare the effect of 2 versions of this CDSS with no CDSS on screening and treatment of dyslipidemia among general practitioners.
We hypothesize that a CDSS will improve physicians performance compared with having no CDSS. In addition, we hypothesize that the alerting method will have a better effect in increasing physician performance than a CDSS that requires manual activation.
| Methods |
|---|
|
|
|---|
Setting
Between May and June 2004, all of the 56 practices using the ELIAS EHR (iSOFT B.V., Leiden, the Netherlands) in the Delft region in the Netherlands were invited to participate in the study. We chose to integrate our intervention into a single EHR to avoid potential bias due to different EHR systems. Only practices that fully replaced paper-based records with electronic records during patient encounters and who have been working in this manner for
1 year were eligible for the study.
Participants
A total of 38 practices (80 general practitioners) with 92 054 eligible patients agreed to participate in the study. Table 1 lists the practice characteristics. Because the 1999 DCGP guidelines restricted lipid management recommendations to men aged 18 to 70 years and women aged 18 to 75 years, we included only these patients in our study.12–16 All patient data were anonymous. The need to obtain patient consent was waived by the Ethical Review Committee of the Erasmus Medical Center because, in accordance with Dutch law, the intervention constituted accepted care and physicians were not forced to follow any suggestions. The Figure shows the study flow.
|
|
Randomization
The general practice, with
1 practitioners, was chosen as unit of randomization because it was a definable entity and a logical foundation for implementing a primary care–based intervention. Additionally, this prevented knowledge gained from decision support in one patient influencing the action on a possible control patient, thereby underestimating the effect of the intervention.18 Practices were randomly assigned into 1 of 3 arms: a control arm and 2 intervention arms (an on-demand arm and an alerting arm). The practices were first stratified by single-handed practices and group practices (ie,
2 general practitioners in the same practice). Each practice was subsequently assigned by simple random allocation to CDSS alerting, CDSS on-demand, or control groups for the complete study period. Randomization was performed with a table of random numbers by a researcher not involved in the study and who was blind to the identity of the practices. The practices were blinded to the other versions of the CDSS available.
CDSS Intervention
The CDSS aids a general practitioner in the primary and secondary prevention of CVD with regard to lipid management. The system is based on the guidelines of the DCGP and is integrated into the EHR. The system uses 3 layers of data available in the EHR to provide decision support: coded data, structured data, and free text data. Coded data rely on widely used coding schemes to record data. In the Netherlands, all practices use the International Classification for Primary Care for coding diagnoses and the Anatomic Therapeutical Classification19 for coding medication. Structured data rely on locally defined coding schemes. For each practice we identified codes on the level of an individual practice used for specific conditions or measurements. Free text data consist of data recorded in free text. For each practitioner, we relied on the recording habits of physicians who, in busy daily practice, often use repetitive free text in their medical narrative.20 After gathering relevant data, the system identifies 2 types of patients: (1) those who need screening for lipid abnormalities and (2) those who need treatment for lipid abnormalities. The Dutch guidelines emphasize the need to screen specific, well-defined populations.21 As a result, the guidelines contain detailed, sometimes complex descriptions of the various conditions that should lead to screening a patient.22 For example, a 50-year–old male smoker without diabetes mellitus or CVD but with hypertension needs screening, whereas as if the patient does not have hypertension, screening is not indicated. If the patient is aged
60 years and smokes but has no other risk factors, screening is again indicated. All patients with known CVD needed screening. When a patient is identified as requiring screening, the guidelines recommend specific actions. The 1999 DCGP guidelines require calculation of a risk score for every patient who needs primary prevention.12–16 This risk score is based on the Framingham risk function for CVD.23 In the 1999 paper-based guidelines, the physician had to deduce the score from a series of tables stratified by age, gender, diabetes mellitus, systolic blood pressure, smoking, and cholesterol/high-density lipoprotein cholesterol ratio. For every age group, a threshold Framingham score exists in which treatment is recommended. Researchers have argued that the relevant tables are complex and might be difficult to use in daily practice.24 In secondary prevention, any patient with CVD attributed to atherosclerosis and with a serum cholesterol level >5 mmol/L (193 mg/dL) should be treated with statins. Once a patient is treated with statins, the 1999 guidelines did not recommend further serum lipid testing.
We developed 2 versions of the CDSS: on-demand and alerting. Both versions analyze and interpret the patient data in the EHR, generating patient-specific guideline recommendations for preventative activities. An overview screen, or dashboard, accessible from the patient record screen presents a patients current risk profile to the user, as well as suggesting an action on lipid management. A user can call on this overview screen at any point during the interaction with the patients record. This overview is interactive: The user can enter a new diagnosis or change measurement values, and the risk profile is immediately updated. All changes made in the overview are registered into the patients record, preventing duplication of tasks. In both versions the user has access to the overview screen to see a patients risk profile and recommendations. The only difference between the 2 versions is the alerting functionality.
In the on-demand version, a user has to actively initiate the overview screen to access the recommendations. In the alerting version, the recommendations are automatically shown to the user during record interaction; that is, the user does not need to initiate the overview screen to access the recommendations.
Outcomes and Measurements
The primary study outcomes were the percentage of screened and treated patients determined by the system according to the DCGP guidelines recommendations on lipid management in relation to the primary and secondary prevention of CVD.12–16 The percentage of screened patients is the number of patients requiring screening and then screened, divided by the total number of patients requiring screening. The percentage of treated patients is the number of patients requiring treatment and then treated, divided by the total number of patients requiring treatment.
Follow-Up Procedures
When the system was implemented by the authors at the practices, it was configured to the local practice setting, and users were given a tutorial on the use of the system. We installed the on-demand version in the on-demand arm and the alerting version in the alerting arm. In the control arm, the CDSS was installed, but all functionality was disabled for daily use and only monitored the actions taken by the general practitioner. At all practices, an initialization procedure was performed directly after installation. This procedure gathered data on the practices baseline characteristics as well as baseline characteristics of the patient population.
After initialization, all data on patient record interactions, patient characteristics, and follow-up data were obtained from the EHR by the CDSS. Eligibility of patients for screening and treatment actions were determined from the computerized patient data (diagnoses, problems, prescriptions) that were available in the EHR. Eligible patients entered the study at the moment the patient record was opened in the practice during the study period. At that moment the CDSS classified the patient as requiring screening or requiring treatment. If a patient required screening, the number of days from entry until screening or follow-up ended was counted. If a patient required treatment, the number of days from entry until treatment or follow-up ended was counted. Patients who were screened after being classified as requiring screening and subsequently classified as requiring treatment were included in both the screening analysis and treatment analysis.
All patients were followed for at least 1 year, or until the general practice switched to another EHR vendor (that is, stopped using the ELIAS system) or until the patient died.
Statistical Analysis
Logistic regression analysis was used to compare practice characteristics at baseline between the 2 treatment arms. Cox regression analyses were used for comparison of screening and treatment rates between intervention arms.
For comparison of screening rates, time of follow-up was considered as time axis in the Cox regression analysis. This analysis was restricted to persons identified as requiring screening by the CDSS. Because the number of visits during follow-up affects the possibility of being screened, we adjusted for this covariate. We adjusted for CVD and diabetes mellitus, and no other covariates had a material impact on the results.
For comparison of treatment rates, time of follow-up (from requiring treatment until censoring) was used as time axis in the Cox regression analysis. This analysis was restricted to persons classified by the CDSS as requiring treatment.
To estimate the effect of the interventions while taking into account the clustered randomization, we conducted a clustered Cox regression analysis by using PROC PHREG. All analyses were performed with SAS release 8.2 (SAS Institute Inc, Cary, NC).
The authors had full access to and take full responsibility for integrity of the data. All authors have read and agree to the manuscript as written.
| Results |
|---|
|
|
|---|
Practice characteristics differed slightly between the treatment arms. For example, screening and treatment performance in the year preceding the intervention were lower in the on-demand arm (Table 1). Patient characteristics were similar between the study arms, except for the percentage of smokers, family history of CVD, and systolic blood pressure (Table 2).
|
For 3210 of the 87 866 patients, the system determined that according to the DCGP guidelines, screening was needed. The percentage was highest in the on-demand arm (Table 3). The majority of patients who were identified as requiring screening were male and aged
60 years,
40% had CVD (primarily angina or a prior myocardial infarction), and
26% had diabetes mellitus (Table 3).
|
For 2953 of the 87 866 patients, the system determined that treatment was needed. A total of 367 patients who were screened during the study period required treatment. Most patients requiring treatment had a Framingham risk score >20%. Between 40% and 50% of patients required treatment because of prior CVD (Table 4).
|
Sixty-five percent of patients requiring screening in the alerting arm were screened versus 35% in the on-demand arm and 25% in the control arm (Table 5). The alerting arm had a 76% increased likelihood to be screened than patients in the control arm (adjusted relative risk [RR]=1.76; 95% confidence interval [CI], 1.41 to 2.20), whereas patients in the on-demand arm had a 28% increased likelihood of being screened (adjusted RR=1.28; 95% CI, 0.98 to 1.68). Of the patients requiring treatment, 66% were treated in the alerting arm, 40% in the on-demand arm, and 36% in the control arm. After adjustment for differences between arms, the likelihood of being treated was 40% higher in the alerting arm (adjusted RR=1.40; 95% CI, 1.15 to 1.70) and 19% higher in the on-demand arm in comparison to the control arm (adjusted RR=1.19; 95% CI, 0.94 to 1.50). The patients in the alerting arm had a significantly higher likelihood of being screened than the on-demand arm (adjusted RR=1.40; 95% CI, 1.08 to 1.81) (Table 5). The treatment likelihood was also elevated but not significantly so (RR=1.18; 95% CI, 0.96 to 1.45).
|
| Discussion |
|---|
|
|
|---|
Explanation of Findings
In busy general practice, time is frequently insufficient to calculate the complex risk scores necessary for decisions on primary prevention. The time needed to read and interpret paper-based guidelines might hamper physician performance. As our results show, displaying recommendations from guidelines to making complex decisions in a way that fits into the workflow of the physician increases performance by obviating the break in the clinical workflow.4
That a system using alerts is associated with better performance than an on-demand system can be explained by the fact that alerting creates awareness of preventive actions that should be taken for a specific patient. In the absence of an alerting system, the physician has to recognize the situation that calls for action. In a previous study we showed that physicians awareness of the need for secondary prevention is frequently triggered by a CVD event25; in these situations, the CVD event functions as an alert. Alerting constitutes a triggering event that prompts the physician to initiate preventive care.
Findings in Relation to Other Studies
In a recent updated major review of 100 CDSS studies on practitioner performance and patient outcomes, the authors found that improved practitioner performance was associated with CDSS that automatically prompted (alerted) users compared with requiring users to activate the system.2 However, none of the 100 studies under review directly studied this observation. Our work is the first study that confirms this association in a randomized controlled trial. Recently, authors have identified the critical determinants for CDSS to be successful.2,4,26 However, we did not evaluate the effect of these factors in our study but rather kept them similar across the intervention groups. This means that we could measure the difference between the intended interventions (alerting and on-demand activation) without needing to consider the various system effects listed by others.
Differences existed between the DCGP guidelines in force during the study period and other international guidelines on dyslipidemia.27,28 The 1999 DCGP guidelines did not advocate routine low-density lipoprotein testing. In addition, various treatment thresholds differed. However, all guidelines required physicians to act on certain information related to CVD risk factors and perform screening or treatment actions. Therefore, although the guidelines differ on what tests should be performed in screening and exactly when to treat, they require the physicians to act on information related to risk factors. Because the need to perform preventive actions is a result of the information about risk factors and not the exact content of a particular guideline, we believe that our results are generalizable to settings in which different guidelines apply.
The clinical implications of our findings could be profound. It is known that both primary and secondary prevention of CVD is suboptimally performed.6–11 Our study shows that interventions such as a CDSS can improve the primary and secondary prevention of CVD and especially if general practitioners are alerted to the need for action. It can be argued that an increase in adherence to the guidelines, as effected by our system, will lead to a decrease in mortality and morbidity due to CVD. However, further research will have to show that this is indeed the case.
Limitations
Recently, authors argued that evaluation of CDSS should focus on the effect on patient outcomes.2 Factors other than physicians actions influence patient outcomes. Patient adherence, for example, is a well-documented factor related to outcomes but is not measured in CDSS studies.29,30 Our study only assesses the effect of CDSS on changing physicians behavior by measuring their actions and does not evaluate patient outcomes. In performing the study, we did not question the validity of the guidelines but studied which approach could best influence physicians behavior given the guidelines. Thus, we believe that if the CDSS is based on well-accepted evidence-based guidelines, improvement in physician adherence to guidelines will result in better patient outcomes. Guidelines, however, evolve over time. Although a CDSS may be an effective intervention to change behavior and affect outcome, systems based on those guidelines need to be continuously updated to reflect changing guidelines.
To ensure that complete medical records were available to the system, general practitioners were only eligible to participate in this study if they did not use any paper records at all. The performance of the CDSS, however, may be influenced by the way physicians capture data in their EHR. We did not use an independent mechanism to assess the physicians EHR.
Randomization was conducted on a practice level. To deal with potential differences in patient characteristics between treatment arms, we adjusted through the Cox regression for baseline differences that confounded the effect estimate while taking the clustered design into account. Because of logistical reasons, the follow-up time differed between practices. Because time is an important determinant for screening and treatment, we adjusted for this difference.
Apart from screening for all CVD risk factors, we focused on treating only 1 risk factor: lipid abnormalities. We did not perform CDSS treatment interventions on other CVD risk factors. Performing these extra interventions will induce additional complexity to the system.21,31 Although we have been able to introduce a CDSS that effectively supports the longitudinal lipid management in primary and secondary prevention of CVD, further research will have to show whether introducing more workflow patterns in the CDSS will have additional effects on physicians performance.
In conclusion, this study shows that an alerting version of a CDSS significantly improves screening and treatment performance for dyslipidemia.
| Acknowledgments |
|---|
Sources of Funding
The study was funded by a nonspecific grant from the Dutch Heart Foundation (2000.B161). The Dutch Heart Foundation in no way interfered with the design or execution or reporting of the study. The system was free to all participants and is not being marketed as a commercial application.
Disclosures
None.
| References |
|---|
|
|
|---|
| Footnotes |
|---|
Related Article:
Circulation 2008 117: 331-332.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Circulation Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2008 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |