(Circulation. 1995;91:899-900.)
© 1995 American Heart Association, Inc.
Articles |
Risk, Predicting Outcomes, and Improving Care
From the Cardiology Section, Denver VA Medical Center, and the University of Colorado School of Medicine, Denver.
Correspondence to Dr Hammermeister, Cardiology (111B), 1055 Clermont, Denver, CO.
Key Words: Editorials • clinical trials • mortality
 |
Introduction |
|---|
 Top Introduction References |
|---|
Although outcomes research and outcomes-driven quality assessment and improvement are the fads of the nineties, interest in healthcare outcomes is hardly new. The Code of Hammurabi in the 19th century BC specified the first outcomes-based standard of care: "If a physician shall make a severe wound with an operating knife and kill (the patient). . .(the surgeon's) hand shall be cut off."1 More recently, Earnest Amory Codman believed so strongly in the use of outcomes to assess the quality of surgery that he formed his own hospital, the End-Result Hospital, in Boston in 1914. We are in the midst of rediscovering what Codman told the Philadelphia Medical Society in 1913: "We must formulate some method of hospital report showing as nearly as possible what are the results of the treatment obtained at different institutions. This report must be made out and published by each hospital in a uniform manner, so that comparison will be possible. With such a report as a starting point, those interested can begin to ask questions as to the management and efficiency."2
It is almost intuitive that if outcomes are to be used as quality indicators, they must be adjusted for severity of patient illness. Iezzoni,3 using the phrase "algebra of effectiveness," stated that healthcare outcomes are a function of clinical and other patient attributes, effectiveness of care, and other factors, including random events. However, the adjustment of healthcare outcome rates for severity of illness and comorbidity is of recent vintage.4 The first large-scale use of risk adjustment driven by reimbursement issues, the Diagnosis Related Groups (DRGs), were introduced for prospective payment for Medicare enrollees in 1983.3 In 1986, the Health Care Financing Agency (HCFA) released hospital mortality rates for Medicare patients.5 6 However, considerable concern was raised over the adequacy of risk adjustment using the limited data available in the Medicare billing database,7 8 9 such that HCFA has stopped the release of these hospital mortality reports.
Although the limited risk-adjusted outcomes from computerized discharge abstract and billing databases may still have uses in quality improvement, attention has turned to more careful risk adjustment measures of severity of illness and comorbidity obtained from the medical record—in part because this seems to be more credible to clinicians. Much of this work has occurred in cardiac surgery. The comparison of predicted and observed operative mortalities as a quality indicator was initiated in single centers in the mid 1980s.10 11 In 1987, we initiated the VA Continuous Improvement in Cardiac Surgery Study, which monitors all patients undergoing cardiac surgery at the 43 VA medical centers that perform these procedures.12 13 Similar risk-adjusted, outcomes-driven quality improvement programs were initiated soon thereafter in New York State14 and northern New England.15
The article by Tu and colleagues16 in
this issue of
Circulation from the province of Ontario, Canada,
represents an important addition to our knowledge of how to
apply outcomes to quality improvement. The initial impetus for the
Ontario Provincial Adult Cardiac Care Network (PACCN) database was to
triage cardiac surgery patients according to the urgency of their
surgery, as there were substantial waiting lists. The cardiovascular
clinical data from the PACCN database were supplemented by comorbidity
and outcomes data (hospital mortality, intensive care unit [ICU]
length of stay, and postoperative length of stay) from an
administrative database. Like many investigators in this field, Tu and
colleagues used logistic regression to identify preoperative patient
characteristics predictive of hospital death, very long ICU length of
stay (
6 days), and very long postoperative length of stay (17
days). However, one of the significant contributions of this study is
that the authors greatly simplified the estimation of these outcomes by
producing a single additive model that is easily remembered as a set of
integer scores for only six risk variables: age (<65 years, 0; 65 to
74, 2; 75, 3), sex (male, 0; female, 1), left ventricular function
(ejection fraction >0.50, 0; 0.35 to 0.50, 1; 0.30 to 0.35, 2; <0.20,
3), type of surgery (coronary artery bypass, 0; single valve
replacement, 2; multivalve or coronary artery bypass graft plus valve,
3), urgency of surgery (elective, 0; urgent, 1; emergency, 4), and
reoperation (no, 0; yes, 2). Out of a total of 16 points, patients with
risk index scores of 0 to 3 are considered low risk; 4 to 7,
intermediate risk; and 8, high risk. This simplification has been
obtained at no apparent loss in predictive power, as their C-index (a
measure of the ability of the risk index to discriminate between
hospital survivors and deaths, with 1.0 being perfect discrimination
and 0.5 being no better than chance) of 0.75 is comparable to several
other models (including ours) that require computers to calculate the
estimated risk of operative death.
Predicted outcomes may be used both for clinical decision making in individual patients and for assessing quality of care by comparing observed and predicted outcomes aggregated by hospital or care provider. The primary impetus of our own work and that ongoing in the state of New York and northern New England has been quality improvement. All three programs feed back summarized risk and outcome information to participating centers and providers, and all three programs have observed a drop in risk-adjusted operative mortality. The role of predicted outcomes in individual patient decision making is still being explored. We have preliminary data showing that our statistical model is able to predict operative mortality about as well as a physician's subjective estimate and that combining the subjective estimate with that from the statistical model provides better discrimination than either alone.
The current demand and enthusiasm for healthcare outcomes information frequently overlooks the limitations of outcomes-driven quality improvement. First and most importantly, risk-adjusted outcomes probably are not very good predictors of quality of care. There is virtually no information to indicate what proportion of healthcare outcomes are the result of patient risk and what proportion result from the processes and structures of care, the quality factors. The most productive use of healthcare outcomes will be "as cues that prompt and motivate the assessment of process and structure in a search for causes that can be remedied."17
There are other important limitations to the ways that outcomes are currently being used for quality assessment and improvement. In most programs, care provider participation in the collection, analysis, and interpretation of the outcomes data is minimal, in direct violation of one of the tenets of continuous quality improvement. The duration of time between the care episode and feedback to the care provider is long—up to a year or more. Data collection is by chart abstraction—both expensive and limited by what has been documented in the medical record. Finally, the sole reliance on outcomes for quality improvement frequently results in focusing on the high outliers—the "bad apples." This finger-pointing is likely to produce a defensive posture that "poisons improvement in quality since it inevitably leads to disaffection, distortion of information, and the loss of a chance to learn."18
Recognizing these deficiencies in current outcomes-driven quality assessment and improvement programs, including our VA Continuous Improvement in Cardiac Surgery Study, we are proposing a new paradigm called participatory continuous improvement.19 This model, currently under development for ischemic heart disease for the Department of Veterans Affairs, will provide patient-specific practice guidelines together with predicted outcomes and costs for several alternative clinical pathways at the point of care. The primary tool for implementation will be a computerized medical record. Our fundamental hypothesis is this: If we, the care providers, have information available to us at the point of care that describes the quality, accessibility, and costs of our care in the context of our peers and society's needs, this information will enable practice decisions that will lead to improved quality, accessibility, and cost-effectiveness of our care.19
Received December 14, 1994; accepted December 14, 1994.
|
References |
|---|
|
TopIntroductionReferences |
|---|
-
Vibbert S. What Works. Whittle Direct Books
LP; 1993:15.
-
Codman EA. The product of a hospital. Surg Gynecol
Obstet. 1914;18:491-496.
-
Iezzoni LI. Measuring the severity of illness and case mix.
In: Goldfield N, Nash DB, eds. Providing Quality Care: the
Challenge to Clinicians. Philadelphia, Pa: American College of
Physicians; 1989:70-105.
-
Roemer MI, Moustafa AT, Hopkins CE. A proposed hospital
quality index: hospital death rates adjusted for case severity.
Health Serv Res. 1968;3:96-118. [Medline]
[Order article via Infotrieve]
-
Brinkley J. US releasing lists of hospitals with abnormal
mortality rates. The New York Times. March 12, 1986: page
1.
-
Health Care Financing Administration. Medicare Hospital
Mortality Information, volume I, 1986 (HCFA publication No.
01-002). Washington, DC: US Government Printing Office; 1987.
-
Blumberg MS. Comments on HCFA hospital death rate statistical
outliers. Health Serv Res. 1987;21:715-740. [Medline]
[Order article via Infotrieve]
-
Kouchoukos NT, Ebert PA, Grover FL, Lindesmith GG. Report of
the ad hoc committee on risk factors for coronary artery bypass
surgery. Ann Thorac Surg. 1988;45:348-349. [Abstract]
-
Green J, Wintfeld N, Sharkey P, Passman LJ. The importance of
severity of illness in assessing hospital mortality.
JAMA. 1990;263: 241-246.
-
Junod FL, Bradley BJ, Payne J, Smeloff EA, Miller GE, Kelly PB
Jr, Ross KA, Shankar KG, McDermott JP. Preoperative risk assessment in
cardiac surgery: comparison of predicted and observed results.
Ann Thorac Surg. 1987;43:59-64. [Abstract]
-
Edwards FH, Albus RA, Zajtchuk R, Graeber GM, Barry M. Quality
assurance model of operative mortality in coronary artery surgery.
Ann Thorac Surg. 1989;47:646-649. [Abstract]
-
Grover FL, Hammermeister KE, Burchfiel C, Cardiac Surgeons of
the Department of Veterans Affairs. Initial report of the Veterans
Administration Preoperative Risk Assessment Study for cardiac surgery.
Ann Thorac Surg. 1990;50:12-28. [Abstract]
-
Hammermeister KE, Johnson R, Marshall G, Grover F. Continuous
assessment and improvement in quality of care: a model from the
Department of Veterans Affairs Cardiac Surgery. Ann Surg. 1994;219:281-290. [Medline]
[Order article via Infotrieve]
-
Hannan EL, Kilburn H, O'Donnel JF, Lukacik G, Shields EP.
Adult open heart surgery in New York State: an analysis of risk
factors and hospital mortality rates. JAMA. 1990;264:2768-2774. [Abstract]
-
O'Connor GT, Plume SK, Olmstead EM, Coffin LH, Morton JR,
Maloney CT, Nowicki ER, Levy DG, Tryzelaar JF, Harnandez F, et al,
for the Northern New England Cardiovascular Study Group. Multivariate
prediction of in-hospital mortality associated with coronary artery
bypass graft surgery. Circulation. 1992;85:
2110-2118.
-
Tu JV, Jaglal SB, Naylor CD, Steering Committee of the
Provincial Adult Cardiac Care Network of Ontario. Multicenter
validation of a risk index for mortality, ICU stay, and overall
hospital length of stay following cardiac surgery.
Circulation. 1995;91:677-684.
[Abstract/Free Full Text]
-
Donabedian A. Quality assessment and assurance: unity
of purpose, diversity of means. Inquiry. 1988;25:173-192. [Medline]
[Order article via Infotrieve]
-
Berwick DM. Continuous improvement as an ideal in health care.
N Engl J Med. 1989;320:53-56. [Medline]
[Order article via Infotrieve]
-
Hammermeister KE. Participatory continuous improvement.
Ann Thorac Surg. In press.
This article has been cited by other articles:
 |
|
 |
|
  I. K. Toumpoulis, C. E. Anagnostopoulos, J. J. DeRose, and D. G. Swistel European system for cardiac operative risk evaluation predicts long-term survival in patients with coronary artery bypass grafting Eur. J. Cardiothorac. Surg., January 1, 2004; 25(1): 51 - 58. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S Al-Ruzzeh, G Asimakopoulos, G Ambler, R Omar, R Hasan, B Fabri, A El-Gamel, A DeSouza, V Zamvar, S Griffin, et al. Validation of four different risk stratification systems in patients undergoing off-pump coronary artery bypass surgery: a UK multicentre analysis of 2223 patients Heart, April 1, 2003; 89(4): 432 - 435. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. M. Shahian, S.-L. Normand, D. F. Torchiana, S. M. Lewis, J. O. Pastore, R. E. Kuntz, and P. I. Dreyer Cardiac surgery report cards: comprehensive review and statistical critique Ann. Thorac. Surg., December 1, 2001; 72(6): 2155 - 2168. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C Sherlaw-Johnson, J Lovegrove, T Treasure, and S Gallivan Likely variations in perioperative mortality associated with cardiac surgery: when does high mortality reflect bad practice? Heart, July 1, 2000; 84(1): 79 - 82. [Abstract] [Full Text] [PDF]
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||