(Circulation. 1996;93:27-33.)
© 1996 American Heart Association, Inc.
Articles |
From the Division of Cardiology, Department of Medicine, University of South Carolina, Columbia, SC (N.A.O.), and the Departments of Cardiology (K.A., E.J.T.), Biostatistics (D.P.M.), Cardiothoracic Nursing (K.J.B.), and Thoracic and Cardiovascular Surgery (D.C., B.L., F.L.), Cleveland Clinic Foundation, Cleveland, Ohio.
Correspondence to Eric J. Topol, MD, Department of Cardiology, Desk F25, 9500 Euclid Ave, Cleveland, OH 44195.
| Abstract |
|---|
|
|
|---|
Methods and Results We reviewed 9442 isolated coronary bypass operations performed from 1989 through 1993 to assess referral patterns of case-mix and outcome. Expected and risk-adjusted mortality rates were computed using logistic regression models derived from the Cleveland Clinic and New York State databases. A mortality comparison was performed using the 1980 to 1988 time period as a historical control. Patients from New York (n=482) had a higher frequency of prior open heart surgery (44.0%) than patients from Ohio (n=6046) (21.5%, P<.001), other states (n=1923) (37.4%, P=.008), and other countries (n=991) (17.3%, P<.001). They were also more likely to be in NYHA functional class III or IV (47.6% versus Ohio 42.7%, P=.037; other states, 41.2%, P=.011; other countries, 34.1%, P=.001). The expected mortality rate was thus higher than among other referral cohorts. The observed 5.2% mortality rate among these patients was significantly greater than the 2.9%, 3.1%, and 1.4% mortality rates observed for patients from Ohio (P=.004), other states (P=.028), and other countries (P<.001). These differences in outcome were not apparent between 1980 and 1988 among referrals from within the United States.
Conclusions Public dissemination of outcome data may have been associated with increased referral of high-risk patients from New York to an out-of-state regional medical center.
Key Words: mortality bypass coronary disease surgery revascularization
| Introduction |
|---|
|
|
|---|
New York State data reporting and analysis are based on participation of 31 institutions and all active surgeons, under the guidance of an independent Cardiac Advisory Committee. Statistical models developed from the registry have been used to predict outcomes on the basis of preoperative risk factors.2
It has been suggested that this quality improvement program played a significant role in the 41% decline in risk-adjusted mortality of isolated CABG (defined as CABG surgery with no other major heart surgery during the same admission) in New York between 1989 and 1992.2 Indeed, physician and hospital profiling programs based on the prospective collation and dissemination of outcome data have been initiated or are being planned for cardiac and other conditions and procedures within and outside New York State.4
We hypothesized that some high-risk patients had migrated out of state for surgery. The purpose of this study was to determine whether cross-border risk-shifting resulted in changes in referral source case-mix and outcome from 1989 through 1993 at the Cleveland Clinic, a major regional, national, and international referral center located in the city of Cleveland, Ohio, 110 miles from the western border of New York state.
| Methods |
|---|
|
|
|---|
Patients were classified according to state or country of residence. Four primary referral source subsets were identified for analytic purposes: Ohio, New York, other out-of-state patients, and out-of-country patients. To control for proximity, additional comparisons were performed using subgroups of patients from border states surrounding Ohio (Pennsylvania, West Virginia, Kentucky, Michigan, and Indiana).
Baseline characteristics were compared using Pearson's
2 statistic.5 Stepwise and manual
building techniques were used to arrive at a 5-year logistic regression
equation for predicting hospital mortality, focusing on strongly
predictive baseline variables (Table 1
).
Major interactions between preoperative risk factors and nonlinearity
in the relationship between age and log-odds of death were
evaluated. The final model was tested for goodness-of-fit by
comparing predicted and observed mortality graphically and using the
Hosmer-Lemeshow test.6 It appeared accurate in predicting
outcome for the full range of data.
|
As an index of illness severity, expected mortality rates for different
referral cohorts were compared using predictions based on logistic
regression models derived from both the Cleveland Clinic and New York
State cardiac surgery databases (Table 2
). This
enabled a comparison of expected, observed, and risk-adjusted
mortality rates for patients from New York who underwent CABG at the
Cleveland Clinic with those treated in New York (see Appendix for
definitions). It also helped determine whether significant differences
in expected mortality would be apparent as a function of the specific
model used.
|
A Cleveland Clinic databasederived logistic regression model of
hospital death with residence as the only risk factor was fit (Table
3
). It was then combined with the previous
risk-factor model to determine whether observed differences in risk
among referral cohorts could be totally explained by the aforementioned
baseline characteristics (Table 4
). Outcomes were then compared
using
Wald's
2 statistic.6 Finally,
referral volumes and hospital mortality for patients from New York and
other sources during the 1989 through 1993 period were compared using
the 1980 through 1988 period as a historical control.
|
|
| Results |
|---|
|
|
|---|
|
Increasing age was a major risk factor for death. In descending order
of magnitude, other characteristics identified in the
multivariable logistic regression model derived from the
Cleveland Clinic database were emergency surgery, prior open heart
surgery, history of congestive heart failure, left
ventricular hypertrophy and femoropopliteal
disease (Table 1
). Patients from New York had a higher expected
mortality rate than other referral cohorts (Table 6
).
|
Predicted mortality rates differed between models. Using regression
coefficients from the 4-year New York State model, all patient groups
appeared more severely ill than otherwise predicted by our model.
Consistently, however, patients from New York had a higher
expected mortality than all other referral cohorts. On average, they
were also at higher risk than the New York State-wide mix (Fig
1
). Observed and risk-adjusted mortality rates
do, however, appear to be declining over time (Fig 2
).
|
|
Relative to patients from Ohio, patients from New York had an odds
ratio for death of 1.7 (95% confidence interval [CI], 1.1 to 2.7)
beyond the risk of being from out of state. In contrast, patients from
foreign countries had an odds ratio of 0.5 (95% CI, 0.3 to 0.8) (Table
3
). When the residence and risk factor models were combined,
better
outcomes among out-of-country patients was almost fully
explained by their lower-risk profile. On the other hand, the
residual independent odds ratio for death of 1.5 (95% CI, 0.9 to 2.4)
suggests that the increased hospital mortality rate among patients from
New York was not purely a function of the major independent risk
factors in our institutional model. However, mortality rates predicted
by the New York state model for the same patients were closer to
observed rates. The incidence of major morbidity among patients from
New York was also significantly higher than among other referral groups
(Table 7
).
|
It has previously been noted by several authors that changes in
surgical case mix with time might have resulted in a paradoxical
increase in CABG mortality despite improved surgical techniques. To
determine whether this trend was equally true among our patient
referral cohorts, a historically controlled substudy was performed. In
comparison with the 1980 through 1988 period, the observed increase in
overall CABG mortality rates from 1989 through 1993 at the Cleveland
Clinic was far more striking among New Yorkers than among patients in
all other groups (Fig 3
).
|
In addition, average yearly volume of New York referrals increased from 61.4 to 96.2, whereas referrals from Ohio and other states decreased from 1285.3 to 1208.4 and 882.1 to 388.2, respectively. As a proportion of total volume, these differences achieved statistical significance (P<.001). On the other hand, referrals from other countries increased from 163.8 to 199.0.
To control for proximity, analyses were repeated after defining a subset of patients (n=1490) from surrounding states directly bordering on Ohio (Pennsylvania, West Virginia, Kentucky, Michigan, and Indiana). The results were consistent. Mortality among patients from New York (5.1%) exceeded that among patients from these states (2.7%). Although not the primary focus of our analysis, we also observed that quantitatively greater mortality rates for combined CABG and valve operations revealed a persistent excess risk among patients referred from New York.
| Discussion |
|---|
|
|
|---|
Possible explanations for these findings include improved quality of care due to the new report card system and a national secular trend. However, since surgeons and hospitals are not only aware that they are under surveillance but that they are also responsible for primary data collection, artifactual increases in patient severity scores could result from the selective emphasis of clinical characteristics.7 8 Another potential source of error is the inclusion of missing data in the final prediction model. Distinguishing the absence of adverse risk factors from truly "missing" data could be important. Indeed, unexplained changes in prevalence of CSRS risk factors have been documented8 and might have resulted in a dilution of the operating pool with higher volumes of inapparently low-risk patients while simultaneously obtaining high-risk credit from the prediction model. It is also possible, though, that underdocumentation of risk factors was more prevalent before implementation of the CSRS. Physicians may have become more careful in categorizing patients with respect to their risk and comorbidities.
With statewide public disclosure and accountability, lack of confidence in statistical risk prediction and adjustment could be an incentive on the part of surgeons to refuse to operate on individual patients clinically perceived to be at very high risk. Conversely, these patients or their primary care physicians may elect to seek care elsewhere. It is unclear whether refusal to operate reduces hospital- and physician-specific risk-adjusted mortality rates.9 The migratory impact of these scenarios was what we sought to evaluate in the current study.
Patients referred from New York State for CABG since 1989 were at higher risk and experienced higher morbidity and mortality than other patients operated on at the Cleveland Clinic, beyond what was expected as a time-related function of increasingly adverse patient characteristics.10 11 This finding, coming during the era of public release of hospital- and surgeon-specific outcome data by the New York State Health Department, suggests an additional component of the temporal change in case mix occasioned by the migration of high-risk patients to regional medical centers in other states.
There were other noteworthy observations from our study. Although it
identified patients from New York as a higher-risk group, our
predictive model did not fully explain their risk even after adjusting
for six major independent risk factors (Table 4
).
Reasons for this include the possibility that "residence" or
"state of referral" represents an interaction term that
captures hidden relationships among less overt risk factors.
Furthermore, although the overall sample size of our data set was more
than 9000, death was a relatively rare occurrence (2.9%), making it
difficult for the model to capture all its predictors and to detect
minor differences among patient groups. This problem was further
highlighted when we attempted to make risk predictions on a
year-to-year basis, creating unstable estimates by further
reducing the sample size.
In an exploratory analysis, we used coefficients from the
high-volume (n=57 187) comprehensive, multi-institutional 4-year
New York model to evaluate our patients because it provided an
opportunity to make direct comparisons with the New York State CABG
population. As previously noted, expected death rates for all Cleveland
Clinic patients based on the New York model were higher than our own
model predicted (Table 6
). This may reflect differences between
the
training sample and test sample. Even then, predictions were not
uniformly reliable, as results for 1989 illustrate. Indeed, the New
York State CSRS underwent changes in data collection instruments and
definitions of variables in January 1991, introducing potentially
significant asymmetry in the multiyear model.8 Thus, the
use of 1-year models might have yielded different results.
Furthermore, the Cleveland Clinic Cardiovascular
Information Registry was not prospectively designed for all the nuances
of this analysis. Risk factor variables, for example, even
when similarly named, cannot be assumed to be qualitatively and
quantitatively identical to those in other databases. One variable
in the New York model (dialysis dependence) was not specifically
recorded at the Cleveland Clinic. Thus, expected mortality derived
using the New York model (excluding this variable) actually
underestimated the expected mortality of our patients. Even then, there
was a trend toward better outcome for patients from New York treated at
the Cleveland Clinic than was predicted by the New York model,
suggesting that the migratory process may have been beneficial to
patients (Figs 2
and 4
).
|
Study Implications
Our study is the first attempt to quantify
the phenomenon of
outmigration of high-risk patients across state borders. If further
studies confirm and extend our findings, certain implications may
assume public-policy significance. The scorecard approach in New
York could be seen as potentiating a regional medical center algorithm
for healthcare delivery.7 12 That high-risk patients
would of their own accord (or prompted by their physicians) seek care
within or across state borders at tertiary referral centers is not in
itself undesirable. If their outcomes are enhanced by the
redistributive process, as our study suggests, then public health is
served.
In-state or out-of-state risk-shifting appears to be a significant by-product of report-card medicine. However, while referred patients may derive a survival benefit (assuming that distance and delay do not pose a hazard), receiving surgeons and institutions might experience an increase in their morbidity and mortality rates in a way that is not adjusted for institutionally derived or other smaller local models of risk adjustment. Because such systems are currently not in place in most states, receiving surgeons and hospitals do not have the incentive to turn down severely ill patients in order to improve their performance scores. If the entire country were to be subject to scorecards, however, particularly high-risk patients might be refused at all centers or be faced with the choice of going abroad for care. Thus, there exists a potential for unintended consequences of policy. Such a scenario could be prevented by protecting designated regional centers and surgeons with reward incentives in a consistent nationally derived database so that they would not be inclined to refuse to accept very-high-risk patients.7 8 Furthermore, because costs of care could potentially be influenced by migration of a critical number of high-risk patients, reimbursement may need to be regionally adjusted accordingly.
There are other ramifications. Rochester, in Genesee County, New York, for example, is considered to be a regional model for national Medicare cost containment.13 In this healthcare delivery model, patients are not, in general, permitted to leave the area for health care. Unable to seek funded optimal care elsewhere, high-risk patients could potentially be entrapped by the local insurance environment on the one hand and the effects of state regulation on the other. Statewide cardiac surgery reporting systems should include information about patients considered for and turned down for CABG surgery in order to keep track of this potential problem. In facilitating "informed referral" and improved care for some patients, it is vital to avoid making it more difficult for the severely ill to obtain surgical care.
The sensitivity of risk prediction to the logistic model used exemplifies one of the many challenges of risk adjustment as a contemporary tool for healthcare policy.14 15 16 Indeed, since the release in 1986 by the Health Care Financing Administration (HCFA) of unadjusted mortality rates for CABG surgery, interest in operative risk assessment as a tool for comparing outcomes has blossomed.17 18 19 The Society for Thoracic Surgeons established the National Database for Thoracic Surgery and has reported a bayesian model for predicting operative mortality from the records of over 80 000 patients.20 21 In a small, prospective validation study of 328 patients at the Texas Heart Institute, predicted mortality agreed with observed mortality only for larger samples of lower-risk groups.22 Estimates for higher-risk groups with smaller sample size were not surprisingly unstablean observation that is probably reflective of the role of chance in both statistical and clinical settings.
On the other hand, the state of Pennsylvania uses claims data and Medisgroups severity to adjust risk for surgeon-specific mortality.23 The validity and reliability of this approach has not been confirmed. In a recent comparison of the New York, Pennsylvania, and HCFA approaches, Green and Wintfeld24 concluded that while the New York State CSRS had the best design and Pennsylvania had the strongest risk-adjustment model, all three needed substantial improvement in data quality control to assure fair comparisons among providers. To maintain the right balance, a quality improvement system has to account for random variation and provide a valid, nonarbitrary, and agreeable method for assessing severity of illness and quality of care while nurturing providers, to allow comparison of process and outcome with peers.14 25 26 Given the aforementioned problems, however, current statistical models are probably better used to adjust mortality rates at an institutional, regional, or national level rather than to predict outcomes accurately for individual high-risk patients investigating alternatives.
Our study illustrates one of the multiple dimensions of the CSRS. Patients can be relocated from one region of the state or country to another as a function of high risk. It is tempting to assume from the foregoing that the apparent improvement in published statistics in the referring institutions and state of origin of these patients reflects a complex selection bias due in part to "outmigration." However, while a small decrease in the overall risk of patients from New York could be ascribed, it is important to emphasize that statewide raw and risk-adjusted mortality rates did indeed decline during the period. In the absence of a randomized trial, alternative explanations must, therefore, continue to be entertained even as the system is improved. Because migration to the Cleveland Clinic in particular appears largely to have been from the western region of New York, we acknowledge that complete assessment may have to await similar analyses from referral institutions in other states.
Conclusion
The recent increase in risk profile of patients
referred for CABG
to the Cleveland Clinic was particularly striking among referrals from
New York State since 1989. Expected and observed mortality among these
patients exceeded that for all other referral cohorts, as well as for
patients who underwent surgery in New York state, and were documented
by the state reporting system (CSRS). However, observed mortality
appears to have been less than predicted, and risk-adjusted
mortality rates continue to decline over time. These observations
highlight the potential role of regional medical centers. Bearing in
mind the limitations of historical controls, our data suggest, for the
first time, a trend toward outmigration of some high-risk patients
to other states during the era of public dissemination of CABG outcome
data.
| Acknowledgments |
|---|
| Appendix |
|---|
|
|
|---|
Expected mortality rate: The sum of the predictive probabilities of death for all patients divided by the total number of patients.
Risk-adjusted mortality rate: The best estimate, based on the statistical model, of what the provider's mortality rate would have been if the provider had a mix of patients identical to the statewide mix.
Received July 17, 1995; revision received September 11, 1995; accepted October 4, 1995.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
C. S. Hollenbeak, C. P. Gorton, Y. P. Tabak, J. L. Jones, A. Milstein, and R. S. Johannes Reductions in Mortality Associated With Intensive Public Reporting of Hospital Outcomes American Journal of Medical Quality, July 1, 2008; 23(4): 279 - 286. [Abstract] [PDF] |
||||
![]() |
B Bridgewater and B Keogh Surgical "league tables": Ischaemic heart disease Heart, July 1, 2008; 94(7): 936 - 942. [Full Text] [PDF] |
||||
![]() |
D. L. Segev, C. E. Simpkins, R. E. Thompson, J. E. Locke, D. S. Warren, and R. A. Montgomery Obesity Impacts Access to Kidney Transplantation J. Am. Soc. Nephrol., February 1, 2008; 19(2): 349 - 355. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. H. Fung, Y.-W. Lim, S. Mattke, C. Damberg, and P. G. Shekelle Systematic Review: The Evidence That Publishing Patient Care Performance Data Improves Quality of Care Ann Intern Med, January 15, 2008; 148(2): 111 - 123. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Hamblin Publishing 'quality' measures: how it works and when it does not? Int. J. Qual. Health Care, August 1, 2007; 19(4): 183 - 186. [Full Text] [PDF] |
||||
![]() |
B. Bridgewater, A. D Grayson, N. Brooks, G. Grotte, B. M Fabri, J. Au, T. Hooper, M. Jones, B. Keogh, and on behalf of the North West Quality Improvement Pr Has the publication of cardiac surgery outcome data been associated with changes in practice in northwest England: an analysis of 25 730 patients undergoing CABG surgery under 30 surgeons over eight years Heart, June 1, 2007; 93(6): 744 - 748. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. L. Higgins Quantifying Risk and Benchmarking Performance in the Adult Intensive Care Unit J Intensive Care Med, May 1, 2007; 22(3): 141 - 156. [Abstract] [PDF] |
||||
![]() |
L. P. Casalino, A. Elster, A. Eisenberg, E. Lewis, J. Montgomery, and D. Ramos Will Pay-For-Performance And Quality Reporting Affect Health Care Disparities? Health Aff., May 1, 2007; 26(3): w405 - w414. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. J. Epstein Do cardiac surgery report cards reduce mortality? Assessing the evidence. Med Care Res Rev, August 1, 2006; 63(4): 403 - 426. [Abstract] [PDF] |
||||
![]() |
M Moscucci and D Share Impact of health care initiatives on outcomes of acute coronary syndromes. Heart, March 1, 2006; 92(3): 293 - 295. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. J.O. Birkmeyer and J. D. Birkmeyer Strategies for improving surgical quality--should payers reward excellence or effort? N. Engl. J. Med., February 23, 2006; 354(8): 864 - 870. [Full Text] [PDF] |
||||
![]() |
J. L. Wolff and C. Boult Moving beyond Round Pegs and Square Holes: Restructuring Medicare To Improve Chronic Care Ann Intern Med, September 20, 2005; 143(6): 439 - 445. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. M. Werner and D. A. Asch Publicly Reporting Quality Information--Reply JAMA, June 22, 2005; 293(24): 3000 - 3001. [Full Text] [PDF] |
||||
![]() |
E. L. Hannan and M. R. Chassin Publicly Reporting Quality Information JAMA, June 22, 2005; 293(24): 2999 - 3000. [Full Text] [PDF] |
||||
![]() |
M. Moscucci, K. A. Eagle, D. Share, D. Smith, A. C. De Franco, M. O'Donnell, E. Kline-Rogers, S. M. Jani, and D. L. Brown Public Reporting and Case Selection for Percutaneous Coronary Interventions: An Analysis From Two Large Multicenter Percutaneous Coronary Intervention Databases J. Am. Coll. Cardiol., June 7, 2005; 45(11): 1759 - 1765. [Abstract] [Full Text] [PDF] |
||||
![]() |
Z. G. Turi The Big Chill: The Deleterious Effects of Public Reporting on Access to Health Care for the Sickest Patients J. Am. Coll. Cardiol., June 7, 2005; 45(11): 1766 - 1768. [Full Text] [PDF] |
||||
![]() |
R. M. Werner, D. A. Asch, and D. Polsky Racial Profiling: The Unintended Consequences of Coronary Artery Bypass Graft Report Cards Circulation, March 15, 2005; 111(10): 1257 - 1263. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. M. Werner and D. A. Asch The Unintended Consequences of Publicly Reporting Quality Information JAMA, March 9, 2005; 293(10): 1239 - 1244. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. R. Narins, A. M. Dozier, F. S. Ling, and W. Zareba The Influence of Public Reporting of Outcome Data on Medical Decision Making by Physicians Arch Intern Med, January 10, 2005; 165(1): 83 - 87. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. L. Hannan, C. Wu, T. J. Ryan, E. Bennett, A. T. Culliford, J. P. Gold, A. Hartman, O. W. Isom, R. H. Jones, B. McNeil, et al. Do Hospitals and Surgeons With Higher Coronary Artery Bypass Graft Surgery Volumes Still Have Lower Risk-Adjusted Mortality Rates? Circulation, August 19, 2003; 108(7): 795 - 801. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Gupta, P. L. Kotler, and R. A. Dudley Considerations in the Development of Intensive Care Unit Report Cards J Intensive Care Med, September 1, 2002; 17(5): 211 - 217. [Abstract] [PDF] |
||||
![]() |
M. R. Chassin Achieving And Sustaining Improved Quality: Lessons From New York State And Cardiac Surgery Health Aff., July 1, 2002; 21(4): 40 - 51. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Pace Surgeons should beware of plans to print individual performance BMJ, April 20, 2002; 324(7343): 979 - 979. [Full Text] |
||||
![]() |
C. D. Naylor Public Profiling of Clinical Performance JAMA, March 13, 2002; 287(10): 1323 - 1325. [Full Text] [PDF] |
||||
![]() |
P. Adab, A. M Rouse, M. A Mohammed, and T. Marshall Performance league tables: the NHS deserves better BMJ, January 12, 2002; 324(7329): 95 - 98. [Full Text] [PDF] |
||||
![]() |
D. M. Shahian, S.-L. Normand, D. F. Torchiana, S. M. Lewis, J. O. Pastore, R. E. Kuntz, and P. I. Dreyer Cardiac surgery report cards: comprehensive review and statistical critique Ann. Thorac. Surg., December 1, 2001; 72(6): 2155 - 2168. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. T. Vaitkus Physician profiling in the catheterization laboratory: a worthwhile strategy or a path to futility? J. Am. Coll. Cardiol., November 1, 2001; 38(5): 1424 - 1426. [Full Text] [PDF] |
||||
![]() |