Outmigration For Coronary Bypass Surgery in an Era of Public Dissemination of Clinical Outcomes
Background Since 1989, New York State has disseminated comparative information on outcomes of coronary bypass surgery to the public. It has been suggested that this program played a significant role in the 41% decrease in the risk-adjusted mortality rate between 1989 and 1992. We hypothesized that some high-risk patients had migrated out of state for surgery.
Methods and Results We reviewed 9442 isolated coronary bypass operations performed from 1989 through 1993 to assess referral patterns of case-mix and outcome. Expected and risk-adjusted mortality rates were computed using logistic regression models derived from the Cleveland Clinic and New York State databases. A mortality comparison was performed using the 1980 to 1988 time period as a historical control. Patients from New York (n=482) had a higher frequency of prior open heart surgery (44.0%) than patients from Ohio (n=6046) (21.5%, P<.001), other states (n=1923) (37.4%, P=.008), and other countries (n=991) (17.3%, P<.001). They were also more likely to be in NYHA functional class III or IV (47.6% versus Ohio 42.7%, P=.037; other states, 41.2%, P=.011; other countries, 34.1%, P=.001). The expected mortality rate was thus higher than among other referral cohorts. The observed 5.2% mortality rate among these patients was significantly greater than the 2.9%, 3.1%, and 1.4% mortality rates observed for patients from Ohio (P=.004), other states (P=.028), and other countries (P<.001). These differences in outcome were not apparent between 1980 and 1988 among referrals from within the United States.
Conclusions Public dissemination of outcome data may have been associated with increased referral of high-risk patients from New York to an out-of-state regional medical center.
Since 1989, the New York State Department of Health has implemented a comprehensive reporting system for coronary artery bypass surgery (CABG).1 2 Although hospital-specific case volume and crude and risk-adjusted mortality rates have been available since then, surgeon-specific data became public in 1991 after a lawsuit filed by Newsday under the Freedom of Information Act.3
New York State data reporting and analysis are based on participation of 31 institutions and all active surgeons, under the guidance of an independent Cardiac Advisory Committee. Statistical models developed from the registry have been used to predict outcomes on the basis of preoperative risk factors.2
It has been suggested that this quality improvement program played a significant role in the 41% decline in risk-adjusted mortality of isolated CABG (defined as CABG surgery with no other major heart surgery during the same admission) in New York between 1989 and 1992.2 Indeed, physician and hospital profiling programs based on the prospective collation and dissemination of outcome data have been initiated or are being planned for cardiac and other conditions and procedures within and outside New York State.4
We hypothesized that some high-risk patients had migrated out of state for surgery. The purpose of this study was to determine whether cross-border risk-shifting resulted in changes in referral source case-mix and outcome from 1989 through 1993 at the Cleveland Clinic, a major regional, national, and international referral center located in the city of Cleveland, Ohio, 110 miles from the western border of New York state.
We performed a retrospective analysis of 9442 isolated CABG operations undertaken at the Cleveland Clinic between January 1, 1989, and December 31, 1993. Data were obtained from the Cardiovascular Information Registry−a regularly updated, quality-controlled clinical database, prospectively collected since 1972, of over 70 000 cardiac operations performed at the Cleveland Clinic since its inception. Chart abstractors go through a 3- to 6-month training process during which 100% of all entries are double entered and manually checked for accuracy. Entered data were subjected to range, consistency, and error and validity checks using a customized software program (Relational database package software program, VAX product) and Cleveland Clinic custom program. A 5% random sample of entries was retrospectively audited using source records.
Patients were classified according to state or country of residence. Four primary referral source subsets were identified for analytic purposes: Ohio, New York, other out-of-state patients, and out-of-country patients. To control for proximity, additional comparisons were performed using subgroups of patients from border states surrounding Ohio (Pennsylvania, West Virginia, Kentucky, Michigan, and Indiana).
Baseline characteristics were compared using Pearson’s χ2 statistic.5 Stepwise and manual building techniques were used to arrive at a 5-year logistic regression equation for predicting hospital mortality, focusing on strongly predictive baseline variables (Table 1⇓). Major interactions between preoperative risk factors and nonlinearity in the relationship between age and log-odds of death were evaluated. The final model was tested for goodness-of-fit by comparing predicted and observed mortality graphically and using the Hosmer-Lemeshow test.6 It appeared accurate in predicting outcome for the full range of data.
As an index of illness severity, expected mortality rates for different referral cohorts were compared using predictions based on logistic regression models derived from both the Cleveland Clinic and New York State cardiac surgery databases (Table 2⇓). This enabled a comparison of expected, observed, and risk-adjusted mortality rates for patients from New York who underwent CABG at the Cleveland Clinic with those treated in New York (see Appendix for definitions). It also helped determine whether significant differences in expected mortality would be apparent as a function of the specific model used.
A Cleveland Clinic database–derived logistic regression model of hospital death with residence as the only risk factor was fit (Table 3⇓). It was then combined with the previous risk-factor model to determine whether observed differences in risk among referral cohorts could be totally explained by the aforementioned baseline characteristics (Table 4⇓). Outcomes were then compared using Wald’s χ2 statistic.6 Finally, referral volumes and hospital mortality for patients from New York and other sources during the 1989 through 1993 period were compared using the 1980 through 1988 period as a historical control.
Of the 9442 isolated CABG operations under consideration, 482 (5.1%) were performed on patients from New York, 6046 (64.0%) on patients from Ohio, 1923 (20.4%) on patients from all other states, and 991 (10.5%) on patients from foreign countries. In comparison with all other referral cohorts, patients from New York had a significantly higher frequency of prior open heart surgery and New York Heart Association (NYHA) functional class III or IV (Table 5⇓). There was also a trend toward older age, higher likelihood of carotid and femoropopliteal diseases, presence of left ventricular hypertrophy, history of congestive heart failure, and history of smoking.
Increasing age was a major risk factor for death. In descending order of magnitude, other characteristics identified in the multivariable logistic regression model derived from the Cleveland Clinic database were emergency surgery, prior open heart surgery, history of congestive heart failure, left ventricular hypertrophy and femoropopliteal disease (Table 1⇑). Patients from New York had a higher expected mortality rate than other referral cohorts (Table 6⇓).
Predicted mortality rates differed between models. Using regression coefficients from the 4-year New York State model, all patient groups appeared more severely ill than otherwise predicted by our model. Consistently, however, patients from New York had a higher expected mortality than all other referral cohorts. On average, they were also at higher risk than the New York State-wide mix (Fig 1⇓). Observed and risk-adjusted mortality rates do, however, appear to be declining over time (Fig 2⇓).
Relative to patients from Ohio, patients from New York had an odds ratio for death of 1.7 (95% confidence interval [CI], 1.1 to 2.7) beyond the risk of being from out of state. In contrast, patients from foreign countries had an odds ratio of 0.5 (95% CI, 0.3 to 0.8) (Table 3⇑). When the residence and risk factor models were combined, better outcomes among out-of-country patients was almost fully explained by their lower-risk profile. On the other hand, the residual independent odds ratio for death of 1.5 (95% CI, 0.9 to 2.4) suggests that the increased hospital mortality rate among patients from New York was not purely a function of the major independent risk factors in our institutional model. However, mortality rates predicted by the New York state model for the same patients were closer to observed rates. The incidence of major morbidity among patients from New York was also significantly higher than among other referral groups (Table 7⇓).
It has previously been noted by several authors that changes in surgical case mix with time might have resulted in a paradoxical increase in CABG mortality despite improved surgical techniques. To determine whether this trend was equally true among our patient referral cohorts, a historically controlled substudy was performed. In comparison with the 1980 through 1988 period, the observed increase in overall CABG mortality rates from 1989 through 1993 at the Cleveland Clinic was far more striking among New Yorkers than among patients in all other groups (Fig 3⇓).
In addition, average yearly volume of New York referrals increased from 61.4 to 96.2, whereas referrals from Ohio and other states decreased from 1285.3 to 1208.4 and 882.1 to 388.2, respectively. As a proportion of total volume, these differences achieved statistical significance (P<.001). On the other hand, referrals from other countries increased from 163.8 to 199.0.
To control for proximity, analyses were repeated after defining a subset of patients (n=1490) from surrounding states directly bordering on Ohio (Pennsylvania, West Virginia, Kentucky, Michigan, and Indiana). The results were consistent. Mortality among patients from New York (5.1%) exceeded that among patients from these states (2.7%). Although not the primary focus of our analysis, we also observed that quantitatively greater mortality rates for combined CABG and valve operations revealed a persistent excess risk among patients referred from New York.
The Cardiac Surgery Reporting System (CSRS) has been in place in New York State since 1989. From 1989 through 1992, yearly CABG volume increased by 31%, average severity of illness (expected mortality) increased from 2.62% to 3.54%, and in-hospital mortality decreased by 21%, resulting in a 41% decline in risk-adjusted mortality (from 4.17% to 2.45%).2
Possible explanations for these findings include improved quality of care due to the new report card system and a national secular trend. However, since surgeons and hospitals are not only aware that they are under surveillance but that they are also responsible for primary data collection, artifactual increases in patient severity scores could result from the selective emphasis of clinical characteristics.7 8 Another potential source of error is the inclusion of missing data in the final prediction model. Distinguishing the absence of adverse risk factors from truly “missing” data could be important. Indeed, unexplained changes in prevalence of CSRS risk factors have been documented8 and might have resulted in a dilution of the operating pool with higher volumes of inapparently low-risk patients while simultaneously obtaining high-risk credit from the prediction model. It is also possible, though, that underdocumentation of risk factors was more prevalent before implementation of the CSRS. Physicians may have become more careful in categorizing patients with respect to their risk and comorbidities.
With statewide public disclosure and accountability, lack of confidence in statistical risk prediction and adjustment could be an incentive on the part of surgeons to refuse to operate on individual patients clinically perceived to be at very high risk. Conversely, these patients or their primary care physicians may elect to seek care elsewhere. It is unclear whether refusal to operate reduces hospital- and physician-specific risk-adjusted mortality rates.9 The migratory impact of these scenarios was what we sought to evaluate in the current study.
Patients referred from New York State for CABG since 1989 were at higher risk and experienced higher morbidity and mortality than other patients operated on at the Cleveland Clinic, beyond what was expected as a time-related function of increasingly adverse patient characteristics.10 11 This finding, coming during the era of public release of hospital- and surgeon-specific outcome data by the New York State Health Department, suggests an additional component of the temporal change in case mix occasioned by the migration of high-risk patients to regional medical centers in other states.
There were other noteworthy observations from our study. Although it identified patients from New York as a higher-risk group, our predictive model did not fully explain their risk even after adjusting for six major independent risk factors (Table 4⇑). Reasons for this include the possibility that “residence” or “state of referral” represents an interaction term that captures hidden relationships among less overt risk factors. Furthermore, although the overall sample size of our data set was more than 9000, death was a relatively rare occurrence (2.9%), making it difficult for the model to capture all its predictors and to detect minor differences among patient groups. This problem was further highlighted when we attempted to make risk predictions on a year-to-year basis, creating unstable estimates by further reducing the sample size.
In an exploratory analysis, we used coefficients from the high-volume (n=57 187) comprehensive, multi-institutional 4-year New York model to evaluate our patients because it provided an opportunity to make direct comparisons with the New York State CABG population. As previously noted, expected death rates for all Cleveland Clinic patients based on the New York model were higher than our own model predicted (Table 6⇑). This may reflect differences between the training sample and test sample. Even then, predictions were not uniformly reliable, as results for 1989 illustrate. Indeed, the New York State CSRS underwent changes in data collection instruments and definitions of variables in January 1991, introducing potentially significant asymmetry in the multiyear model.8 Thus, the use of 1-year models might have yielded different results.
Furthermore, the Cleveland Clinic Cardiovascular Information Registry was not prospectively designed for all the nuances of this analysis. Risk factor variables, for example, even when similarly named, cannot be assumed to be qualitatively and quantitatively identical to those in other databases. One variable in the New York model (dialysis dependence) was not specifically recorded at the Cleveland Clinic. Thus, expected mortality derived using the New York model (excluding this variable) actually underestimated the expected mortality of our patients. Even then, there was a trend toward better outcome for patients from New York treated at the Cleveland Clinic than was predicted by the New York model, suggesting that the migratory process may have been beneficial to patients (Figs 2⇑ and 4⇓).
Our study is the first attempt to quantify the phenomenon of outmigration of high-risk patients across state borders. If further studies confirm and extend our findings, certain implications may assume public-policy significance. The scorecard approach in New York could be seen as potentiating a regional medical center algorithm for healthcare delivery.7 12 That high-risk patients would of their own accord (or prompted by their physicians) seek care within or across state borders at tertiary referral centers is not in itself undesirable. If their outcomes are enhanced by the redistributive process, as our study suggests, then public health is served.
In-state or out-of-state risk-shifting appears to be a significant by-product of report-card medicine. However, while referred patients may derive a survival benefit (assuming that distance and delay do not pose a hazard), receiving surgeons and institutions might experience an increase in their morbidity and mortality rates in a way that is not adjusted for institutionally derived or other smaller local models of risk adjustment. Because such systems are currently not in place in most states, receiving surgeons and hospitals do not have the incentive to turn down severely ill patients in order to improve their performance scores. If the entire country were to be subject to scorecards, however, particularly high-risk patients might be refused at all centers or be faced with the choice of going abroad for care. Thus, there exists a potential for unintended consequences of policy. Such a scenario could be prevented by protecting designated regional centers and surgeons with reward incentives in a consistent nationally derived database so that they would not be inclined to refuse to accept very-high-risk patients.7 8 Furthermore, because costs of care could potentially be influenced by migration of a critical number of high-risk patients, reimbursement may need to be regionally adjusted accordingly.
There are other ramifications. Rochester, in Genesee County, New York, for example, is considered to be a regional model for national Medicare cost containment.13 In this healthcare delivery model, patients are not, in general, permitted to leave the area for health care. Unable to seek funded optimal care elsewhere, high-risk patients could potentially be entrapped by the local insurance environment on the one hand and the effects of state regulation on the other. Statewide cardiac surgery reporting systems should include information about patients considered for and turned down for CABG surgery in order to keep track of this potential problem. In facilitating “informed referral” and improved care for some patients, it is vital to avoid making it more difficult for the severely ill to obtain surgical care.
The sensitivity of risk prediction to the logistic model used exemplifies one of the many challenges of risk adjustment as a contemporary tool for healthcare policy.14 15 16 Indeed, since the release in 1986 by the Health Care Financing Administration (HCFA) of unadjusted mortality rates for CABG surgery, interest in operative risk assessment as a tool for comparing outcomes has blossomed.17 18 19 The Society for Thoracic Surgeons established the National Database for Thoracic Surgery and has reported a bayesian model for predicting operative mortality from the records of over 80 000 patients.20 21 In a small, prospective validation study of 328 patients at the Texas Heart Institute, predicted mortality agreed with observed mortality only for larger samples of lower-risk groups.22 Estimates for higher-risk groups with smaller sample size were not surprisingly unstable—an observation that is probably reflective of the role of chance in both statistical and clinical settings.
On the other hand, the state of Pennsylvania uses claims data and Medisgroups severity to adjust risk for surgeon-specific mortality.23 The validity and reliability of this approach has not been confirmed. In a recent comparison of the New York, Pennsylvania, and HCFA approaches, Green and Wintfeld24 concluded that while the New York State CSRS had the best design and Pennsylvania had the strongest risk-adjustment model, all three needed substantial improvement in data quality control to assure fair comparisons among providers. To maintain the right balance, a quality improvement system has to account for random variation and provide a valid, nonarbitrary, and agreeable method for assessing severity of illness and quality of care while nurturing providers, to allow comparison of process and outcome with peers.14 25 26 Given the aforementioned problems, however, current statistical models are probably better used to adjust mortality rates at an institutional, regional, or national level rather than to predict outcomes accurately for individual high-risk patients investigating alternatives.
Our study illustrates one of the multiple dimensions of the CSRS. Patients can be relocated from one region of the state or country to another as a function of high risk. It is tempting to assume from the foregoing that the apparent improvement in published statistics in the referring institutions and state of origin of these patients reflects a complex selection bias due in part to “outmigration.” However, while a small decrease in the overall risk of patients from New York could be ascribed, it is important to emphasize that statewide raw and risk-adjusted mortality rates did indeed decline during the period. In the absence of a randomized trial, alternative explanations must, therefore, continue to be entertained even as the system is improved. Because migration to the Cleveland Clinic in particular appears largely to have been from the western region of New York, we acknowledge that complete assessment may have to await similar analyses from referral institutions in other states.
The recent increase in risk profile of patients referred for CABG to the Cleveland Clinic was particularly striking among referrals from New York State since 1989. Expected and observed mortality among these patients exceeded that for all other referral cohorts, as well as for patients who underwent surgery in New York state, and were documented by the state reporting system (CSRS). However, observed mortality appears to have been less than predicted, and risk-adjusted mortality rates continue to decline over time. These observations highlight the potential role of regional medical centers. Bearing in mind the limitations of historical controls, our data suggest, for the first time, a trend toward outmigration of some high-risk patients to other states during the era of public dissemination of CABG outcome data.
Observed mortality rate: The number of observed deaths divided by the number of patients.
Expected mortality rate: The sum of the predictive probabilities of death for all patients divided by the total number of patients.
Risk-adjusted mortality rate: The best estimate, based on the statistical model, of what the provider’s mortality rate would have been if the provider had a mix of patients identical to the statewide mix.
We wish to acknowledge the contributions of Mark A. Hlatky, MD, Jeffery Lefkovits, MD, and Becky Zuti and the staff of the Cleveland Clinic Cardiovascular Information Registry.
- Received July 17, 1995.
- Revision received September 11, 1995.
- Accepted October 4, 1995.
- Copyright © 1996 by American Heart Association
Coronary Artery Bypass Surgery in New York State 1990-1992. New York State Department of Health, December 1993.
Zinman D. Heart surgeons rated: state reveals patient mortality records. Newsday, Nassau and Suffolk Ed. December 18, 1991:3.
Data diligence: New York tries unprecedented doctor-specific tracking to sharpen medical practice. Physician’s Weekly. November 7, 1994;Vol 11.
Matthews D, Farewell VT. Using and understanding medical statistics. In: Approximate Significant Tests for Contingency Tables. 2nd ed. Basel, Switzerland: Karger; 1988:39-57.
Hosmer DW, Lemeshow S. Applied Logistic Regression. New York, NY: John Wiley & Sons Inc; 1989.
Byer MJ. Faint hearts. New York Times. March 21, 1992:A23.
Lytle BW, Cosgrove DM III. Coronary artery bypass surgery. Curr Probl Surg. 1992;29:746-747.
Freudenheim M. Rochester serves as model in controlling health cost. New York Times (late edition). August 25, 1992:A1.
Orchard C. Comparing health care outcomes. BMJ. 1994;308:1493-1496.
DeDombal FT, Clamp SE, Softley A, Unwin BJ, Staniland JR. Prediction of individual patient prognosis. Med Decis Making. 1986;6:18-22.
Health Care Financing Administration. Medicare Hospital Mortality Information 1986. HCFA Pub. No. 01-002. Washington, DC: US Government Printing Office; 1987.
Bailey RC. Some uses of a modified Makeham model to evaluate medical practice. J Wash Acad Sci. 1988;78:339-353.
The Society of Thoracic Surgeons National Cardiac Surgery Database Manual for Data Managers. Minneapolis, Minn: Summit Medical Systems; 1993.
Alexander WA, Keats AS. Validation of the Society of Thoracic Surgeons (STS) National Cardiac Surgery Database (NCSD) utilizing a single institutional model of CABG outcomes. Circulation. 1994;90(suppl II):I-529. Abstract.
Pennsylvania Health Care Cost Containment Council. A Consumer Guide to Coronary Artery Bypass Surgery. Harrisburg, Pa: Pennsylvania Health Care Cost Containment Council; 1992.
Green J, Wintfeld N. Consumer report cards on coronary artery bypass surgery: comparison of three leading approaches. Circulation. 1994;90(suppl II):I-529. Abstract.
Ellis SG, Omoigui N, Bittl JA, Lincoff AM, Wolfe MW, Howell G, Topol EJ. Analysis and comparison of operator specific outcomes in interventional cardiology: from a multicenter database of 4860 quality-controlled procedures. Circulation. In press.
Smith DW. Evaluating risk adjustment by partitioning variation in hospital mortality rates. Stat Med. 1994;13:1-13.