Improvement in Congenital Heart Surgery in a Developing Country
The Guatemalan Experience
Background— In 1997, an effort was made to disseminate US pediatric cardiac surgical practices to create a new comprehensive program in Guatemala. The objective of this study was to describe the improvement of the program by exploring the reduction in risk-adjusted in-hospital mortality.
Methods and Results— A retrospective cohort study of surgical procedures performed in Guatemala from February 1997 to July 2004 was conducted. Data were divided into 3 time periods (1997 to 1999, 2000 to 2002, and 2003 to 2004) and compared with a US benchmark (2000 Kids’ Inpatient Database of 27 states and 313 institutions). The risk adjustment for congenital heart surgery (RACHS-1) method was used to adjust for case mix. Mortality rates, standardized mortality ratios, and 95% confidence intervals were calculated. A total of 1215 surgical procedures were included. Median age was 3.1 years (range, 1 day to 17.9 years). The overall mortality was 10.7% (n=130). The RACHS-1 method showed better discrimination than in prior reports (area under receiver operating characteristic curve=0.854). A decreasing trend in mortality rate was observed in every RACHS-1 risk category over the 3 time periods. When compared against the US benchmark, the reduction in risk-adjusted mortality was noted by a decrease of standardized mortality ratio from 10.0 (95% confidence interval, 7.2 to 13.7) in 1997–1999, to 7.8 (95% confidence interval, 5.9 to 10.0) in 2000–2002, and to 5.7 (95% confidence interval, 3.8 to 8.3) in 2003–2004.
Conclusions— In a short time period, mortality after congenital heart surgery has been reduced substantially in Guatemala. Measurement of risk-adjusted mortality is a useful method to assess pediatric cardiac program improvement in the developing world.
Received February 12, 2007; accepted June 25, 2007.
A large underserved population of children with congenital heart disease is present in many developing countries. These children have very limited or no access to cardiovascular surgical care and will face irreversible heart and lung damage or death. Consequently, methods to provide and improve cardiovascular surgical care in developing countries are needed. In recent years, several strategies have been implemented to address this need. These strategies include transferring children to first-world countries for surgical care, organizing surgical trips to developing countries, and, more recently, the creation of local cardiovascular surgical programs. The latter strategy is due in part to new policies in humanitarian medicine that promote a shift toward taking care of children in their local environment as well as to the fact that more children can be treated at a fraction of the cost.1–3
Editorials pp 1874 and 1876
Clinical Perspective p 1887
Although implementation of new cardiovascular programs is an essential first step in improving outcomes for those children, an important second step is to evaluate the performance of those programs. Evaluation of performance and improvement are necessary for reasons similar to those in the first world: constant self-evaluation, including benchmarking with similar programs, and creation of care improvement strategies.4 In 1997, an effort was made to disseminate US pediatric cardiac surgical practices to create a new comprehensive pediatric cardiac care program in Guatemala. The objective of this study was to examine the reduction in risk-adjusted in-hospital mortality in the new cardiovascular program and to compare those results with congenital heart surgery outcomes in the United States.
Guatemala is a Latin American country with >12 million inhabitants and with a land mass similar to the state of Pennsylvania. Guatemala’s population is among the most deprived in the continent: According to the United Nations Development Program, in 1998 the infant mortality rate was 43 per 1000 live births compared with 7 per 1000 in the United States. Healthcare access in most areas was <8%, and ≈80% of the population lived below the poverty line, defined in Guatemala as <$2 per day.5 In 1995, the government spent only 1.8% of the gross domestic product on health care, ranking near the bottom of the hemisphere alongside countries like Haiti (1.3%) and the Dominican Republic (1.8%).6,7 Serious geographic and language barriers worsen the problem; although Spanish is the main language, 23 different native languages are also spoken by the indigenous population. On the basis of data derived from previous surgical missions in the country and estimates of children infrequently sent to the United States and other countries, before 1997 probably <3% of Guatemalan children with congenital heart disease had access to cardiac surgical care.
Creation of the Cardiac Program
The goals of the new pediatric cardiac care program, known as Unidad de Cirugia Cardiovascular Pediatrica de Guatemala (UNICARP), were to improve outcomes of children with congenital heart disease in Guatemala by training local staff to eventually become independent. A foundation was established locally and in the United States in 1997 to serve as a fundraising instrument to acquire equipment for the pediatric cardiac unit and also to sponsor surgical procedures for the poor population.
Among the total patients operated on, 95% have been subsidized in total. In this group, 50% were paid by the social security system (in Guatemala, a family qualifies for social security, and thus for full coverage of the costs of surgery for congenital conditions, if the parent of the child is employed by the government or employed by a participant in social security who employs >10 persons, and the employee has been working for a minimum of 3 months at the time of surgery).8 The remaining 45% were paid for by the Ministry of Health and the foundation. Only 5% of the patients paid some proportion of the cost of their care: 70% paid <20%, 29% paid between 20% and 40%, and <1% paid the full cost.
Since the program was initiated, the foundation has successfully urged the government to assign a defined part of the budget to pay for surgeries, salaries, and infrastructure, concentrating its own resources mainly on renovations and equipment. Although this effort has been successful, the foundation’s fundraising and subsidies for certain services are and will always likely be an important aid for the pediatric cardiac unit. In partnership with the Friends of the Aldo Castañeda Foundation in the United States, efforts are in place to create an endowment that will secure a constant economic assistance for the future.
The staff includes 3 surgeons from Guatemala, trained by a senior surgeon (A.R.C.) who had extensive experience operating in the United States, 8 pediatric cardiologists, 2 intensivists, and 3 anesthesiologists, as well as intensive care and ward nurses, respiratory therapists, echocardiography technicians, and support personnel. The cardiovascular program expanded in the last years to 2 cardiac operating rooms, 1 cardiac catheterization laboratory, a 6-bed intensive care unit, a 4-bed stepdown unit, and an 18-bed general ward. More than 1300 surgical procedures have been performed, and in the outpatient clinic >20 000 patients have been seen. The center in Guatemala has become a referral center for children from El Salvador, Honduras, Nicaragua, Belize, Dominican Republic, and Haiti.
Data recorded during the surgical admission were retrieved from medical records by a physician reviewer. A retrospective cohort study was conducted including all the cases that underwent surgical intervention for congenital heart disease from February 1997 to July 2004. Demographic and in-hospital variables were collected (eg, surgical procedure, diagnosis, surgeon, age at surgery, length of stay, complications, and infections).
To compare mortality in Guatemala and the United States, we used the Healthcare Cost and Utilization Project Kids’ Inpatient Database (KID) for calendar year 2000. This database is a nationwide sample of US inpatient pediatric discharges and consists of a stratified random sample of 2 516 833 discharges from 2784 institutions in 27 states, including pediatric hospitals, academic medical centers, and specialty hospitals. The database includes a 10% sample of uncomplicated births from these institutions and an 80% sample of other pediatric discharges for patients aged ≤20 years.9
For the Guatemalan and the KID databases, we selected all patients aged <18 years who underwent surgical repair of a congenital heart defect, using International Classification of Diseases, Ninth Revision, Clinical Modification codes. Cases with codes for cardiac transplantation were excluded, as were premature infants and newborns aged ≤30 days undergoing closure of a patent ductus arteriosus only.10
The risk adjustment for congenital heart surgery (RACHS-1) method was used to adjust for differences in case mix when analyzing in-hospital mortality in Guatemala and when comparing in-hospital mortality for Guatemala and the United States. To apply this method, cases were assigned to 1 of 6 predefined risk categories on the basis of the presence or absence of specific International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis and procedure codes; risk category 1 has the lowest risk for death and risk category 6 the highest. Cases with combinations of cardiac surgical procedures (eg, repair of coarctation of the aorta and ventricular septal defect repair) were placed in the risk category corresponding to the single highest risk procedure. Additional clinical variables incorporated as part of the RACHS-1 adjustment are as follows: age group (≤30 days, 31 days to 1 year, ≥1 year), prematurity, the presence of a major noncardiac structural anomaly, and the presence of combinations of cardiac surgical procedures.4,10,11
Validation of the RACHS-1 Method
The RACHS-1 method has been previously validated and applied to US and European administrative databases.10–12 However, validation for a developing country data set has not been published. To evaluate the performance of RACHS-1 in the Guatemalan database, a logistic regression model with outcome of in-hospital death and predictor variables of risk category, age group, and presence of combinations of procedures was fitted. Prematurity and the presence of noncardiac structural anomalies were not included because these were only present in a small number of cases and deaths. The area under the receiver operating characteristic curve was computed for the model as a measure of the model’s performance. The area under the receiver operating characteristic curve quantifies how well the model is able to predict mortality, with an area of 0.5 indicating a model with no predictive power and an area of 1.0 indicating that the model predicts outcome perfectly every time.
Surgical procedures in Guatemala were then divided into 3 time periods: 1997 to 1999, 2000 to 2002, and 2003 to 2004. Within each time period, mortality rates were calculated overall and by surgical risk category. The proportions of procedures performed by the senior surgeon in each of the 3 time periods were compared with the Fisher exact test.
A logistic regression model with outcome of in-hospital death and predictor variables of risk category plus the additional clinical variables was generated for cases of congenital heart surgery in the benchmark KID 2000 database. This model was then used to calculate the predicted probability of death for each individual case in the Guatemalan cohort. The average predicted probability of death for all patients within a given time period, calculated by summing the predicted probabilities for each patient and dividing by the total number of patients undergoing surgery in that time period, represents the expected mortality rate for the time period, adjusted for case mix. The standardized mortality ratio (SMR) for each period was then calculated as the observed mortality rate divided by the expected mortality rate. The 95% confidence intervals (CIs) were also generated.
The UNICARP hospital internal review board approved this study. A waiver of informed consent was obtained. Patient data were made anonymous in our database in compliance with the hospital requirements.
The authors had full access to and take full responsibility for the integrity of the data. All authors have read and agree to the manuscript as written.
A total of 1328 surgical procedures were performed between February 1997 and July 2004. Among these, 1215 were performed in children aged <18 years and were eligible. Patients excluded from further analysis (n=113) included 54 who were aged ≥18 years and 59 who were not able to be assigned to a risk category. Ninety-six percent of the cases were from Guatemala and 4% from either 1 of the Central American countries or the Caribbean islands Dominican Republic, St Vincent, and Haiti. The median age at surgery was 3.1 years (range, 1 day to 17.9 years). Most of the procedures were performed in the RACHS-1 risk categories 1 and 2, representing 79% (n=960) of the total (Table 1).
The most frequent procedure was closure of patent ductus arteriosus at 34% (n=408), followed by ventricular septal defect repair at 14% (n=170) and atrial septal defect repair at 12% (n=140) (Table 2).
A postoperative complication was present in 341 of 1215 procedures (28%). The most frequent were pleural effusion at 8% (n=92), pneumothorax at 3% (n=34), low cardiac output at 2% (n=27), bleeding at 2% (n=25), capillary leak syndrome at 2% (n=24), and arrhythmia at 2% (n=21). Nosocomial infections were present in 13.8% (n=168) of the cases. The median stay in the intensive care unit was 1 day (range, 1 to 78), and the median length of hospital stay was 6 days (1 to 97). The overall mortality rate was 10.7% (n=130).
When the performance of the RACHS-1 method in the Guatemalan data set was assessed, odds ratios for in-hospital mortality increased with increasing risk category and age group (highest for age ≤30 days). The area under the receiver operating characteristic curve was 0.854 for the model (Table 3). The proportion of surgical procedures done in each RACHS-1 risk category remained similar in the 3 time periods, but the total number increased from 72 in 1997 to 229 in 2003.
Figure 1 depicts the mortality rate for each surgical risk category over each time period. In every risk category, a decreasing trend in mortality rates across the 3 time periods was observed. In 2003–2004, the mortality rate decreased in category 1 to 0.5%, in category 2 to 7.4%, in category 3 to 23.3%, and in category 4 to 25%. The proportion of procedures performed by the senior surgeon decreased over time. He was the primary surgeon in 53% of the cases in 1997–1999, 18% in 2000–2002, and in only 4% in 2003–2004 (P<0.001). Likewise, his presence as either the primary surgeon or first assistant decreased from 62% in 1997 to 1999, to 22% in 2000 to 2002, and to 5% in 2003 to 2004 (P<0.001).
To compare the overall reduction in mortality across the 3 categories, SMRs were calculated. The 1997 to 1999 SMR was 10.0 (95% CI, 7.2 to 13.7), indicating a 10-fold increased risk for death compared with the 2000 US average. In 2000 to 2002, the risk was reduced to SMR 7.8 (95% CI, 5.9 to 10.0). In the last time period, 2003 to 2004, the SMR was reduced to 5.7 (95% CI, 3.8 to 8.3) and was statistically significant compared with the 1997 to 1999 time period (P=0.008) (Figure 2).
In a relatively short time period, mortality after congenital heart surgery has been reduced substantially in Guatemala. When the case complexity and in-hospital mortality in Guatemala alone are analyzed, a decreasing trend in risk-adjusted mortality was demonstrated in every category. For example, mortality in category 4 (the highest complexity performed on a regular basis in Guatemala at the time) fell from 57% in 1997 to 1999 to 25% in 2003 to 2004. In addition, when the Guatemalan mortality is compared against the US benchmark (KID 2000 database), a decrease in SMR from 10 in 1997 to 1999 to 5.7 in 2003 to 2004 was observed, showing a statistically significant difference (P=0.008). If no improvements occurred over time, and assuming that mortality remained as in 1997 to 1999, one would expect 10 additional deaths during 2000 to 2002 (of 537 total patients) and 18 additional deaths during 2003 to 2004 (of 382 total patients), mortalities that did not occur.
Few attempts have been made in the past to evaluate pediatric cardiac programs in developing countries. A Colombian study explored risk-adjusted mortality from 2001 to 2003 in the 4 centers for pediatric cardiac surgery in that country. Procedures in RACHS-1 categories 1 through 4 were performed, with a mortality rate of 0.7% in category 1, 7.2% in category 2, 20.7% in category 3, and 33.8% in category 4. A reduction in overall mortality was described from 10.9% in 2001 to 8.6% in 2002 and 7.7% in 2003; however, this comparison was not risk adjusted.13
Novick et al1 also explored risk-adjusted mortality in their program, which performs surgical procedures in several countries, including in the United States. They made comparisons between the first 5 years of their experience and the last 5 years, showing improvement based on overall reduction of mortality. Unfortunately, this conclusion was again based on non–risk-adjusted mortality.
Although the conditions in which cardiac surgery are practiced in the United States compared with the developing world are different, the KID 2000 database was selected as a benchmark because it comprises a wide variety of centers performing from just a few dozen to >600 procedures per year, providing a broad representation of congenital heart surgery in the United States, with diversity in case mix and volume.
We also found that the RACHS-1 method performed well in our Guatemalan database, with an area under the receiver operating characteristic curve of 0.854. It was even higher than in prior reports.4,12,14 Risk adjustment is important to accurately compare in-hospital mortality.4,15 This method would be useful for similar programs in the developing world.
Although it is difficult to quantify objective factors that might influence a reduction in mortality, certain changes appear to be important. In 2001, a pediatric intensive care unit was inaugurated, separating the services from the adult intensive care unit. At the same time, doctors and nurses were selected and further trained to work exclusively for the pediatric unit. It is interesting to note that the most substantial reduction in mortality was found in the same time period (2000 to 2001), with SMRs >10 before 2001 and <8 in subsequent years. This reduction was reflected by lower mortality rates in every one of the RACHS-1 categories.
Furthermore, our data show that the reduction in mortality found in all the RACHS-1 categories comes with an increase in volume; annual volumes were <150 in early years, increasing to >200 cases in 2001 and beyond. Although we cannot separate the effect of volume from other changes happening at the same time, case volume has proven to be an important determinant of difference in mortality for congenital heart surgery among institutions.16,17
The described improvement could not have been driven simply by the senior surgeon (A.R.C.) performing the surgeries because we found that his presence in the operating department has decreased significantly over time. He was the primary surgeon in 53% of the cases in 1997 to 1999, in 18% in 2000 to 2002, and in only 4% in 2003 to 2004 (P<0.001). Likewise, his presence as either the primary surgeon or first assistant decreased from 62% in 1997 to 1999, to 22% in 2000 to 2002, and to 5% in 2003 to 2004 (P<0.001).
Additionally, improvement has occurred in the preoperative echocardiographic diagnosis and in the catheterization laboratory; interventional procedures such as ductal stent placement or electrophysiological diagnostic and therapeutic procedures contributed to improvement of the patient’s condition before surgery. Improvements in anesthesia, cardiopulmonary bypass, and the routine implementation of early extubation have also reduced complications and in-hospital length of stay and consequently reduced hospital costs.18,19 Change of the sterilization system may have also reduced postoperative generalized acute systemic inflammatory responses.
This study has some limitations. A statistical difference was not found across time periods within each risk category because of small sample sizes after the data were divided into risk categories, thus reducing statistical power. However, a significant statistical difference was found when SMRs were analyzed, which allowed evaluation of all procedures, adjusted for the case mix. In this study we cannot analyze the effects of the socioeconomic factors on the presurgical and postoperative states because this information is available only in the aggregate but not for individual patients.
In conclusion, this study demonstrates substantial improvement in outcomes after congenital heart surgery in Guatemala and further shows that measurement of risk-adjusted mortality is feasible in pediatric cardiac programs in the developing world. The interest in the first world for outcome assessments and quality improvement is needed equally in the developing world, where resources are far more limited, and better results may stimulate appropriations from local and international sources.
Source of Funding
This study was supported in part by the Tommy Kaplan Cardiology Fellowship.
Country Reports on Human Rights Practices for 1998: Report Submitted to the Committee on Foreign Relations US Senate and the Committee on International Relations, U.S. House of Representatives/Guatemala. Washington, DC: Government Printing Office; 1999.
Press OU, ed. Human Development Report 1999. New York, NY: United Nations Development Program; 1999: 168–170.
Guatemala: el rostro rural del desarrollo humano. New York, NY: United Nations Development Program; 1999: 63–73.
Boethig D, Jenkins KJ, Hecker H, Thies WR, Breymann T. The RACHS-1 risk categories reflect mortality and length of hospital stay in a large German pediatric cardiac surgery population. Eur J Cardiothorac Surg. 2004; 26: 12–17.
Vélez J, Sandoval N, Cadavid E, Zapata J. Cooperative study of operatory mortality in the correction of congenital cardiopathies in Colombia. Revista Colombiana de Cardiologia. 2005; 11: 397–400.
Larsen SH, Pedersen J, Jacobsen J, Johnsen SP, Hansen OK, Hjortdal V. The RACHS-1 risk categories reflect mortality and length of stay in a Danish population of children operated for congenital heart disease. Eur J Cardiothorac Surg. 2005; 28: 877–881.
Hannan EL, Racz M, Kavey RE, Quaegebeur JM, Williams R. Pediatric cardiac surgery: the effect of hospital and surgeon volume on in-hospital mortality. Pediatrics. 1998; 101: 963–969.
Jenkins KJ, Newburger JW, Lock JE, Davis RB, Coffman GA, Iezzoni LI. In-hospital mortality for surgical repair of congenital heart defects: preliminary observations of variation by hospital caseload. Pediatrics. 1995; 95: 323–330.
The large underserved population of children with congenital heart disease in the developing world has very limited or no access to cardiovascular surgical care and, if untreated, will face irreversible heart and lung damage or death. The pediatric cardiac unit of Guatemala was created in 1997 to serve as a referral center for the Central American and Caribbean regions. This study demonstrates a rapid reduction in risk-adjusted mortality for this center compared with a US benchmark. Factors that might have influenced the reduction in mortality are also described. The current interest in outcome assessments and quality improvement is crucial in the developing world, where children should receive care that aims to achieve standards similar to those in the first world. In countries like Guatemala, resources are far more limited than they are in the United States, and better surgical results may stimulate appropriations from local and international sources. When dealing with international patients, risk-adjusted outcome information might enable the clinician to refer patients to centers with the best results for the given cardiac disease.