Donate Help Contact The AHA Sign In Home
American Heart Association
Circulation
Search: search_blue_button Advanced Search
« Previous Article | Table of Contents | Next Article »
Circulation. 2007;116:984-992
Published online before print August 13, 2007, doi: 10.1161/CIRCULATIONAHA.107.708537
CLINICAL PERSPECTIVE
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow CME: Take the course for this article:
 Circulation: August 28, 2007, Volume 116, Number 9
Right arrow All Versions of this Article:
116/9/984    most recent
CIRCULATIONAHA.107.708537v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow Request Permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Ingelsson, E.
Right arrow Articles by Vasan, R. S.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Ingelsson, E.
Right arrow Articles by Vasan, R. S.
Right arrowPubmed/NCBI databases
*Compound via MeSH
*Substance via MeSH
Medline Plus Health Information
*Metabolic Syndrome
Related Collections
Right arrow Epidemiology
Right arrow Pathophysiology
Right arrow Risk Factors

(Circulation. 2007;116:984-992.)
© 2007 American Heart Association, Inc.

Epidemiology

Multimarker Approach to Evaluate the Incidence of the Metabolic Syndrome and Longitudinal Changes in Metabolic Risk Factors

The Framingham Offspring Study

Erik Ingelsson, MD, PhD; Michael J. Pencina, PhD; Geoffrey H. Tofler, MD; Emelia J. Benjamin, MD, ScM; Katherine J. Lanier, MA; Paul F. Jacques, MD; Caroline S. Fox, MD, MPH; James B. Meigs, MD, MPH; Daniel Levy, MD; Martin G. Larson, ScD; Jacob Selhub, PhD; Ralph B. D’Agostino, Sr, PhD; Thomas J. Wang, MD*; Ramachandran S. Vasan, MD*

From the Framingham Heart Study, Boston University School of Medicine, Framingham, Mass (E.I., M.J.P., E.J.B., C.S.F., D.L., M.G.L., R.B.D., T.J.W., R.S.V.); Department of Mathematics (M.J.P., K.J.L., R.B.D.) and Cardiology Division and Preventive Medicine (E.J.B., R.S.V.), Boston University School of Medicine, Boston, Mass; Royal North Shore Hospital, Sydney, Australia (G.H.T.); Jean Mayer US Department of Agriculture Human Nutrition Research Center on Aging, Tufts University, Boston, Mass (P.F.J., J.S.); Division of Endocrinology, Brigham and Women’s Hospital, Boston, Mass (C.S.F.); Department of Medicine (J.B.M.) and Cardiology Division (T.J.W.), Massachusetts General Hospital, Boston (T.J.W.); and National Heart, Lung, and Blood Institute, Bethesda, Md (D.L.).

Correspondence to Ramachandran S. Vasan, MD, FACC, Framingham Heart Study, 73 Mt Wayte Ave, Framingham, MA 01702–5803. E-mail vasan{at}bu.edu

Received April 10, 2007; accepted June 15, 2007.


*    Abstract
up arrowTop
 *Abstract
down arrowIntroduction
 down arrowMethods
 down arrowResults
 down arrowDiscussion
 down arrowReferences
 
Background— The metabolic syndrome (MetS) is associated with increased cardiovascular risk. We evaluated the relative contributions of circulating biomarkers representing distinct biological pathways to the incidence of MetS and to longitudinal changes of its constituent risk factors.

Methods and Results— We measured 8 circulating biomarkers reflecting inflammation (C-reactive protein), hemostasis (plasminogen activator inhibitor-1, fibrinogen), neurohormonal activity (aldosterone, renin, B-type natriuretic peptide, N-terminal proatrial natriuretic peptide), and endothelial dysfunction (homocysteine) in 2292 Framingham Offspring Study participants (mean age, 57 years; 56% women). We related the biomarker panel to incidence of MetS on follow-up initially and then related biomarkers associated with incident MetS to longitudinal change in its components. On follow-up (mean, 2.9 years), 282 participants (of 1473 participants without prevalent MetS at baseline) developed MetS. After adjustment for clinical risk factors, the biomarker panel was associated with incident MetS (P=0.008). On backward elimination, plasminogen activator inhibitor-1 and aldosterone remained associated with incident MetS (multivariable-adjusted odds ratio per 1-SD increment log marker, 1.31 [P=0.004] and 1.21 [P=0.015], respectively). In multivariable analyses evaluating longitudinal change in MetS components (analyzed as continuous variables), plasminogen activator inhibitor-1 was significantly and positively associated with changes in fasting glucose, systolic blood pressure, and triglycerides (all P<0.05). Serum aldosterone was associated positively with change in systolic blood pressure (P=0.023) and inversely with change in high-density lipoprotein cholesterol (P=0.001).

Conclusions— Higher circulating plasminogen activator inhibitor-1 and aldosterone levels are associated with the development of MetS and with longitudinal change of its components, suggesting that these biomarkers and related pathways play a key role in mediating metabolic risk.


Key Words: aldosterone • epidemiology • metabolic syndrome X • plasminogen activator inhibitor 1 • risk factors


*    Introduction
up arrowTop
 up arrowAbstract
 *Introduction
down arrowMethods
 down arrowResults
 down arrowDiscussion
 down arrowReferences
 
Metabolic risk factors, including visceral obesity, glucose intolerance, and dyslipidemia, often cluster in individuals,1,2 a construct called the metabolic syndrome (MetS). Although the clinical use of MetS has been questioned,3,4 there is agreement that clustering of risk factors portends an increased risk of cardiovascular disease. It is widely accepted that insulin resistance is a key feature of MetS, although overlap between the 2 conditions is only partial.5 Indeed, a variety of metabolic, neurohormonal, and inflammatory abnormalities may accompany MetS. In cross-sectional studies, the presence of MetS has been correlated with higher circulating concentrations of inflammatory markers,6,7 hemostatic markers,8 and neurohormonal activation.9,10 Elevation of several of these biomarkers may antedate development of risk factors such as type 2 diabetes11–13 and hypertension.14–16 However, it is less clear if alterations of biomarkers are pathogenetically related to the incidence of MetS itself. Previous studies have not examined conjoint and relative contributions of multiple biomarkers representing key pathogenetic pathways for predicting the development of MetS and its components.

Clinical Perspective p 992

We hypothesized that select circulating biomarkers are associated with the incidence of MetS and with longitudinal tracking of its components. We tested this hypothesis by evaluating a comprehensive panel of biomarkers representing the following pathways: inflammation (high-sensitivity C-reactive protein), hemostasis (plasminogen activator inhibitor-1 [PAI-1], fibrinogen), neurohormonal activation (aldosterone, renin, B-type natriuretic peptide [BNP], N-terminal proatrial natriuretic peptide [N-ANP]), and endothelial dysfunction (homocysteine).


*    Methods
up arrowTop
 up arrowAbstract
 up arrowIntroduction
 *Methods
down arrowResults
 down arrowDiscussion
 down arrowReferences
 
Study Sample
The design and selection criteria of the Framingham Offspring Study have been described previously.17 Participants were eligible for the present investigation if they attended both the sixth (1995 to 1998; baseline) and seventh (1998 to 2001; follow-up visit) examination cycles. Of 3264 eligible attendees, we excluded 972 individuals for the following reasons: prevalent cardiovascular disease (coronary heart disease, cerebrovascular disease, intermittent claudication, or congestive heart failure; n=359), prevalent diabetes (fasting plasma glucose ≥126 mg/dL or use of insulin or oral hypoglycemic agents18; n=228), serum creatinine >2.0 mg/dL (n=7), missing covariates (n=226), or missing biomarker data (n=152). After exclusions, 2292 individuals (mean age, 57 years; 56% women) remained eligible for our study. This larger sample was the sampling frame for analyses of longitudinal change in individual components of MetS (see the Statistical Analyses section for rationale). A smaller sample of 1473 participants was eligible for analyses of the incidence of MetS after exclusion of 819 participants with prevalent MetS at baseline. The Institutional Review Board at Boston University Medical Center approved the study, and all participants gave written informed consent.

Assessment of Clinical Covariates
At each Framingham Heart Study examination, participants undergo routine medical history, physical examination using a standardized protocol (including blood pressure [BP] measurement and anthropometry), and laboratory assessment of risk factors. A physician measured BP using a mercury column sphygmomanometer after the participant had rested in a seated position for 5 minutes; the average of 2 readings obtained on the participant’s left arm constituted the examination BP. Fasting levels of plasma glucose, triglycerides, and high-density lipoprotein (HDL) cholesterol were measured with standardized assays. Cigarette smoking was defined by self-reported use within the year preceding the baseline examination.

At examination cycle 5 ({approx}4 years before the study baseline), fasting plasma glucose and insulin levels were measured. We calculated homeostasis model assessment insulin resistance index (HOMA-IR) using this formula:


Formula 1

Biomarker Measurements at Baseline Examination
At the baseline examination, phlebotomy was performed on fasting participants who had rested for 5 to 10 minutes in a supine position, typically between 8 and 9 AM. Specimens were stored at –80°C without freeze-thaw cycles until assay.

High-sensitivity C-reactive protein was measured with a Dade Behring BN100 nephelometer. ELISA methods were used to measure plasma levels of PAI-1 antigen (TintElize PAI-1, Biopool, Ventura, Calif). Plasma fibrinogen was determined by the Clauss method. Serum aldosterone was measured by radioimmunoassay (Quest Diagnostics, Cambridge, Mass), and plasma renin was measured by immunochemiluminometric assay (Nichols assay, Quest Diagnostics). Plasma BNP and N-terminal proatrial natriuretic peptide were measured with high-sensitivity immunoradiometric assays (Shionogi, Osaka, Japan). Total plasma homocysteine was measured with high-performance liquid chromatography with fluorometric detection. The average interassay coefficients of variation for the biomarkers were as follows: high-sensitivity C-reactive protein, 2.2%; PAI-1, 7.7%; fibrinogen, 2.6%; aldosterone, 4.0% for high concentrations and 9.8% for low concentrations; renin, 2.0% for high concentrations and 10.0% for low concentrations; BNP, 12.2%; N-terminal proatrial natriuretic peptide, 12.7%; and homocysteine, 9%.

Incidence of MetS
The incidence of MetS was ascertained at the seventh examination (mean, 2.9 years from baseline; range, 1.0 to 6.2 years) among 1473 individuals (878 women) free of the condition at baseline. MetS was defined according to the modified definition of the National Cholesterol Education Program Adult Treatment Panel III guidelines20 as the presence of ≥3 of the following: increased waist circumference (≥102 cm for men, ≥88 cm for women), elevated BP (≥130 mm Hg systolic, ≥85 mm Hg diastolic) or treatment for high BP, hyperglycemia (fasting glucose ≥100 mg/dL) or treatment with oral hypoglycemic agents or insulin, hypertriglyceridemia (≥150 mg/dL) or treatment with lipid-lowering treatment, and low HDL cholesterol (<40 mg/dL in men, <50 mg/dL in women).

Longitudinal Changes in Component Risk Factors on Follow-Up
For biomarkers related to MetS incidence, we also evaluated the relations to longitudinal changes in levels of metabolic risk factors that define MetS (see below for conservative analytical strategy). Thus, we evaluated changes in levels of the following risk factors between the baseline and follow-up examinations: waist circumference, systolic and diastolic BPs, plasma glucose, triglycerides, and HDL cholesterol. The larger sample of 2292 participants was eligible for these analyses to maximize our statistical power; exclusion of people with prevalent MetS was not necessary for these analyses; rather, we excluded individuals on treatment at baseline for a given risk factor.

Statistical Analyses
All biomarkers, except fibrinogen and BNP, displayed right-skewed distributions and were logarithmically transformed. Because more than one quarter of BNP values were below the assay detection limit and were assigned the minimum detectable value, we used sex-specific quartiles of BNP in our models. Additionally, all other markers were sex standardized to facilitate easier comparisons and to account for sex differences in their distributions. This sex standardization was accomplished by subtracting from each value the sex-specific mean and dividing by sex-specific SD. Pairwise Pearson correlation coefficients were calculated to assess the relations among the biomarkers.

Multimarker Panel and Incidence of MetS
To minimize multiple statistical testing, we used a conservative 2-step strategy to relate the biomarkers to the incidence of MetS. First, we used multivariable logistic regression to relate the entire panel of biomarkers to the incidence of MetS. We adjusted for the following covariates (all defined at the baseline examination): age, sex, systolic and diastolic BPs, fasting glucose, HDL cholesterol, logarithm of triglycerides, and sex-standardized waist circumference. We estimated a global probability value for the biomarker panel as a whole using a likelihood ratio test in which –2 log likelihood for the model with clinical covariates and biomarkers was subtracted from –2 log likelihood for the model with clinical covariates only. The purpose of this global probability value was to determine whether the biomarker panel as a whole was associated with MetS incidence. We performed further analyses of individual biomarkers only if the global probability value was <0.05. Second, we selected a final set of informative biomarkers with backward elimination using an individual threshold of P<0.05 for retention of biomarkers in the model. In secondary analyses, we added HOMA-IR as a covariate to the multivariable model to explore whether the association of biomarkers with incident MetS was confounded by insulin resistance.

Additional analyses focused on the biomarkers that were individually retained on backward elimination (referred to as informative biomarkers for simplicity). We assessed the effect modification of the biomarker-MetS relations by age and sex by incorporating appropriate interaction terms in multivariable models. We also assessed their conjoint influence on the incidence of MetS by cross-classifying participants according to sex-standardized median values and determining whether individuals with values above the median (high) on ≥1 biomarkers alone and in combination had a greater risk of MetS compared with those with concentrations of informative biomarkers below the median (low).

In addition, we constructed separate receiver-operating characteristics curves for the informative biomarkers to estimate sex-specific cutoff points that maximized the sums of sensitivity and specificity for predicting new-onset MetS.21 These cutoff points were used to examine the sensitivities, specificities, positive predictive values (PPVs), and negative predictive values (NPVs) for incident MetS. We estimated the c statistic (area under the receiver-operating characteristics curve) for multivariable models incorporating the informative biomarkers and compared it with that for models without the biomarkers.

Informative Biomarkers and Longitudinal Change in Metabolic Risk Factors
For the informative biomarkers in the panel, we examined the relations of the baseline biomarker levels (modeled together) to longitudinal changes in the metabolic risk factors constituting MetS, with separate analyses for each risk factor. We excluded participants using BP-lowering or lipid-lowering treatment at baseline for analyses of changes in BP and triglycerides (individuals with diabetes at baseline were excluded from the study sample as noted earlier). We used sex-pooled multivariable linear regression models with the change in levels of risk factors as the dependent continuous variables; change in waist circumference was standardized within sex. For systolic BP, diastolic BP, plasma glucose, and logarithm of triglycerides, we used censored normal regression to account for treatment for high BP, diabetes, or dyslipidemia at the follow-up examination.22 Models were adjusted for age, sex, baseline body mass index (except for models with waist circumference as the dependent variable), and baseline level of the individual risk factor analyzed. Two-sided values of P<0.05 were considered statistically significant. All analyses were performed with SAS 9.1 (SAS Institute, Cary, NC) and STATA 8.2 (Stata Corp, College Station, Tex).

The authors had full access to and take full responsibility for the integrity of the data. All authors have read and agree to the manuscript as written.


*    Results
up arrowTop
 up arrowAbstract
 up arrowIntroduction
 up arrowMethods
 *Results
down arrowDiscussion
 down arrowReferences
 
Baseline Biomarkers and Clinical Characteristics
The baseline clinical characteristics of the study sample are shown in Table 1. Most of the biomarkers demonstrated a modest pairwise correlation (correlation coefficients ranging from –0.23 to 0.31; Table 2). Stronger correlations were evident for CRP and fibrinogen (r=0.52) and for the 2 natriuretic peptides (r=0.57).


View this table:
[in this window]
[in a new window]

  		TABLE 1. Baseline Characteristics of the Study Sample*


View this table:
[in this window]
[in a new window]

  		TABLE 2. Correlations Among Biomarkers*

Relations of the Multimarker Panel and Incidence of MetS
On follow-up, 282 participants (142 women) developed new-onset MetS. The panel of 8 biomarkers was associated with incidence of MetS (global test, P=0.008; Table 3). In stepwise logistic regression with backward elimination of biomarkers (adjusting for age, sex, systolic and diastolic BPs, glucose, high-density lipoprotein (HDL) cholesterol, logarithm of triglycerides, and sex-standardized waist circumference, all forced into the model), PAI-1 and aldosterone were significantly associated with MetS (P=0.004 and P=0.015, respectively; Table 3). There was no effect modification by age or sex for either of these markers (interaction terms were not significant).


View this table:
[in this window]
[in a new window]

  		TABLE 3. Biomarker Concentrations and Incident MetS Excluding Participants With Prevalent MetS (n=1473)

When HOMA-IR measured 4 years before the baseline was added as a covariate to the multivariable model, the relations of PAI-1 (multivariable-adjusted odds ratio [OR] per SD of log marker, 1.30; 95% CI, 1.07 to 1.58; P=0.007) and aldosterone (multivariable-adjusted OR per SD of log marker, 1.22; 95% CI, 1.04 to 1.43; P=0.014) to incident MetS remained robust; SD of log PAI-1 corresponds to 18.9 ng/mL, and SD of log aldosterone corresponds to 7.2 ng/dL increase above the median in original units. Additionally, HOMA-IR was significantly associated with incidence of MetS (multivariable-adjusted OR, 1.13; 95% CI, 1.04 to 1.23 per SD [1.8]; P=0.004). These results remained essentially identical when insulin resistance, defined as the highest quartile of HOMA-IR, was incorporated into the model as a binary variable instead of as a continuous variable.

The incidence of MetS increased across quartiles of PAI-1 and aldosterone (Table 4). There was a statistically significant 19% to 23% increase in risk of MetS per quartile increment in PAI-1 and aldosterone (trend tests, Table 4). Table 4 also presents results for the risk of MetS for different combinations of high versus low PAI-1 and aldosterone (defined on the basis of sex-standardized median levels), with the group with both PAI-1 and aldosterone under the median serving as the referent. Individuals with levels of both PAI-1 and aldosterone above the median experienced a doubling in the risk for incident MetS (Table 4).


View this table:
[in this window]
[in a new window]

  		TABLE 4. Logistic Regression Analysis Examining Biomarkers in Quartiles* and in Combinations to Incidence of MetS Excluding Participants With Prevalent MetS at Baseline

Evaluation of Performance of PAI-1 and Aldosterone for Predicting MetS
To evaluate the potential clinical usefulness of these markers for predicting new-onset MetS, we calculated the sensitivities, specificities, PPVs, and NPVs for sex-specific cut points derived from receiver-operating characteristics curve analyses (based on maximizing the sum of sensitivity and specificity). The cut points chosen for PAI-1 were ≥21.9 ng/mL for men (sensitivity, 0.75; specificity, 0.57; PPV, 0.61; NPV, 0.72) and ≥27.7 ng/mL for women (sensitivity, 0.53; specificity, 0.86; PPV, 0.69; NPV, 0.75). The threshold chosen for aldosterone was ≥11.0 ng/dL for both men and women (sensitivity, 0.46; specificity, 0.63; PPV, 0.53; NPV, 0.57 for men; and sensitivity, 0.54; specificity, 0.55; PPV, 0.42; NPV, 0.67 for women). The c statistic for predicting incident MetS was 0.81 for a model that incorporated age, sex, and the aforementioned clinical covariates. If PAI-1 and aldosterone were incorporated simultaneously into the model, the c statistic increased to 0.82 (P=0.11 versus model without the 2 biomarkers).

Relations of PAI-1 and Aldosterone and Longitudinal Tracking of Metabolic Risk Factors
Given that PAI-1 and aldosterone were the biomarkers retained in analyses relating the panel to incident MetS, analyses of individual components of MetS focused on these 2 biomarkers (modeled together). In multivariable-adjusted models, PAI-1 was significantly and positively associated with longitudinal changes in fasting glucose, systolic BP, and triglycerides (Table 5). Aldosterone was significantly associated with an increase in mean systolic BP and a decrease in mean HDL cholesterol on follow-up (Table 5).


View this table:
[in this window]
[in a new window]

  		TABLE 5. Linear and Censored Normal Regression Analysis* Examining the Relations of Change in Individual Risk Factors According to PAI-1 and Aldosterone Levels (Modeled Together)


*    Discussion
up arrowTop
 up arrowAbstract
 up arrowIntroduction
 up arrowMethods
 up arrowResults
 *Discussion
down arrowReferences
 
Principal Findings
In a community-based sample, we jointly examined the association of MetS and its components with 8 biomarkers representing distinct biological pathways a priori hypothesized to be related to incidence of MetS. In multivariable analyses relating the biomarkers conjointly to MetS, both PAI-1 and aldosterone were significantly associated with the incidence of MetS, relations that remained robust on adjustment for HOMA-IR. PAI-1 and aldosterone also were associated with longitudinal change in several metabolic risk factors. Specifically, higher PAI-1 was associated with longitudinal increases in fasting plasma glucose, systolic BP, and triglycerides. Higher aldosterone was directly related to change in systolic BP and inversely related to change in HDL cholesterol. These findings notwithstanding, receiver-operating characteristics curve analyses evaluating the potential utility of these biomarkers for predicting MetS demonstrated a suboptimal performance overall, suggesting that these biomarkers are unlikely to prove useful as screening tools. Nonetheless, the association of both PAI-1 and aldosterone with tracking of risk factors is of great pathophysiological interest.

Previous Studies of the Biomarkers and MetS
Many cross-sectional studies have demonstrated that individual biomarkers are associated with MetS.6–10,23 Other studies have demonstrated that elevation of select biomarkers antedates the development of weight gain,24,25 diabetes,11–13 and hypertension.14–16 For instance, higher CRP levels predict the incidence of diabetes13 and hypertension,15 suggesting a potential role for inflammation in the pathogenesis of these conditions. Furthermore, biomarkers of endothelial dysfunction have been reported to predict incident diabetes.12 Data from some randomized trials indicate that therapy with angiotensin-converting enzyme inhibitors or angiotensin-receptor blockers is associated with a reduced risk of diabetes,9 providing indirect evidence that neurohormonal activation may precede glucose intolerance. Prior studies have not examined the joint and comparative value of a panel of biomarkers representing distinct biological pathways for predicting the development of MetS.

PAI-1 and MetS
Several cross-sectional studies have demonstrated that higher PAI-1 levels are associated with prevalent MetS26 and its components (such as obesity,27 increased blood glucose levels,28 or dyslipidemia29). Increased PAI-1 levels have even been proposed to be a true component of MetS.26,30 Furthermore, a higher PAI-1 level also has been demonstrated to predict the incidence of diabetes in community-based samples.31 However, it is unclear whether PAI-1 levels predict incident MetS prospectively.

Several potential mechanisms may explain the association of PAI-1 with new-onset MetS and with tracking of several metabolic risk factors. It has been suggested that PAI-1 participates in the development and regulation of adipose tissue. Studies indicate that PAI-1 synthesis is increased by insulin, glucocorticoids, angiotensin II, fatty acids, and cytokines.32 Furthermore, experimental studies indicate that PAI-1 may have a causal role in the development of obesity and insulin resistance.33–35 PAI-1–deficient mice develop lower body weight, glucose levels, and triglyceride levels, as well as lesser insulin resistance, compared with wild-type mice when fed a high-energy diet.34,35 These PAI-1–deficient animals also have normal levels of peroxisome proliferator–activated receptor-{gamma} and adiponectin mRNA, key regulators of glucose and lipid metabolism, when fed a high-energy diet, in contrast to downregulation of these pathways in their wild-type counterparts with normal PAI-1.34,35 These findings are consistent with the notion that the absence of PAI-1 in adipocytes protects against insulin resistance by promoting glucose uptake and adipocyte differentiation via increased peroxisome proliferator–activated receptor-{gamma} expression.33 Furthermore, it was demonstrated recently that a pharmacological inhibitor of PAI-1 can diminish glucose-stimulated PAI-1 increases and reduce differentiation in human adipocytes.36 In addition, it could decrease adiposity and levels of glucose and triglycerides in a murine model.36

Apart from the above-discussed mechanisms that might link increased PAI-1 concentrations to the obesity, hyperglycemia, and dyslipidemia components of MetS, studies also link increased PAI-1 levels to the high BP component. Higher PAI-1 levels may reflect endothelial dysfunction,37 an important component of hypertension. Higher vascular PAI-1 appears to accelerate perivascular and medial fibrosis, whereas suppression of PAI-1 protects against the vascular changes observed in experimental models of hypertension.38

Aldosterone and MetS
Two recent cross-sectional studies have demonstrated that higher aldosterone concentrations are associated with a greater prevalence of MetS and components of MetS such as increased BP, triglycerides, and waist circumference and decreased HDL cholesterol.39,40 However, these findings contrast with another report41 that did not demonstrate an association of aldosterone and MetS. Low HDL cholesterol also has been correlated with high aldosterone concentrations in other studies,42–44 and weight loss has been shown to decrease serum concentrations of several markers in the renin-angiotensin-aldosterone system, including aldosterone.45 Several recent cross-sectional studies have reported direct correlations between aldosterone and insulin resistance.40,44,46 Further evidence for the involvement of the renin-angiotensin-aldosterone system in the metabolic processes underlying development of MetS is provided by clinical trials demonstrating that blockers of the renin-angiotensin-aldosterone system (angiotensin-converting enzyme inhibitors and angiotensin receptor blockers) decrease the risk of future diabetes.47 We have previously reported that higher aldosterone concentrations are associated with hypertension prospectively.48 However, we are not aware of other published data relating aldosterone levels to the development of MetS and tracking of HDL cholesterol.

High aldosterone concentrations might cause increased BP through multiple mechanisms, including increased renal sodium retention, potentiation of angiotensin II action, impaired endothelial function, and reduced vascular compliance.48 Less is known about the potential mechanisms linking high aldosterone concentrations to low HDL, but they might involve the metabolism of adipose tissue, which is known to have high activity of the renin-angiotensin-aldosterone system.49 In vitro studies support a role for aldosterone in the adipogenesis by mediating early adipocyte differentiation,50,51 but it is less clear whether this could contribute to the observed inverse relation with HDL cholesterol.

Multiple Biomarkers and Prediction of MetS
Of the 8 biomarkers evaluated, PAI-1 and aldosterone emerged as the biomarkers most significantly associated with MetS after adjustment for clinical covariates. One possible interpretation of the findings is that PAI-1 and aldosterone are key markers of incident MetS in the community because only these markers remained significant in the multivariable models. Experimental studies support this interpretation, as discussed above. However, it must be recognized that these 2 biomarkers were chosen on the basis of backward elimination in regression models. Biomarkers with greater analytical precision or low interassay variation may be retained in models without indicating lesser biological relevance of biomarkers that are not retained in the models. These cautionary comments notwithstanding, the present investigation extends the multimarker concept to the prediction of MetS and its components, a concept that recently has been applied to cardiovascular disease52 and hypertension.53

Study Strengths and Limitations
The strengths of the present study include the large community-based sample; routine assessment of circulating biomarkers; availability of standardized clinical covariates, including insulin resistance; and use of multivariable analyses to examine the biomarkers conjointly and comparatively. Nonetheless, the present study has several limitations. First, for select biomarkers, measurement-related issues may have limited our power to detect a relation. For plasma BNP levels, a significant proportion of individuals (26% of women, 41% of men) were below the detection limit of the assay.54 For plasma renin and serum aldosterone, we used single-occasion measurements obtained on a random sodium diet and without a period of longer rest in a supine position. Thus, it may be questioned whether a single reading adequately represents the renin and aldosterone profiles of participants. Second, as noted above, in a stepwise selection procedure, the biomarkers retained in the final model are not necessarily the ones that are biologically most relevant. Third, there has been considerable debate regarding the choice of different definitions of MetS23,55,56 and regarding the utility of the condition itself.3,4 These controversies notwithstanding, our data provide insights into which biomarkers predict the development of metabolic traits alone and in combination (exemplified by MetS). Fourth, because our sample consists predominantly of white and middle-aged to elderly participants, the generalizability of our findings to other ethnicities and younger individuals is unknown. Fifth, because measurements of insulin levels were not available at the baseline of the present investigation, we had to use measurements from the prior examination cycle {approx}4 years earlier. The fact that insulin resistance measured 4 years before the baseline was a strong predictor of incident MetS after baseline suggests that these exploratory analyses were of some value. Nevertheless, the time interval between insulin measurements and the baseline examination is an important limitation, and there may be some residual confounding resulting from the lack of characterization of insulin resistance using contemporaneous measurements. In addition, the mean follow-up time in our study was only 2.9 years, so our results reflect the association of biomarkers with metabolic syndrome over a short-term period. Finally, we acknowledge that the panel of biomarkers will always be somewhat arbitrary based on practical limitations; it may not be feasible for any given study to examine all possible biomarkers.

Conclusions
In our large community-based cohort, a conservative analytical strategy evaluating a panel of important biomarkers demonstrated that higher plasma PAI-1 and serum aldosterone concentrations are associated with future development of MetS and with longitudinal changes in metabolic risk factors. Analyses of the discriminative utility of these biomarkers for predicting incident MetS demonstrated unsatisfactory performance overall, suggesting that these biomarkers are unlikely to prove useful as screening to ols. Our findings are primarily of pathophysiological importance. Further studies are warranted to confirm our findings and to elucidate the pathophysiological mechanisms underlying the observed associations.


*    Acknowledgments
 
Sources of Funding

This work was supported by the Swedish Heart-Lung Foundation and the Swedish Society of Medicine (Dr Ingelsson); NIH/NHLBI NO1-HC-25195, K23-HL074077-01 (Dr Wang), and K24-HL-04334 (Dr Vasan); and a career development award from the ADA (Dr Meigs).

Disclosures

None.


*    References
up arrowTop
 up arrowAbstract
 up arrowIntroduction
 up arrowMethods
 up arrowResults
 up arrowDiscussion
 *References
 

  1. Kylin E. Studien über das Hypertonie-Hyperglykämie-Hyperurikämie-syndrom. Zentralblatt für innere Medizin. 1923; 44: 105–127.
  2. Reaven GM. Banting Lecture 1988: role of insulin resistance in human disease. Diabetes. 1988; 37: 1595–1607.[Abstract]
  3. Kahn R, Buse J, Ferrannini E, Stern M. The metabolic syndrome: time for a critical appraisal: joint statement from the American Diabetes Association and the European Association for the Study of Diabetes. Diabetes Care. 2005; 28: 2289–2304.[Abstract/Free Full Text]
  4. Reaven GM. The metabolic syndrome: is this diagnosis necessary? Am J Clin Nutr. 2006; 83: 1237–1247.[Abstract/Free Full Text]
  5. Reilly MP, Wolfe ML, Rhodes T, Girman C, Mehta N, Rader DJ. Measures of insulin resistance add incremental value to the clinical diagnosis of metabolic syndrome in association with coronary atherosclerosis. Circulation. 2004; 110: 803–809.[CrossRef][Medline] [Order article via Infotrieve]
  6. Festa A, D’Agostino R Jr, Howard G, Mykkanen L, Tracy RP, Haffner SM. Chronic subclinical inflammation as part of the insulin resistance syndrome: the Insulin Resistance Atherosclerosis Study (IRAS). Circulation. 2000; 102: 42–47.[Abstract/Free Full Text]
  7. Ridker PM, Buring JE, Cook NR, Rifai N. C-reactive protein, the metabolic syndrome, and risk of incident cardiovascular events: an 8-year follow-up of 14 719 initially healthy American women. Circulation. 2003; 107: 391–397.[CrossRef][Medline] [Order article via Infotrieve]
  8. Mansfield MW, Heywood DM, Grant PJ. Circulating levels of factor VII, fibrinogen, and von Willebrand factor and features of insulin resistance in first-degree relatives of patients with NIDDM. Circulation. 1996; 94: 2171–2176.[Medline] [Order article via Infotrieve]
  9. Prasad A, Quyyumi AA. Renin-angiotensin system and angiotensin receptor blockers in the metabolic syndrome. Circulation. 2004; 110: 1507–1512.[CrossRef][Medline] [Order article via Infotrieve]
  10. Olsen MH, Hansen TW, Christensen MK, Gustafsson F, Rasmussen S, Wachtell K, Borch-Johnsen K, Ibsen H, Jorgensen T, Hildebrandt P. N-terminal pro brain natriuretic peptide is inversely related to metabolic cardiovascular risk factors and the metabolic syndrome. Hypertension. 2005; 46: 660–666.[Abstract/Free Full Text]
  11. Festa A, Williams K, Tracy RP, Wagenknecht LE, Haffner SM. Progression of plasminogen activator inhibitor-1 and fibrinogen levels in relation to incident type 2 diabetes. Circulation. 2006; 113: 1753–1759.[Abstract/Free Full Text]
  12. Meigs JB, Hu FB, Rifai N, Manson JE. Biomarkers of endothelial dysfunction and risk of type 2 diabetes mellitus. JAMA. 2004; 291: 1978–1986.[Abstract/Free Full Text]
  13. Pradhan AD, Manson JE, Rifai N, Buring JE, Ridker PM. C-reactive protein, interleukin 6, and risk of developing type 2 diabetes mellitus. JAMA. 2001; 286: 327–334.[Abstract/Free Full Text]
  14. Freitag MH, Larson MG, Levy D, Benjamin EJ, Wang TJ, Leip EP, Wilson PW, Vasan RS. Plasma brain natriuretic peptide levels and blood pressure tracking in the Framingham Heart Study. Hypertension. 2003; 41: 978–983.[Abstract/Free Full Text]
  15. Sesso HD, Buring JE, Rifai N, Blake GJ, Gaziano JM, Ridker PM. C-reactive protein and the risk of developing hypertension. JAMA. 2003; 290: 2945–2951.[Abstract/Free Full Text]
  16. Wang TJ, Evans JC, Meigs JB, Rifai N, Fox CS, D’Agostino RB, Levy D, Vasan RS. Low-grade albuminuria and the risks of hypertension and blood pressure progression. Circulation. 2005; 111: 1370–1376.[Abstract/Free Full Text]
  17. Kannel WB, Feinleib M, McNamara PM, Garrison RJ, Castelli WP. An investigation of coronary heart disease in families: the Framingham Offspring Study. Am J Epidemiol. 1979; 110: 281–290.[Abstract/Free Full Text]
  18. Report of the Expert Committee on the Diagnosis and Classification of Diabetes Mellitus. Diabetes Care. 2003; 26 (suppl 1): S5–S20.[CrossRef][Medline] [Order article via Infotrieve]
  19. Matthews DR, Hosker JP, Rudenski AS, Naylor BA, Treacher DF, Turner RC. Homeostasis model assessment: insulin resistance and beta-cell function from fasting plasma glucose and insulin concentrations in man. Diabetologia. 1985; 28: 412–419.[CrossRef][Medline] [Order article via Infotrieve]
  20. Grundy SM, Cleeman JI, Daniels SR, Donato KA, Eckel RH, Franklin BA, Gordon DJ, Krauss RM, Savage PJ, Smith SC Jr, Spertus JA, Costa F. Diagnosis and management of the metabolic syndrome: an American Heart Association/National Heart, Lung, and Blood Institute scientific statement. Circulation. 2005; 112: 2735–2752.[CrossRef][Medline] [Order article via Infotrieve]
  21. Kraemer H. Evaluating Medical Tests: Objective and Quantitative Guidelines. Newbury Park, Calif: Sage Publications; 1992.
  22. Tobin MD, Sheehan NA, Scurrah KJ, Burton PR. Adjusting for treatment effects in studies of quantitative traits: antihypertensive therapy and systolic blood pressure. Stat Med. 2005; 24: 2911–2935.[CrossRef][Medline] [Order article via Infotrieve]
  23. Alberti KG, Zimmet PZ. Definition, diagnosis and classification of diabetes mellitus and its complications, part 1: diagnosis and classification of diabetes mellitus provisional report of a WHO consultation. Diabet Med. 1998; 15: 539–553.[CrossRef][Medline] [Order article via Infotrieve]
  24. Barzilay JI, Forsberg C, Heckbert SR, Cushman M, Newman AB. The association of markers of inflammation with weight change in older adults: the Cardiovascular Health Study. Int J Obes (Lond). 2006; 30: 1362–1367.[CrossRef][Medline] [Order article via Infotrieve]
  25. Duncan BB, Schmidt MI, Chambless LE, Folsom AR, Carpenter M, Heiss G. Fibrinogen, other putative markers of inflammation, and weight gain in middle-aged adults: the ARIC study: Atherosclerosis Risk in Communities. Obes Res. 2000; 8: 279–286.[Medline] [Order article via Infotrieve]
  26. Mertens I, Verrijken A, Michiels JJ, Van der PM, Ruige JB, Van Gaal LF. Among inflammation and coagulation markers, PAI-1 is a true component of the metabolic syndrome. Int J Obes (Lond). 2006; 30: 1308–1314.[CrossRef][Medline] [Order article via Infotrieve]
  27. Cigolini M, Targher G, Bergamo Andreis IA, Tonoli M, Agostino G, De Sandre G. Visceral fat accumulation and its relation to plasma hemostatic factors in healthy men. Arterioscler Thromb Vasc Biol. 1996; 16: 368–374.[Abstract/Free Full Text]
  28. Meigs JB, Mittleman MA, Nathan DM, Tofler GH, Singer DE, Murphy-Sheehy PM, Lipinska I, D’Agostino RB, Wilson PW. Hyperinsulinemia, hyperglycemia, and impaired hemostasis: the Framingham Offspring Study. JAMA. 2000; 283: 221–228.[Abstract/Free Full Text]
  29. Gray RP, Panahloo A, Mohamed-Ali V, Patterson DL, Yudkin JS. Proinsulin-like molecules and plasminogen activator inhibitor type 1 (PAI-1) activity in diabetic and non-diabetic subjects with and without myocardial infarction. Atherosclerosis. 1997; 130: 171–178.[CrossRef][Medline] [Order article via Infotrieve]
  30. Alessi MC, Juhan-Vague I. PAI-1 and the metabolic syndrome: links, causes, and consequences. Arterioscler Thromb Vasc Biol. 2006; 26: 2200–2207.[Abstract/Free Full Text]
  31. Festa A, D’Agostino R Jr, Tracy RP, Haffner SM. Elevated levels of acute-phase proteins and plasminogen activator inhibitor-1 predict the development of type 2 diabetes: the Insulin Resistance Atherosclerosis Study. Diabetes. 2002; 51: 1131–1137.[Abstract/Free Full Text]
  32. Skurk T, Hauner H. Obesity and impaired fibrinolysis: role of adipose production of plasminogen activator inhibitor-1. Int J Obes Relat Metab Disord. 2004; 28: 1357–1364.[CrossRef][Medline] [Order article via Infotrieve]
  33. Liang X, Kanjanabuch T, Mao SL, Hao CM, Tang YW, Declerck PJ, Hasty AH, Wasserman DH, Fogo AB, Ma LJ. Plasminogen activator inhibitor-1 modulates adipocyte differentiation. Am J Physiol Endocrinol Metab. 2006; 290: E103–E113.[Abstract/Free Full Text]
  34. Ma LJ, Mao SL, Taylor KL, Kanjanabuch T, Guan Y, Zhang Y, Brown NJ, Swift LL, McGuinness OP, Wasserman DH, Vaughan DE, Fogo AB. Prevention of obesity and insulin resistance in mice lacking plasminogen activator inhibitor 1. Diabetes. 2004; 53: 336–346.[Abstract/Free Full Text]
  35. Schafer K, Fujisawa K, Konstantinides S, Loskutoff DJ. Disruption of the plasminogen activator inhibitor 1 gene reduces the adiposity and improves the metabolic profile of genetically obese and diabetic ob/ob mice. FASEB J. 2001; 15: 1840–1842.[Free Full Text]
  36. Crandall DL, Quinet EM, El Ayachi S, Hreha AL, Leik CE, Savio DA, Juhan-Vague I, Alessi MC. Modulation of adipose tissue development by pharmacological inhibition of PAI-1. Arterioscler Thromb Vasc Biol. 2006; 26: 2209–2215.[CrossRef][Medline] [Order article via Infotrieve]
  37. Huvers FC, De Leeuw PW, Houben AJ, De Haan CH, Hamulyak K, Schouten H, Wolffenbuttel BH, Schaper NC. Endothelium-dependent vasodilatation, plasma markers of endothelial function, and adrenergic vasoconstrictor responses in type 1 diabetes under near-normoglycemic conditions. Diabetes. 1999; 48: 1300–1307.[Abstract]
  38. Kaikita K, Fogo AB, Ma L, Schoenhard JA, Brown NJ, Vaughan DE. Plasminogen activator inhibitor-1 deficiency prevents hypertension and vascular fibrosis in response to long-term nitric oxide synthase inhibition. Circulation. 2001; 104: 839–844.[Abstract/Free Full Text]
  39. Bochud M, Nussberger J, Bovet P, Maillard MR, Elston RC, Paccaud F, Shamlaye C, Burnier M. Plasma aldosterone is independently associated with the metabolic syndrome. Hypertension. 2006; 48: 239–245.[Abstract/Free Full Text]
  40. Kidambi S, Kotchen JM, Grim CE, Raff H, Mao J, Singh RJ, Kotchen TA. Association of adrenal steroids with hypertension and the metabolic syndrome in blacks. Hypertension. 2007; 49: 704–711.[Abstract/Free Full Text]
  41. Egan BM, Papademetriou V, Wofford M, Calhoun D, Fernandes J, Riehle JE, Nesbitt S, Michelson E, Julius S. Metabolic syndrome and insulin resistance in the TROPHY sub-study: contrasting views in patients with high-normal blood pressure. Am J Hypertens. 2005; 18: 3–12.[Medline] [Order article via Infotrieve]
  42. Goodfriend TL, Egan B, Stepniakowski K, Ball DL. Relationships among plasma aldosterone, high-density lipoprotein cholesterol, and insulin in humans. Hypertension. 1995; 25: 30–36.[Abstract/Free Full Text]
  43. Lind L, Lithell H, Wide L, Ljunghall S. Metabolic cardiovascular risk factors and the renin-aldosterone system in essential hypertension. J Hum Hypertens. 1992; 6: 27–29.[Medline] [Order article via Infotrieve]
  44. Cascella T, Palomba S, Tauchmanova L, Manguso F, Di Biase S, Labella D, Giallauria F, Vigorito C, Colao A, Lombardi G, Orio F. Serum aldosterone concentration and cardiovascular risk in women with polycystic ovarian syndrome. J Clin Endocrinol Metab. 2006; 91: 4395–4400.[Abstract/Free Full Text]
  45. Engeli S, Bohnke J, Gorzelniak K, Janke J, Schling P, Bader M, Luft FC, Sharma AM. Weight loss and the renin-angiotensin-aldosterone system. Hypertension. 2005; 45: 356–362.[Abstract/Free Full Text]
  46. Giacchetti G, Ronconi V, Turchi F, Agostinelli L, Mantero F, Rilli S, Boscaro M. Aldosterone as a key mediator of the cardiometabolic syndrome in primary aldosteronism: an observational study. J Hypertens. 2007; 25: 177–186.[Medline] [Order article via Infotrieve]
  47. Abuissa H, Jones PG, Marso SP, O’Keefe JH Jr. Angiotensin-converting enzyme inhibitors or angiotensin receptor blockers for prevention of type 2 diabetes: a meta-analysis of randomized clinical trials. J Am Coll Cardiol. 2005; 46: 821–826.[Abstract/Free Full Text]
  48. Vasan RS, Evans JC, Larson MG, Wilson PW, Meigs JB, Rifai N, Benjamin EJ, Levy D. Serum aldosterone and the incidence of hypertension in nonhypertensive persons. N Engl J Med. 2004; 351: 33–41.[Abstract/Free Full Text]
  49. Engeli S, Schling P, Gorzelniak K, Boschmann M, Janke J, Ailhaud G, Teboul M, Massiera F, Sharma AM. The adipose-tissue renin-angiotensin-aldosterone system: role in the metabolic syndrome? Int J Biochem Cell Biol. 2003; 35: 807–825.[CrossRef][Medline] [Order article via Infotrieve]
  50. Penfornis P, Viengchareun S, Le Menuet D, Cluzeaud F, Zennaro MC, Lombes M. The mineralocorticoid receptor mediates aldosterone-induced differentiation of T37i cells into brown adipocytes. Am J Physiol Endocrinol Metab. 2000; 279: E386–E394.[Abstract/Free Full Text]
  51. Rondinone CM, Rodbard D, Baker ME. Aldosterone stimulated differentiation of mouse 3T3-L1 cells into adipocytes. Endocrinology. 1993; 132: 2421–2426.[Abstract]
  52. Wang TJ, Gona P, Larson MG, Tofler GH, Levy D, Newton-Cheh C, Jacques PF, Rifai N, Selhub J, Robins SJ, Benjamin EJ, D’Agostino RB, Vasan RS. Multiple biomarkers for the prediction of first major cardiovascular events and death. N Engl J Med. 2006; 355: 2631–2639.[Abstract/Free Full Text]
  53. Wang TJ, Gona P, Larson MG, Levy D, Benjamin EJ, Tofler GH, Jacques PF, Meigs JB, Rifai N, Selhub J, Robins SJ, Newton-Cheh C, Vasan RS. Multiple biomarkers and the risk of incident hypertension. Hypertension. 2007; 49: 432–438.[Abstract/Free Full Text]
  54. Wang TJ, Larson MG, Levy D, Benjamin EJ, Corey D, Leip EP, Vasan RS. Heritability and genetic linkage of plasma natriuretic peptide levels. Circulation. 2003; 108: 13–16.[CrossRef][Medline] [Order article via Infotrieve]
  55. Executive summary of the Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III). JAMA. 2001; 285: 2486–2497.[Free Full Text]
  56. Alberti KG, Zimmet P, Shaw J. Metabolic syndrome: a new world-wide definition: a consensus statement from the International Diabetes Federation. Diabet Med. 2006; 23: 469–480.[CrossRef][Medline] [Order article via Infotrieve]
 

CLINICAL PERSPECTIVE

The metabolic syndrome has been associated with increased risk of cardiovascular disease. In the present study, we evaluated the relative contributions of circulating biomarkers representing distinct biological pathways to the incidence of metabolic syndrome and to longitudinal changes of its components. We measured a panel of 8 circulating biomarkers reflecting inflammation (C-reactive protein), hemostasis (plasminogen activator inhibitor-1, fibrinogen), neurohormonal activity (aldosterone, renin, B-type natriuretic peptide, N-terminal proatrial natriuretic peptide), and endothelial dysfunction (homocysteine) in 2292 Framingham Offspring Study participants. First, we related the biomarker panel to the incidence of the metabolic syndrome. Next, we related biomarkers significantly related to the metabolic syndrome (in the first step) to longitudinal change in the individual components of the metabolic syndrome. Using a backward elimination procedure, plasminogen activator inhibitor-1 and aldosterone were associated with incident metabolic syndrome. Plasminogen activator inhibitor-1 was positively associated with longitudinal changes in fasting glucose, systolic blood pressure, and triglycerides. Aldosterone was associated positively with change in systolic blood pressure and inversely with change in high-density lipoprotein cholesterol. We conclude that higher circulating plasminogen activator inhibitor-1 and aldosterone levels are associated with development of metabolic syndrome and with longitudinal change of its components, suggesting that these biomarkers and related pathways play a key role in mediating metabolic risk.


*    Footnotes
 
*Drs Wang and Vasan contributed equally to this article. Back

Guest Editor for this article was Robert H. Eckel, MD.




This article has been cited by other articles:


Home page
 HypertensionHome page
A. W. Krug and M. Ehrhart-Bornstein
Aldosterone and Metabolic Syndrome: Is Increased Aldosterone in Metabolic Syndrome Patients an Additional Risk Factor?
Hypertension, May 1, 2008; 51(5): 1252 - 1258.
[Full Text] [PDF]

Home page
 J Am Coll CardiolHome page
J. Sanz, P. R. Moreno, and V. Fuster
The year in atherothrombosis.
J. Am. Coll. Cardiol., March 4, 2008; 51(9): 944 - 955.
[Full Text] [PDF]

Home page
 HypertensionHome page
N. J. Brown
Aldosterone and Vascular Inflammation
Hypertension, February 1, 2008; 51(2): 161 - 167.
[Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow CME: Take the course for this article:
 Circulation: August 28, 2007, Volume 116, Number 9
Right arrow All Versions of this Article:
116/9/984    most recent
CIRCULATIONAHA.107.708537v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow Request Permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow