doi: 10.1161/01.CIR.0000100687.99687.CE
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Circulation. 2003;108:2191.)
© 2003 American Heart Association, Inc.
Clinician Update |
Cardiac Biomarkers for Risk Stratification of Patients With Acute Pulmonary Embolism
From the Cardiovascular Division, Brigham and Women’s Hospital, Harvard Medical School, Boston, Mass.
Correspondence to Samuel Z. Goldhaber, MD, Cardiovascular Division, Brigham and Women’s Hospital, Harvard Medical School, 75 Francis St, Boston, MA 02115. E-mail sgoldhaber{at}partners.org
 |
Introduction |
|---|
 Top Introduction Pathophysiology of Cardiac...Accuracy of Cardiac Biomarkers...Incorporation of Cardiac...Future Research Perspectives on...References |
|---|
Patients with pulmonary embolism (PE) present with a wide spectrum of clinical acuity that necessitates different therapeutic strategies. Most patients maintain normal systolic arterial pressure and normal right ventricular function. With therapeutic levels of anticoagulation, they will likely have a benign clinical course. Unfortunately, some PE patients suffer rapid clinical deterioration with adverse clinical events, including death from right ventricular failure or the need for cardiopulmonary resuscitation, mechanical ventilation, or administration of pressors for systolic arterial hypotension. Selected patients will benefit from thrombolysis or surgical embolectomy in addition to anticoagulation.1,2
Severe dyspnea, cyanosis, and syncope indicate life-threatening PE. The clinical examination may reveal signs of acute right ventricular dysfunction, including tachycardia, a low arterial blood pressure, distended neck veins, an accentuated P 2, or a tricuspid regurgitation murmur. On the ECG, T-wave inversion or a pseudoinfarction pattern (Qr) in the anterior precordial leads indicates right ventricular dilation and dysfunction.3 Chest computed tomography or magnetic resonance imaging may not only confirm PE but also demonstrate right ventricular dilation.
Echocardiography has emerged as the principal tool for risk stratification in acute PE. From a prognostic point of view, echocardiography helps to classify patients with PE into 3 groups: Low-risk PE (no right ventricular dysfunction), with a hospital mortality of <4%, submassive PE (right ventricular dysfunction and a preserved arterial pressure), with a hospital mortality of 5% to 10%, and massive PE (right ventricular dysfunction and cardiogenic shock), with a hospital mortality of approximately 30%.4 Right ventricular dysfunction on the echocardiogram is an independent and powerful predictor of early death in patients with acute PE.5 Indirect signs are systolic pulmonary artery hypertension manifested by an increased tricuspid regurgitant velocity >2.6 m/sec and reduced inspiratory collapse of a dilated inferior vena cava due to elevated central venous pressure.6
The major drawbacks of echocardiography are its limited round-the-clock availability and its cost. Another problem is occasional poor imaging quality of the right ventricle, particularly in patients with obesity or chronic lung disease. This latter problem can be circumvented by transesophageal echocardiography, an imaging modality requiring even more specialized skills, expense, and procedure-related risk.
Cardiac biomarkers, including troponins and natriuretic peptides, have emerged as promising tools for risk assessment of patients with acute PE. We will summarize results of contemporary trials on cardiac biomarkers for risk stratification, and we will provide practical recommendations on how to incorporate biomarker test results into the management strategy of patients with acute PE.
|
Pathophysiology of Cardiac Biomarker Elevation in Pulmonary Embolism |
|---|
|
TopIntroductionPathophysiology of Cardiac...Accuracy of Cardiac Biomarkers...Incorporation of Cardiac...Future Research Perspectives on...References |
|---|
Troponins
Cardiac troponins are the most sensitive and specific biomarkers of myocardial cell damage, reflecting microscopic myocardial necrosis.7 Elevated troponin levels predict adverse outcomes in patients with acute myocardial infarction and in critically ill patients without acute coronary syndromes.8
Troponin is a regulatory protein of the thin filament of striated muscle, and consists of three subunits: C at 18 kDa, I at 21 kDa, and T at 37 kDa. The concentration of the troponin I and T subunits is increased in blood for many days after myocardial infarction, because release from the structural elements requires degradation of the myofibril itself.9 Elevations of troponin levels in PE patients are mild and of short duration compared with elevations in patients with acute coronary syndromes.10 In acute PE, troponin levels correlate well with the extent of right ventricular dysfunction.11–14 Some PE patients have initially negative troponin test results but may show a release of troponin 6 to 12 hours later.10,11,13 Myocardial ischemia and micro-infarction due to alterations in oxygen supply and demand of the failing right ventricle probably play a major role in the pathogenesis of troponin release (Figure 1). Release of troponin can occur in patients with PE in the absence of angiographic coronary artery disease.12 An abrupt increase in right ventricular wall tension with compression of the right coronary artery and direct myocardial micro-injury is a possible explanation.
|
Natriuretic Peptides
The natriuretic peptides are useful diagnostic and prognostic biomarkers for patients with congestive heart failure. In contrast to atrial natriuretic peptide that originates mainly from atrial tissue, brain natriuretic peptide (BNP) is produced to a larger degree from ventricular myocytes. The principal stimulus for BNP synthesis and secretion is cardiomyocyte stretch. BNP is a 32 amino acid peptide hormone first isolated from porcine brain tissue. The human BNP gene is located on chromosome 1. In plasma, the intact 108 amino acid prohormone (proBNP), the biologically active BNP (plasma half life 20 minutes), and the remaining part of the prohormone, N-terminal (NT)-proBNP (76 amino acids, plasma half life 60 to 120 minutes), can be measured by immunoassay.
Prohormones in normal ventricular myocytes are not stored to a significant amount. Thus, it takes several hours for the plasma natriuretic peptide levels to increase significantly after the onset of acute myocardial stretch.15 This process includes myocardial BNP messenger ribonucleic acid (mRNA) synthesis, prohormone synthesis, and prohormone release into the circulation (Figure 1). Similar to cardiac troponins, elevations in BNP and NT-proBNP are associated with right ventricular dysfunction in acute PE.16–18 Natriuretic peptide levels are also increased in patients with right ventricular pressure overload due to causes other than PE, including primary pulmonary hypertension, chronic thromboembolic pulmonary hypertension, and chronic lung disease.19–22
|
Accuracy of Cardiac Biomarkers for the Prediction of Adverse Clinical Outcomes |
|---|
|
TopIntroductionPathophysiology of Cardiac...Accuracy of Cardiac Biomarkers...Incorporation of Cardiac...Future Research Perspectives on...References |
|---|
Troponins and natriuretic peptides are accurate in identifying low-risk PE patients and have a high negative predictive value for in-hospital death (Table). The cut-off levels for troponins are the lower detection limits for myocardial ischemia reported by the manufacturer. However, the cut-off level for the BNP Triage assay to predict a benign clinical outcome in PE patients is lower (<50 pg/mL) than the congestive heart failure cut-off level of 90 pg/mL.
|
At this time, none of the biomarkers have been proven superior over others. Keeping in mind that troponin and BNP release into the circulation may take several hours after the onset of myocardial injury, a second biomarker test 6 to 12 hours after an initially negative test should be obtained in a PE patient with a symptom duration <6 hours.17 Similar to troponin assays, bedside point-of-care BNP assays, using whole blood or plasma samples, have precision, analytical sensitivity, stability, and a rapid turnaround time.23
|
Incorporation of Cardiac Biomarkers Into Risk Stratification |
|---|
|
TopIntroductionPathophysiology of Cardiac...Accuracy of Cardiac Biomarkers...Incorporation of Cardiac...Future Research Perspectives on...References |
|---|
The principal role for cardiac biomarkers in PE patients is to differentiate between low risk and intermediate risk (Figure 2). Biomarker tests are not necessary in patients with obvious PE-related shock. According to the European Pulmonary Embolism Task Force Guidelines4 and the results from the largest PE thrombolysis study (Management Strategy and Prognosis in Pulmonary Embolism Trial-3),1 fibrinolysis should be considered not only for patients with PE-related shock but also for patients with a preserved arterial pressure and evidence of right ventricular dysfunction in the absence of an increased bleeding risk. In hemodynamically stable PE patients with increased troponin and/or BNP levels, further risk stratification with echocardiography should be undertaken. Elevated cardiac biomarkers have not yet been incorporated into formal guidelines for treatment decisions (anticoagulation alone versus thrombolysis versus embolectomy) regarding PE patients. In patients with normal biomarker levels, echocardiography need not ordinarily be ordered because right ventricular function will almost always be normal.
|
|
Future Research Perspectives on Cardiac Biomarkers |
|---|
|
TopIntroductionPathophysiology of Cardiac...Accuracy of Cardiac Biomarkers...Incorporation of Cardiac...Future Research Perspectives on...References |
|---|
An elevated biomarker level in combination with echocardiographic right ventricular dysfunction may help to identify a subgroup of hemodynamically stable patients at especially high risk of adverse clinical events. Future research will investigate whether the natriuretic peptides have incremental prognostic information in the presence of troponin release. Because of the short half-life of the natriuretic peptides, particularly BNP, these biomarkers may also be helpful in serial monitoring and gauging the success of different PE treatment regimens.
In conclusion, low cardiac troponin and natriuretic peptide values identify low-risk patients through their high negative predictive value for adverse outcomes. In PE patients with normal biomarker levels, echocardiography may not be necessary. In hemodynamically stable PE patients with increased cardiac troponin or BNP levels, however, right ventricular dysfunction should be confirmed by echocardiography. Finally, the prognostic implications of elevated biomarkers and abnormal echocardiography require further investigation.24–26
|
References |
|---|
|
TopIntroductionPathophysiology of Cardiac...Accuracy of Cardiac Biomarkers...Incorporation of Cardiac...Future Research Perspectives on...References |
|---|
1. Konstantinides S, Geibel A, Heusel G, et al. Heparin plus alteplase compared with heparin alone in patients with submassive pulmonary embolism. N Engl J Med. 2002; 347: 1143–1150.
2. Grifoni S, Olivotto I, Cecchini P, et al. Short-term clinical outcome of patients with acute pulmonary embolism, normal blood pressure, and echocardiographic right ventricular dysfunction. Circulation. 2000; 101: 2817–2822.
3. Kucher N, Walpoth N, Wustmann K, et al. QR in V1: an ECG sign associated with right ventricular strain and adverse clinical outcome in pulmonary embolism. Eur Heart J. 2003; 24: 1113–1119.
4. Guidelines on diagnosis and management of acute pulmonary embolism. Task Force on Pulmonary Embolism, European Society of Cardiology. Eur Heart J. 2000; 21: 1301–1336.
5. Goldhaber SZ, Visani L, De Rosa M. Acute pulmonary embolism: clinical outcomes in the International Cooperative Pulmonary Embolism Registry (ICOPER). Lancet. 1999; 353: 1386–1389.[CrossRef][Medline] [Order article via Infotrieve]
6. Goldhaber SZ. Echocardiography in the management of pulmonary embolism. Ann Intern Med. 2002; 136: 691–700.
7. Alpert JS, Thygesen K, Antman E, et al. Myocardial infarction redefined: a consensus document of the Joint European Society of Cardiology/American College of Cardiology Committee for the redefinition of myocardial infarction. J Am Coll Cardiol. 2000; 36: 959–969.
8. Ammann P, Maggiorini M, Bertel O, et al. Troponin as a risk factor for mortality in critically ill patients without acute coronary syndromes. J Am Coll Cardiol. 2003; 41: 2004–2009.
9. Wu AH, Feng YJ, Moore R, et al. Characterization of cardiac troponin subunit release into serum after acute myocardial infarction and comparison of assays for troponin T and I. American Association for Clinical Chemistry Subcommittee on cTnI Standardization. Clin Chem. 1998; 44: 1198–1208.
10. Muller-Bardorff M, Weidtmann B, Giannitsis E, et al. Release kinetics of cardiac troponin T in survivors of confirmed severe pulmonary embolism. Clin Chem. 2002; 48: 673–675.
11. Konstantinides S, Geibel A, Olschewski M, et al. Importance of cardiac troponins I and T in risk stratification of patients with acute pulmonary embolism. Circulation. 2002; 106: 1263–1268.
12. Giannitsis E, Muller-Bardorff M, Kurowski V, et al. Independent prognostic value of cardiac troponin T in patients with confirmed pulmonary embolism. Circulation. 2000; 102: 211–217.
13. Pruszczyk P, Bochowicz A, Torbicki A, et al. Cardiac troponin T monitoring identifies high-risk group of normotensive patients with acute pulmonary embolism. Chest. 2003; 123: 1947–1952.
14. Meyer T, Binder L, Hruska N, et al. Cardiac troponin I elevation in acute pulmonary embolism is associated with right ventricular dysfunction. J Am Coll Cardiol. 2000; 36: 1632–1636.
15. Hama N, Itoh H, Shirakami G, et al. Rapid ventricular induction of brain natriuretic peptide gene expression in experimental acute myocardial infarction. Circulation. 1995; 92: 1558–1564.
16. Tulevski II, Hirsch A, Sanson BJ, et al. Increased brain natriuretic peptide as a marker for right ventricular dysfunction in acute pulmonary embolism. Thromb Haemost. 2001; 86: 1193–1196.[Medline] [Order article via Infotrieve]
17. Kucher N, Printzen G, Goldhaber SZ. Prognostic role of brain natriuretic peptide in acute pulmonary embolism. Circulation. 2003; 107: 2545–2547.
18. Kucher N, Printzen G, Doernhoefer T, et al. Low pro-brain natriuretic peptide levels predict benign clinical outcome in acute pulmonary embolism. Circulation. 2003; 107: 1576–1578.
19. Nagaya N, Nishikimi T, Okano Y, et al. Plasma brain natriuretic peptide levels increase in proportion to the extent of right ventricular dysfunction in pulmonary hypertension. J Am Coll Cardiol. 1998; 31: 202–208.
20. Nagaya N, Nishikimi T, Uematsu M, et al. Plasma brain natriuretic peptide as a prognostic indicator in patients with primary pulmonary hypertension. Circulation. 2000; 102: 865–870.
21. Bando M, Ishii Y, Sugiyama Y, et al. Elevated plasma brain natriuretic peptide levels in chronic respiratory failure with cor pulmonale. Respir Med. 1999; 93: 507–514.[CrossRef][Medline] [Order article via Infotrieve]
22. Tulevski II, Groenink M, van Der Wall EE, et al. Increased brain and atrial natriuretic peptides in patients with chronic right ventricular pressure overload: correlation between plasma neurohormones and right ventricular dysfunction. Heart. 2001; 86: 27–30.
23. Maisel AS, McCord J, Nowak RM, et al. Bedside B-Type natriuretic peptide in the emergency diagnosis of heart failure with reduced or preserved ejection fraction: results from the Breathing Not Properly multinational study. J Am Coll Cardiol. 2003; 41: 2010–2017.
24. Janata K, Holzer M, Laggner AN, et al. Cardiac troponin T in the severity assessment of patients with pulmonary embolism: cohort study. BMJ. 2003; 326: 312–313.
25. ten Wolde M, Tulevski II, Mulder JW, et al. Brain natriuretic peptide as a predictor of adverse outcome in patients with pulmonary embolism. Circulation. 2003; 107: 2082–2084.
26. Pruszczyk P, Kostrubiec M, Bochowicz A, et al. N-terminal pro-brain natriuretic peptide in patients with acute pulmonary embolism. Eur Resp J. 2003; 22: 649–653.
This article has been cited by other articles:
 |
|
 |
|
  D. Jimenez, F. Uresandi, R. Otero, J. L. Lobo, M. Monreal, D. Marti, J. Zamora, A. Muriel, D. Aujesky, and R. D. Yusen Troponin-Based Risk Stratification of Patients With Acute Nonmassive Pulmonary Embolism: Systematic Review and Metaanalysis Chest, October 1, 2009; 136(4): 974 - 982. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. L. Todd and V. F. Tapson Thrombolytic Therapy for Acute Pulmonary Embolism: A Critical Appraisal Chest, May 1, 2009; 135(5): 1321 - 1329. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. D. Schmitto, H. Doerge, H. Post, M. Coulibaly, C. Sellin, A. F. Popov, S. Sossalla, and F. A. Schoendube Progressive right ventricular failure is not explained by myocardial ischemia in a pig model of right ventricular pressure overload Eur. J. Cardiothorac. Surg., February 1, 2009; 35(2): 229 - 234. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Lippi, G. Targher, and S. Z. Goldhaber Natriuretic Peptides for Assessing the Prognosis of Acute Pulmonary Embolism Chest, June 1, 2008; 133(6): 1531 - 1532. [Full Text] [PDF]
|
|
|
|
 |
|
 M. T. Lu, T. Cai, H. Ersoy, A. G. Whitmore, R. Quiroz, S. Z. Goldhaber, and F. J. Rybicki Interval Increase in Right-Left Ventricular Diameter Ratios at CT as a Predictor of 30-day Mortality after Acute Pulmonary Embolism: Initial Experience Radiology, January 1, 2008; 246(1): 281 - 287. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 V. Palmieri, E. A. Palmieri, and A. Celentano Functional limitation and right ventricular dysfunction at 6-month follow-up in patients with non-massive pulmonary embolism: useful outcomes for testing therapy of acute submassive pulmonary embolism? Eur. Heart J., October 2, 2007; 28(20): 2430 - 2431. [Full Text] [PDF]
|
|
|
|
|
|
L. K. Moores There's No Place Like Home Chest, July 1, 2007; 132(1): 7 - 8. [Full Text] [PDF]
|
|
|
|
 |
|
 G. Piazza and S. Z. Goldhaber Acute Pulmonary Embolism: Part II: Treatment and Prophylaxis Circulation, July 18, 2006; 114(3): e42 - e47. [Full Text] [PDF]
|
|
|
|
|
|
M. Maeder, T. Fehr, H. Rickli, and P. Ammann Sepsis-Associated Myocardial Dysfunction: Diagnostic and Prognostic Impact of Cardiac Troponins and Natriuretic Peptides Chest, May 1, 2006; 129(5): 1349 - 1366. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. Ghaye, A. Ghuysen, P.-J. Bruyere, V. D'Orio, and R. F. Dondelinger Can CT Pulmonary Angiography Allow Assessment of Severity and Prognosis in Patients Presenting with Pulmonary Embolism? What the Radiologist Needs to Know RadioGraphics, January 1, 2006; 26(1): 23 - 39. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Babuin and A. S. Jaffe Troponin: the biomarker of choice for the detection of cardiac injury Can. Med. Assoc. J., November 8, 2005; 173(10): 1191 - 1202. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. Kucher, R. Quiroz, S. McKean, A. A Sasahara, and S. Z Goldhaber Extended enoxaparin monotherapy for acute symptomatic pulmonary embolism Vascular Medicine, November 1, 2005; 10(4): 251 - 256. [Abstract] [PDF]
|
|
|
|
 |
|
 S. Z. Goldhaber Thrombolytic Therapy for Patients With Pulmonary Embolism Who Are Hemodynamically Stable But Have Right Ventricular Dysfunction: Pro Arch Intern Med, October 24, 2005; 165(19): 2197 - 2199. [Full Text] [PDF]
|
|
|
|
|
|
G. Thabut and D. Logeart Thrombolysis for Pulmonary Embolism in Patients With Right Ventricular Dysfunction: Con Arch Intern Med, October 24, 2005; 165(19): 2200 - 2203. [Full Text] [PDF]
|
|
|
|
|
|
M. Kostrubiec, P. Pruszczyk, A. Bochowicz, R. Pacho, M. Szulc, A. Kaczynska, G. Styczynski, A. Kuch-Wocial, P. Abramczyk, Z. Bartoszewicz, et al. Biomarker-based risk assessment model in acute pulmonary embolism Eur. Heart J., October 2, 2005; 26(20): 2166 - 2172. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. Giannitsis and H. A. Katus Risk Stratification in Pulmonary Embolism Based on Biomarkers and Echocardiography Circulation, September 13, 2005; 112(11): 1520 - 1521. [Full Text] [PDF]
|
|
|
|
|
|
L. Binder, B. Pieske, M. Olschewski, A. Geibel, B. Klostermann, C. Reiner, and S. Konstantinides N-Terminal Pro-Brain Natriuretic Peptide or Troponin Testing Followed by Echocardiography for Risk Stratification of Acute Pulmonary Embolism Circulation, September 13, 2005; 112(11): 1573 - 1579. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Piazza and S. Z. Goldhaber The Acutely Decompensated Right Ventricle: Pathways for Diagnosis and Management Chest, September 1, 2005; 128(3): 1836 - 1852. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. Kucher, E. Rossi, M. De Rosa, and S. Z. Goldhaber Prognostic Role of Echocardiography Among Patients With Acute Pulmonary Embolism and a Systolic Arterial Pressure of 90 mm Hg or Higher Arch Intern Med, August 8, 2005; 165(15): 1777 - 1781. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. Kucher and S. Z. Goldhaber Management of Massive Pulmonary Embolism Circulation, July 12, 2005; 112(2): e28 - e32. [Full Text] [PDF]
|
|
|
|
 |
|
 A. Geibel, M. Zehender, W. Kasper, M. Olschewski, C. Klima, and S. V. Konstantinides Prognostic value of the ECG on admission in patients with acute major pulmonary embolism Eur. Respir. J., May 1, 2005; 25(5): 843 - 848. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Leacche, D. Unic, S. Z. Goldhaber, J. D. Rawn, S. F. Aranki, G. S. Couper, T. Mihaljevic, R. J. Rizzo, L. H. Cohn, L. Aklog, et al. Modern surgical treatment of massive pulmonary embolism: Results in 47 consecutive patients after rapid diagnosis and aggressive surgical approach J. Thorac. Cardiovasc. Surg., May 1, 2005; 129(5): 1018 - 1023. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
U. J. Schoepf, N. Kucher, F. Kipfmueller, R. Quiroz, P. Costello, and S. Z. Goldhaber Right Ventricular Enlargement on Chest Computed Tomography: A Predictor of Early Death in Acute Pulmonary Embolism Circulation, November 16, 2004; 110(20): 3276 - 3280. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Macrea, P. Pruszczyk, and A. Torbicki Cardiac Troponin T Monitoring and Acute Pulmonary Embolism Chest, August 1, 2004; 126(2): 655 - 656. [Full Text] [PDF]
|
|
|
|
 |
|
 M.R. Loebinger and J.C. Bradley Thrombolysis in pulmonary embolism: are we under-using it? QJM, June 1, 2004; 97(6): 361 - 364. [Full Text] [PDF]
|
|
|
|
 |
|
 S. Z. Goldhaber, E. S. Nadel, M. E. King, and A. Sharma Case 17-2004 - A 42-Year-Old Woman with Cardiac Arrest Several Weeks after an Ankle Fracture N. Engl. J. Med., May 27, 2004; 350(22): 2281 - 2290. [Full Text] [PDF]
|
|
|
|
|
|
R. Quiroz, N. Kucher, U. J. Schoepf, F. Kipfmueller, S. D. Solomon, P. Costello, and S. Z. Goldhaber Right Ventricular Enlargement on Chest Computed Tomography: Prognostic Role in Acute Pulmonary Embolism Circulation, May 25, 2004; 109(20): 2401 - 2404. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. Bochowicz, M. Kostrubiec, and P. Pruszczyk Serum Myoglobin in Pulmonary Embolism Circulation, April 27, 2004; 109(16): e194 - e194. [Full Text] [PDF]
|
|
|
|
|
|
S. Z. Goldhaber and C. G. Elliott Acute Pulmonary Embolism: Part II: Risk Stratification, Treatment, and Prevention Circulation, December 9, 2003; 108(23): 2834 - 2838. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||