doi: 10.1161/01.CIR.0000035924.70846.20
(Circulation. 2002;106:2284.)
© 2002 American Heart Association, Inc.
Current Perspective |
Clinical Implications of Percutaneous Coronary Intervention-Clopidogrel in Unstable angina to prevent Recurrent Events (PCI-CURE) Study
A US Perspective
From the Division of Cardiovascular Medicine, Mayo Clinic, Rochester, Minn (P.B.B.); and the Division of Cardiology, University of North Carolina, Chapel Hill (S.S.).
Correspondence to Peter B. Berger, MD, Division of Cardiovascular Diseases-West 16, Mayo Clinic, 200 First St S.W., Rochester, MN 55905. E-mail berger.peter{at}mayo.edu
 |
Introduction |
|---|
 Top Introduction Applicability to US PracticePretreatment With a ClopidogrelDuration of Therapy With...Remaining Unanswered QuestionsOngoing TrialsConclusionReferences |
|---|
The long-term use of both aspirin and clopidogrel for patients with an acute coronary syndrome was analyzed in the Clopidogrel in Unstable angina to prevent Recurrent Events (CURE) study.1 In that trial, 12 562 patients with an acute coronary syndrome were randomly assigned to receive aspirin and clopidogrel or aspirin and placebo for a period of no more than 1 year. The results of the trial after a mean duration of follow-up of 9 months revealed an approximately 20% relative reduction in the risk of vascular death, myocardial infarction, and stroke. There were 2658 patients (21%) who underwent percutaneous coronary intervention (PCI) in the CURE trial a median of 10 days after enrollment. This was decided not by study protocol but by physician preference, which was influenced by the therapy to which patients were randomized; among these patients, the reduction in events (cardiovascular death, myocardial infarction [MI], or an urgent repeat revascularization procedure) from the PCI procedure through 30 days was an absolute1.9% (6.4% versus 4.5%), which was a relative reduction of 30% (P=0.03).2 From the time of PCI through the remainder of follow-up, there was an absolute reduction in cardiovascular (CV) death and MI of 2% (8.0% versus 6.0%), which was a relative risk reduction of 25% (P=0.047).
Although dual antiplatelet therapy with clopidogrel and aspirin is often considered to be of benefit only in patients treated with stents, an interesting finding in PCI-CURE was that pretreatment and continued therapy with clopidogrel appeared to be as beneficial among the 19% of patients who did not receive a coronary stent as among the 81% of patients who did.2 There had not previously been evidence of benefit from thienopyridines use among patients undergoing PCI without stent placement, although it seems logical that there might be.3
|
Applicability to US Practice |
|---|
|
TopIntroductionApplicability to US PracticePretreatment With a ClopidogrelDuration of Therapy With...Remaining Unanswered QuestionsOngoing TrialsConclusionReferences |
|---|
There are several features of the analysis of PCI in the CURE trial that make extrapolation to the general practice of PCI in the United States difficult. First, the study prohibited medical centers from participation, including a great many in the United States that performed PCI in the majority of patients with an acute coronary syndrome.2,4 Why such centers were excluded was not specified, but regardless of the reason, that clearly makes the application of the results of the PCI-CURE trial to medical centers which do perform PCI in the majority of the cases problematic. Second, the use of platelet glycoprotein IIb/IIIa inhibitors in the 72 hours before study entry was prohibited.2,4 Because such drugs are far more frequently used in the United States, that exclusion criteria also poses a difficultly in applying the results of the PCI-CURE trial more generally. Another difference between the CURE study and United States practice patterns is the fact that more than 70% of patients undergoing PCI in the United States receive a platelet glycoprotein (GP) IIb/IIIa inhibitor at the time of the procedure, and this percentage is even greater in acute coronary syndrome (ACS) patients; yet, the use of these drugs during PCI in the CURE trial was uncommon (23.7%).2
In view of these issues, what findings from the PCI-CURE analysis can be used to help guide patient treatment? The study attempts to address 2 issues that physicians must deal with in every patient undergoing PCI in their practice: when a thienopyridine should be started and how long it should be continued after the procedure.
|
Pretreatment With a Clopidogrel |
|---|
|
TopIntroductionApplicability to US PracticePretreatment With a ClopidogrelDuration of Therapy With...Remaining Unanswered QuestionsOngoing TrialsConclusionReferences |
|---|
The beneficial effects of pretreatment with a thienopyridine before PCI have been reported in several studies, although none were randomized comparisons.5–7 In the PCI-CURE analysis, all patients in the aspirin and clopidogrel arm had been pretreated with clopidogrel for a median of 10 days, whereas the majority of patients in the aspirin and placebo arm received no pretreatment, although 25% did receive an open-label thienopyridine before PCI. The frequency of death or MI in the 30 days after PCI was significantly less among patients who had been pretreated than in those who had not (4.4% versus 2.8%, a relative risk reduction of 34%; P=0.04). Importantly, unlike all recent PCI trials of GPIIb/IIIa antagonists, systematic monitoring of post-PCI myocardial enzymes was not performed in the CURE study, which likely led to an underestimation of the actual number of myocardial infarctions and potentially the magnitude of pretreatment benefit.
A critically important potential confounder of the PCI-CURE results is that there were important differences between the 2 groups of patients — not at study entry, but at the time the PCI procedure was performed; in fact, the differences were apparent within several hours of randomization (Figure). Because the event curves for clopidogrel and placebo-treated patients in the CURE trial separated within hours and continued to separate throughout the study, a significantly greater proportion of patients had suffered an adverse event in the placebo arm even before the PCI procedure was performed (MI or refractory ischemia before PCI: 15.3% placebo versus 12.1% clopidogrel, P=0.008). This adds bias to the analysis in favor of dual antiplatelet therapy. However, it is unlikely that this difference accounted entirely for the relatively large difference in events observed in the following 30 days between the 2 groups of patients.
|
Yet another source of bias in the PCI-CURE substudy derives from the fact that 25% of patients assigned to placebo received open-label clopidogrel before their PCI procedure, so that if pretreatment with clopidogrel is beneficial, this study design would serve to underestimate the true benefits of clopidogrel therapy. All these reasons make it difficult to be definitive about any conclusions regarding the actual benefit of pretreatment with clopidogrel one might reach from the study.
|
Duration of Therapy With Clopidogrel |
|---|
|
TopIntroductionApplicability to US PracticePretreatment With a ClopidogrelDuration of Therapy With...Remaining Unanswered QuestionsOngoing TrialsConclusionReferences |
|---|
The other major issues addressed regarding the most appropriate duration of therapy with clopidogrel after the procedure are slightly less subject to bias. Because no patient in the aspirin alone arm received clopidogrel for more than 30 days after PCI and all patients in the aspirin and clopidogrel arm did receive it for the duration of the study (a median of 8 additional months), analysis of the frequency of adverse events during this time period can shed light on the most appropriate duration of clopidogrel after PCI. After 30 days and until the end of follow-up, continued clopidogrel treatment was associated with a 21% relative reduction in the frequency of cardiovascular death and MI in the 2 groups compared with placebo, representing an absolute 0.8% fewer deaths or MIs (3.1% versus 3.9%, P=NS). Thus, 125 patients must be treated to reduce the frequency of this end point, which was primarily driven by nonfatal MI in 1 patient. When one considers that the difference was driven entirely by a difference in MI rather than death, the results are not overwhelming and significantly impact cost-effectiveness.8 This finding could have been anticipated by prior studies, which revealed very low event rates in relatively well patients who undergo a successful PCI procedure and who are treated with a thienopyridine for even just 2 weeks, and certainly in those treated for 4 weeks.9–12 However, there are clearly subsets of patients who are at a markedly increased risk of thrombotic events long-term after a PCI, and often at sites in the vascular bed other than the treated site; we believe such patients are more likely to benefit from long-term dual antiplatelet therapy than the general population of PCI patients. Such patients include those with diabetes mellitus and chronic renal failure, patients with multivessel disease and particularly diffuse disease who have a relatively large atherosclerotic burden even after a successful PCI procedure treated the culprit vessel, patients with old vein grafts, patients with multiple events while on aspirin, and those peripheral and cerebrovascular disease. Such patients are at markedly increased risk compared with patients without these characteristics and are, we believe, much more likely to derive the greatest absolute benefit from prolonged antiplatelet therapy than patients without these characteristics.
|
Remaining Unanswered Questions |
|---|
|
TopIntroductionApplicability to US PracticePretreatment With a ClopidogrelDuration of Therapy With...Remaining Unanswered QuestionsOngoing TrialsConclusionReferences |
|---|
Procedure-Related Bleeding Risk
The PCI-CURE analysis identified no significant increase in major or minor bleeding associated with clopidogrel pretreatment in patients who underwent a percutaneous revascularization, even in the minority of patients also treated with a GPIIb/IIIa antagonist. However, when clopidogrel is started, it is typically unknown what method of revascularization will be best for the individual patient. In fact, on the basis of the results of recent randomized trials, the proportion of ACS patients who undergo early angiography and subsequently require a surgical revascularization is not trivial. In the Treat angina with Aggrastat and determine Costs of Therapy with Invasive or Conservative Strategies (TACTICS-TIMI 18) and FRagmin during InStability in Coronary artery disease (FRISC II) trials, in patient populations similar to that in CURE but in whom early angiography was allowed, 32% to 45% of patients who were revascularized underwent CABG as their primary mode of therapy.13,14 Therefore, a remaining important issue is the problem associated with the early treatment of ACS patients with clopidogrel before diagnostic coronary angiography is performed. Several previous observational studies have suggested that if a patient undergoes bypass surgery while on therapy with a thienopyridine and aspirin, the risk of major complications and the need for blood products is significantly increased. In 2 single-center observational studies,15,16 the risk of reoperation for bleeding was reported to be 5- to 10-fold higher in patients on clopidogrel and aspirin. However, even though reported observational studies have tried to adjust for differences between patients that went to surgery while on therapy, there is undoubtedly bias in such analyses. The best data about the risk of coronary artery bypass grafting (CABG) while on a thienopyridine are from the CURE trial itself.1 In the CURE trial, patients who received clopidogrel any time within the 5 days before surgery had a relative 50% increase in the risk of major or life threatening bleeding, which is an absolute increase of 3%. It is entirely likely that if the therapy were continued until the day of or prior to surgery, the risk would have been higher. American medical centers whose practice patterns are such that patients undergo coronary angiography very soon after admission and, if coronary angiography reveals surgical anatomy, send their patients to surgery within a day or two, will not have the opportunity to derive the same benefit from a longer duration of pretreatment and will likely run an even higher risk of bleeding than the 50% increase in risk of major or life threatening bleeding seen in the CURE study.
Therefore, a reasonable approach to minimizing bleeding risks or treatment delays, an approach we suggest might be to withhold clopidogrel pretreatment before diagnostic angiography in patients at increased risk of requiring CABG, particularly if angiography will be performed after minimal delay. Such patients would include those with known multivessel disease shown by prior angiograms (particularly if they were not to be amenable to PCI but were amenable to surgery), those with known borderline left main disease, and those with diabetes mellitus or with chronic renal failure; these are patient groups in whom PCI appears to be associated with a much higher risk than CABG. Also, patients with old vein grafts and those with peripheral and cerebrovascular disease are much more likely to require CABG than patients without any of these characteristics. Note that the list of patients more likely to require CABG is similar to the list of patients most likely to benefit from long-term treatment with clopidogrel after a successful PCI procedure.
As it is likely that only a small percentage of ACS patients actually require an emergent surgical revascularization, another strategy for minimizing bleeding would be to delay surgery for at least 5 days in patients already receiving clopidogrel. However, as a large proportion of such patients would have to remain hospitalized awaiting surgery, the additional cost of such a strategy may be prohibitive at many institutions.
Use With GPIIb/IIIa Antagonists
Perhaps the most important questions facing the interventional cardiologist regarding clopidogrel pretreatment, which are unanswered by the PCI-CURE trial, are whether pretreatment with clopidogrel remains beneficial if a platelet GPIIb/IIIa inhibitor is administered, or visa versa, and whether the benefit of aspirin plus clopidogrel plus a GPIIb/IIIa antagonist is additive without an unacceptable bleeding risk. The results of the Evaluation of Platelet IIb/IIIa Inhibitor for STENTing (EPISTENT) Trial analysis suggested that pretreatment was no longer beneficial in reducing adverse events at 30 days when the IIb/IIIa inhibitor abciximab was used; however, the duration of pretreatment with the slower-onset ticlopidine was unknown and likely short.6 On the other hand, the results of the do Tirofiban (Aggrastat) And ReoPro Give similar Efficacy outcomes Trial (TARGET) suggested that event rates were lower among patients pretreated with clopidogrel even when all patients received a GPIIb/IIIa antagonist.7 However, both the EPISTENT and TARGET trials were not designed to answer the question about the benefit of pretreatment with a thienopyridine among patients receiving a GPIIb/IIIa inhibitor, and therefore the optimal use of a GPIIb/IIIa antagonist when a patient has already received clopidogrel, or of clopidogrel in a patient already receiving a GPIIb/IIIa antagonist, remains unclear.
|
Ongoing Trials |
|---|
|
TopIntroductionApplicability to US PracticePretreatment With a ClopidogrelDuration of Therapy With...Remaining Unanswered QuestionsOngoing TrialsConclusionReferences |
|---|
The Clopidogrel for Reduction of Events During Observation (CREDO) trial is directly designed to answer many of the above questions, avoiding many of the limitations from which the PCI-CURE trial suffers. This trial is designed to determine whether more complete platelet inhibition with a thienopyridine at the time of a percutaneous revascularization procedure is beneficial, as well as the most effective duration of clopidogrel treatment in patients after a percutaneous revascularization. In the CREDO trial, more than 2100 patients in whom a percutaneous revascularization (with or without a stent) was planned were randomly assigned to receive a 300 mg loading dose of clopidogrel 3 to 24 hours before the procedure or a 75 mg/d dose of clopidogrel (which is equivalent in speed of action to the old most-studied regimen of a ticlopidine 500 mg loading dose) immediately before the procedure. Importantly, 400 patients were prespecified at the time of randomization to receive abciximab during the PCI procedure. Patients assigned to the 300 mg loading dose received 75 mg/d of clopidogrel for 1 year with aspirin. Patients assigned to clopidogrel without a loading dose received 75 mg/d of clopidogrel for only 4 weeks, followed by placebo with aspirin for 11 months. The results of the trial will be announced this year and will help further define the role of combination antiplatelet therapy in the PCI population.
In the Intracoronary Stenting and Antithrombotic Regimen-Rapid Early Action for Coronary Treatment (ISAR-REACT) study, patients who had symptoms of coronary artery disease and scheduled to undergo coronary angiography and who are extremely unlikely to require CABG within days of angiography will receive a loading dose of 600 mg clopidogrel at least 2 hours before the procedure; those with insulin-dependent diabetes mellitus and those with a positive biomarker are excluded. Patients receiving this large dose of clopidogrel before their procedure will then be randomized to receive either abciximab and reduced dose heparin or standard dose heparin and placebo. The primary end point of the study is the composite rate of death, MI, and urgent target vessel revascularization within 30 days; secondary end points include the incidence of major and minor bleeding complications, as well as indices of restenosis after 6 months.
|
Conclusion |
|---|
|
TopIntroductionApplicability to US PracticePretreatment With a ClopidogrelDuration of Therapy With...Remaining Unanswered QuestionsOngoing TrialsConclusionReferences |
|---|
As a retrospective (although prospectively designed) analysis of a very large randomized ACS trial, the PCI-CURE study provides strong, consistent support for the benefit of clopidogrel pretreatment in patients undergoing a PCI. However, because of the limitations discussed above, its results cannot be considered definitive proof of the degree of benefit, especially when attempting to apply these results to the treatment of ACS patients in the US, where early angiography and GPIIb/IIIa antagonist use is the norm. The development of an integrative strategy for optimizing early antiplatelet therapies in patients undergoing a PCI, with our without and ACS, awaits the results of the CREDO and ISAR-REACT trials. The benefit of continuing clopidogrel and aspirin therapy for the long term in ACS patients does not seem to be diminished if a patient undergoes a PCI, but the cost-effectiveness of such a generalized strategy will continue to be debated. The ability to identify those patients most likely to derive the greatest benefit from chronic dual antiplatelet therapy will be critical for this to be optimized.
|
References |
|---|
|
TopIntroductionApplicability to US PracticePretreatment With a ClopidogrelDuration of Therapy With...Remaining Unanswered QuestionsOngoing TrialsConclusionReferences |
|---|
1. The Clopidogrel in Unstable Angina to Prevent Recurrent Events Trial Investigators. Effects of clopidogrel in addition to aspirin in patients with acute coronary syndromes without ST-segment elevation. N Engl J Med. 2001; 345: 494–502.
2. Mehta S, Yusuf S, Peters R, et al. Effects of pretreatment with clopidogrel and aspirin followed by long-term therapy in patients undergoing percutaneous coronary intervention: the PCI-CURE study. Lancet.;. 2001; 358: 527–533.[CrossRef][Medline] [Order article via Infotrieve]
3. Berger PB, Dzavik V, Penn IM, et al. Does ticlopidine reduce reocclusion and other adverse events after successful balloon angioplasty of occluded coronary arteries? Results from the Total Occlusion Study of Canada. Am Heart J. 2001; 142: 776–781.[CrossRef][Medline] [Order article via Infotrieve]
4. Mehta SR, Yusuf S. The Clopidogrel in Unstable angina to prevent Recurrent Events (CURE) trial programe: rationale, design and baseline characteristics including a meta-analysis of the effects of thienopyridines in vascular disease. Eur Heart J. 2000; 21: 2033–2041.
5. Steinhubl SR, Lauer MS, Mukherjee DP, et al. The duration of pretreatment with ticlopidine prior to stenting is associated with the risk of procedure-related non-Q-wave myocardial infarctions. J Am Coll Cardiol. 1998; 32: 1366–1370.
6. Steinhubl SR, Ellis SG, Wolski K, et al. Ticlopidine pretreatment before coronary stenting is associated with sustained decrease in adverse cardiac events: data from the Evaluation of Platelet IIb/IIIa Inhibitor for STENTing (EPISTENT) Trial. Circulation. 2001; 103: 1403–1409.
7. Topol EJ, Moliterno DJ, Herrmann HC, et al. Comparison of two platelet glycoprotein IIb/IIIa inhibitors, tirofiban and abciximab, for the prevention of ischemic events with percutaneous coronary revascularization. N Engl J Med. 2001; 344: 1888–1894.
8. Gaspoz J-M, Coxson PG, Goldman PA, et al. Cost effectiveness of aspirin, clopidogrel, or both for secondary prevention of coronary heart disease. N Engl J Med. 2002; 346: 1800–1806.
9. Schuhlen H, Kastrati A, Pache J, et al. Sustained benefit over four years from an initial combined antiplatelet regimen after coronary stent placement in the ISAR trial. Intracoronary Stenting and Antithrombotic Regimen. Am J Cardiol. 2001; 87: 397–400.[CrossRef][Medline] [Order article via Infotrieve]
10. Berger PB, Bell MR, Grill DE, et al. Safety and efficacy of ticlopidine for only two weeks after successful intracoronary stent placement. Circulation. 1999; 99: 248–253.
11. Topol EJ, Mark DB, Lincoff AM, et al. Outcomes at 1 year and economic implications of platelet glycoprotein IIb/IIIa blockade in patients undergoing coronary stenting: results from a multicentre randomised trial. Lancet. 1999; 354: 2019–2024.[CrossRef][Medline] [Order article via Infotrieve]
12. O’Shea JC, Buller CE, Cantor WJ, et al. Long-term efficacy of platelet glycoprotein IIb/IIIa integrin blockade with eptifibatide in coronary stent intervention. JAMA. 2002; 287: 618–621.
13. Cannon C, Weintraub W, Demopoulos L, et al. Comparison of early invasive and conservative strategies in patients with unstable coronary syndromes treated with the glycoprotein IIb/IIIa inhibitor tirofiban. N Engl J Med. 2001; 344: 1879–1887.
14. Wallentin L, Lagerqvist B, Husted S, et al. Outcome at 1 year after an invasive compared to a non-invasive strategy in unstable coronary-artery disease: the FRISC II invasive randomised trial. Lancet. 2000; 356: 9–16.[CrossRef][Medline] [Order article via Infotrieve]
15. Yende S, Wunderink RG. Effect of clopidogrel on bleeding after coronary artery bypass surgery. Crit Care Med. 2001; 29: 2271–2275.[CrossRef][Medline] [Order article via Infotrieve]
16. Hongo RH, Ley J, Dick SE, et al. The effect of clopidogrel in combination with aspirin when given before coronary artery bypass grafting. J Am Coll Cardiol. 2002; 40: 231–237.
This article has been cited by other articles:
 |
|
 |
|
  M. S. Sabatine, C. P. Cannon, C. M. Gibson, J. L. Lopez-Sendon, G. Montalescot, P. Theroux, B. S. Lewis, S. A. Murphy, C. H. McCabe, E. Braunwald, et al. Effect of Clopidogrel Pretreatment Before Percutaneous Coronary Intervention in Patients With ST-Elevation Myocardial Infarction Treated With Fibrinolytics: The PCI-CLARITY Study JAMA, September 14, 2005; 294(10): 1224 - 1232. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S J Walsh, M S Spence, D Crossman, and A A J Adgey Clopidogrel in non-ST segment elevation acute coronary syndromes: an overview of the submission by the British Cardiac Society and the Royal College of Physicians of London to the National Institute for Clinical Excellence, and beyond Heart, September 1, 2005; 91(9): 1135 - 1140. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 Authors/Task Force Members, S. Silber, P. Albertsson, F. F. Aviles, P. G. Camici, A. Colombo, C. Hamm, E. Jorgensen, J. Marco, J.-E. Nordrehaug, et al. Guidelines for Percutaneous Coronary Interventions: The Task Force for Percutaneous Coronary Interventions of the European Society of Cardiology Eur. Heart J., April 2, 2005; 26(8): 804 - 847. [Full Text] [PDF]
|
|
|
|
 |
|
 J. M. Brophy and L. Joseph Medical Decision Making with Incomplete Evidence--Choosing a Platelet Glycoprotein IIbIIIa Receptor Inhibitor for Percutaneous Coronary Interventions Med Decis Making, March 1, 2005; 25(2): 222 - 228. [Abstract] [PDF]
|
|
|
|
 |
|
 L. Chen, A. W. Bracey, R. Radovancevic, J. R. Cooper Jr, C. D. Collard, W. K. Vaughn, and N. A. Nussmeier Clopidogrel and bleeding in patients undergoing elective coronary artery bypass grafting J. Thorac. Cardiovasc. Surg., September 1, 2004; 128(3): 425 - 431. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Kastrati, J. Mehilli, H. Schuhlen, J. Dirschinger, F. Dotzer, J. M. ten Berg, F.-J. Neumann, H. Bollwein, C. Volmer, M. Gawaz, et al. A Clinical Trial of Abciximab in Elective Percutaneous Coronary Intervention after Pretreatment with Clopidogrel N. Engl. J. Med., January 15, 2004; 350(3): 232 - 238. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. S. Forrester Toward understanding the evolution of plaque rupture: Correlating vascular pathology with clinical outcomes J. Am. Coll. Cardiol., November 5, 2003; 42(9): 1566 - 1568. [Full Text] [PDF]
|
|
|
|
 |
|
 D. J. Kereiakes Adjunctive Pharmacotherapy before Percutaneous Coronary Intervention in Non-ST-Elevation Acute Coronary Syndromes: The Role of Modulating Inflammation Circulation, October 21, 2003; 108(90161): III-22 - 27. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G. N. Levine, M. J. Kern, P. B. Berger, D. L. Brown, L. W. Klein, D. J. Kereiakes, T. A. Sanborn, A. K. Jacobs, and for the American Heart Association Diagnostic and Management of Patients Undergoing Percutaneous Coronary Revascularization Ann Intern Med, July 15, 2003; 139(2): 123 - 136. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. R. Steinhubl, P. B. Berger, J. T. Mann III, E. T. A. Fry, A. DeLago, C. Wilmer, E. J. Topol, and for the CREDO Investigators Early and Sustained Dual Oral Antiplatelet Therapy Following Percutaneous Coronary Intervention: A Randomized Controlled Trial JAMA, November 20, 2002; 288(19): 2411 - 2420. [Abstract] [Full Text] [PDF]
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||