| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Circulation. 2007;115:1234-1243.)
© 2007 American Heart Association, Inc.
Genetics |
Transgene as a Therapeutic Strategy in No-Option Critical Limb Ischemia PatientsFrom Ohio State University, Section of Vascular Medicine (S.R.), Columbus; Mount Sinai School of Medicine (S.R., J.O.), New York, NY; Cleveland Clinic Foundation (S.D.), Cleveland, Ohio; St Elizabeths Medical Center, Boston, Mass (A.P.); Washington Hospital Center, Washington, DC (J.L.); University of Michigan, Ann Arbor (P.M.G.); Ochsner Clinic Foundation, New Orleans, La (C.K.G.); Genzyme Corp, Cambridge, Mass (K.M., R.K.); and Atlanta Cardiology Research Institute, P.C., Atlanta, Ga (N.C.). Dr Deitcher is currently at Nuvelo Corp, San Carlos, Calif.
Correspondence to Sanjay Rajagopalan, MD, Section of Vascular Medicine, 473 W 12th Ave, Division of Cardiovascular Medicine, Ohio State University, Columbus, OH 43210-1252. E-mail sanjay.rajagopalon{at}osumc.edu
Received December 13, 2005; accepted December 18, 2006.
| Abstract |
|---|
|
|
|---|
(ie, Ad2/HIF-1
/VP16 or HIF-1
) into the lower extremity of patients with critical limb ischemia would be safe and might result in a durable clinical response.
Methods and Results This phase I dose-escalation program included 2 studies: a randomized, double-blind, placebo-controlled study and an open-label extension study. In total, 34 no-option patients with critical limb ischemia received HIF-1
at doses of 1x108 to 2x1011 viral particles. No serious adverse events were attributable to study treatment. Five deaths occurred: 3 in HIF-1
and 2 in placebo patients. In the first (randomized) study, 7 of 21 HIF-1
patients met treatment failure criteria and had major amputations. Three of the 7 placebo patients rolled over to receive HIF-1
in the extension study. No amputations occurred in the 2 highest-dose groups of Ad2/HIF-1
/VP16 (1x1011 and 2x1011 viral particles). The most common adverse events included peripheral edema, disease progression, and peripheral ischemia. At 1 year, limb status observations in HIF-1
patients included complete rest pain resolution in 14 of 32 patients and complete ulcer healing in 5 of 18 patients.
Conclusions HIF-1
therapy in patients with critical limb ischemia was well tolerated, supporting further, larger, randomized efficacy trials.
Key Words: angiogenesis gene therapy growth substances hypoxia peripheral vascular disease
| Introduction |
|---|
|
|
|---|
2% of patients
50 years of age with documented peripheral arterial disease and carries a poor prognosis for life and limb. The 1-year mortality rate in patients with CLI is
25% and may be as high as 45% in those who have undergone amputation.1,2 Limb loss in CLI is a manifestation of advanced systemic atherosclerosis and is a consequence of marked impairment in tissue perfusion in the lower extremities. The current standard of care includes peripheral bypass grafting and percutaneous approaches to improve lower-extremity blood flow; however, a substantial number of patients are ineligible for these treatments or experience short-lived improvements.1 Moreover, results to date from both recombinant protein and gene-based formulations of single growth factors, with several exceptions, have been disappointing.36 One potential explanation is that the use of a single angiogenic cytokine may be insufficient to generate adequate and durable neovascularization. A combination of
2 cytokines, acting by differing mechanisms, may act synergistically to achieve a more robust and durable biological response.7,8 Alternatively, the use of a relevant transcription factor may allow the initiation of a coordinated series of cellular events, all of which may conspire to recapitulate vessel growth and/or physiological events to normalize cellular hypoxia.9 The transcription factor hypoxia-inducible factor-1
(HIF-1
) used in the present study is one such factor that may normalize intracellular oxygen levels by increasing the synthesis of multiple proangiogenic cytokines (eg, vascular endothelial growth factors, angiopoietins, endothelial nitric oxide synthase) and genes that facilitate survival of ischemic tissue (eg, metabolic pathways that favor glucose over fatty acid metabolism). This phase I dose-escalation study represents the first report on the safety of a modified, constitutively active form of HIF-1
(Ad2/HIF-1
/VP16) in patients with advanced atherosclerosis and tissue ischemia.
Editorial p 1180
Clinical Perspective p 1243
| Methods |
|---|
|
|
|---|
Selection Criteria for Inclusion in the Trial
CLI patients between 21 and 85 years of age with no options for surgical or endovascular revascularization and total or subtotal occlusion of at least 1 main artery in a limb confirmed by angiography were recruited to the study from 5 centers in the United States. CLI was defined as Rutherford category (RC) 4 or 5 present for a minimum of 4 weeks without response to conventional therapies, with lack of further revascularization options confirmed by both the investigator and an independent reviewer at the institution. Patients willing and able to discontinue other nonhealing ulcer treatments at least 3 days before treatment and to give written informed consent participated. Exclusion criteria included contraindications to growth factor therapy that have been published previously (eg, history of cancer within 5 years, active diabetic retinopathy),1315 inflammatory arteritides, RC6 status, prior successful lower extremity arterial surgery, angioplasty, or lumbar sympathectomy during the 2 months before screening. Patients who had participated in other experimental protocols within 30 days of enrollment or who had ever been enrolled in a similar vascular endothelial growth factor or fibroblast growth factor adenoviral or plasmid gene therapy protocol were excluded.
Study Design
This phase I program consisted of 2 dose-escalation safety studies: a randomized, double-blind, placebo-controlled (RDBPC) design and an open-label extension (open-label) design. There was no intent to conduct hypothesis-driven comparisons of safety or efficacy between the treatment groups. Figure 1 describes the flow of patients through the studies by dosing cohort. The first study was placebo controlled to ensure objectivity of initial safety evaluations by investigators and an independent Data Safety Monitoring Board. On the basis of results of preclinical safety and bioactivity testing, 5 dosing cohorts were evaluated, increasing from 1x108 to 1x1010 viral particles (vp) in [1/2] log increments. The Data Safety Monitoring Board reviewed safety data from each dosing cohort before treatment of the next-higher-dosing cohort and monitored safety data throughout the study. A total of 28 patients were enrolled in the RDBPC study with a 3:1 ratio of HIF-1
to placebo randomization (ie, 21 patients received HIF-1
and 7 received placebo): the 1x108-, 3x108-, and 3x109-vp groups comprised 4 patients each (3 HIF-1
and 1 placebo), and 6 subjects receiving HIF-1
and 2 receiving placebo were prospectively allotted to the 1x109- and 1x1010-vp groups (in which detectable bioactivity had been predicted on the basis of preclinical studies).
|
In the RDBPC study, patients were designated as having experienced treatment failure by the investigator on the basis of prospectively defined criteria encompassing a blinded assessment of the onset or worsening of symptoms that originally qualified them for the study such as worsening rest pain, delayed ulcer healing, and development of osteomyelitis. Patients adjudicated as treatment failures were unblinded, and if the patient had been receiving placebo and still met the original entry criteria, he/she was eligible to receive the highest dose of HIF-1
deemed safe by the Data Safety Monitoring Board as part of an open-label extension study. Of the 7 HIF-1
treated patients who met treatment failure criteria, 4 proceeded to major amputation, whereas 3 placebo patients met treatment failure criteria in the RDBPC trial and rolled over to receive active HIF-1
. (Another 3 HIF-1
treated patients underwent amputation before being classified treatment failures.)
The open-label study was modified during its course on the basis of the safety data accrued to that point and new, supportive preclinical toxicity data. The modified open-label study was expanded to include treatment of 3 patients each with doses of 3x1010, 1x1011, and 2x1011 vp. (Patient 38, in screening at the end of the study, received 1x1010 vp.) A maximal dose of 2x1011 vp was chosen on the basis of additional preclinical toxicity data, in consultation with FDA and continuing Data Safety Monitoring Board review.
Study Assessments
Patients returned for posttreatment follow-up on days 3, 7, 14, 21, 30, 45, 60, and 90; at 6 months; and at 1 year. Safety variables included adverse event reports and changes from baseline in physical examinations, clinical laboratory evaluations, adenoviral antibody titer measurement, retinal eye examinations, and examinations of the index limb to assess rest pain, ulcer status, and RC.16 Observations related to limb status included changes in ischemic rest pain, healing of ischemic ulcers, and ankle brachial index (ABI). Additionally, 3-dimensional gadolinium contrastenhanced and/or 3-dimensional time-of-flight magnetic resonance angiography (MRA) were performed to detect changes in vascularization. Maximal intensity projections in similar orientations were used to compare pretreatment and posttreatment studies. An increase in the number of visible vessels or an increase in the intensity or apparent size of a previously visible vessel was considered an improvement. An independent reviewer blinded to patient treatment assignment scored the MRA data according to predefined specifications.
Ad2/HIF-1
/VP16
Ad2/HIF-1
/VP16 is a recombinant, replication-deficient adenovirus with an insert containing the DNA-binding and dimerization domains from the HIF-1
subunit, as well as a herpes virus VP16 transactivation domain to enable constitutive activation.1719 Ad2/HIF-1
/VP16 is propagated in human 293 cells, a permanent cell line of primary human embryonal kidney cells that were immortalized with sheared fragments of human type 5 adenovirus DNA. The bulk substance was purified with column chromatography, filtration (for vector concentration), and final sterile filtration. The resulting preformulated drug substance subsequently underwent final dilution in formulation buffer consisting of phosphate-buffered saline with 10% sucrose. Ad2/HIF-1
/VP16 is manufactured by Genzyme Corp (Cambridge, Mass).
Procedures for Administering Ad2/HIF-1
/VP16
In all but 1 dose cohort, the total dose of Ad2/HIF-1
/VP16 or placebo (ie, phosphate-buffered saline with 10% sucrose) was administered as a single treatment of 10 direct intramuscular injections with a volume of 100 µL for a total dose of 1.0 mL given into a single limb. In the 2x1011-vp cohort of the open-label study, the total dose of Ad2/HIF-1
/VP16 consisted of twenty 100-µL direct intramuscular injections of 1x1011 vp to achieve a total dose of 2x1011 vp given into a single limb for a total volume of 2.0 mL. The placement of the injections was at the discretion of the investigator and based on patient anatomy and the location of the occluded artery or arteries within the affected limb.
Statistical Analysis
All patients receiving
1 HIF-1
or placebo injections were included in safety analyses and limb status observations. By-group summary statistics were displayed for actual data reported at assessed time points, including descriptive statistics (sample count, mean, median, SD, minimum, and maximum) for continuous variables and frequencies and percentages for categorical variables. No hypothesis testing between groups was planned or conducted in these phase I dose-escalation studies. All statistical analyses were conducted in a validated SAS system (SAS Institute Inc, Cary, NC).
All authors had full access to and take full responsibility for the integrity of the data. All authors have read and agree to the manuscript as written.
| Results |
|---|
|
|
|---|
/VP16 phase I program, including demographics, diabetes status, time spent in the baseline RC class before the study, and the total number of arteries occluded at baseline. In addition, at baseline, 34% of individuals were on acetylsalicylic acid, 47% were on an angiotensin-converting enzyme inhibitor, 63% were on warfarin, 32% were on clopidogrel, and 58% were on statins. During the study, 47% of individuals were on acetylsalicylic acid, 58% were on an angiotensin-converting enzyme inhibitor, 68% were on warfarin, 45% were on clopidogrel, and 68% were on statins. Figures 2 and 3
in both studies, 32 began the study with rest pain, 18 began with ulcers, and 16 began with both rest pain and ulcers. Of the 7 patients who received placebo treatment, 6 had rest pain, 3 had ulcers, and 2 had both rest pain and ulcers.
|
|
|
Safety End Points
Adverse Event Profile
All but 1 HIF-1
patient experienced adverse events in either the RDBPC study or the open-label study. Table 2 summarizes the adverse events occurring in at least 10% of patients by World Health Organization Adverse Reactions Terminology preferred term. Most of these events were mild or moderate in severity and were not serious. None was judged to be related to treatment. A total of 21 unique patients experienced serious adverse events: 17 patients who received HIF-1
only, 3 placebo patients, and 1 patient who had adverse events both as a placebo and as a treated (rollover) patient. The serious adverse events that occurred were consistent with the patients extensive disease and comorbidities.
|
Deaths and Amputations
Five patients died while during the course of the present study, 3 from the HIF-1
group and 2 from the placebo group. No death was judged to be related to active treatment (Table 3). The median time to death was similar in the treatment and placebo groups. In the RDBPC study, 7 HIF-1
patients and 3 placebo patients met treatment failure criteria; another 3 HIF-1
patients underwent amputation before meeting treatment failure criteria. The distribution of time to treatment failure and/or amputation was similar in the HIF-1
and placebo groups.
|
All 10 major amputations occurred in HIF-1
patients (29% of 34 treated patients): 7 patients in the RDBPC study and 3 in the open-label study, including 1 patient who had received placebo before receiving active HIF-1
(Table 3). Although 0 of the 7 placebo patients from the RDBPC study experienced a major amputation, there were 2 deaths and 3 treatment failures, with only 2 patients remaining in the placebo group at 1 year. Despite the presence of RC4 and RC5 patients in roughly equal proportions at baseline, the RC5 patients constituted 90% of those who eventually required major amputations, 60% of those who died, and 71% of those who withdrew. In addition, although diabetics constituted only 37% of patients overall, they constituted 80% of the deaths in the trial.
Potential Risk of Ad2/HIF-1
/VP16 Mediated Angiogenesis
The adverse events described below are of special interest in the present study because of the theoretical risks of using a replication-deficient adenoviral vector and of the constitutively active HIF-1
transgene. Dependent edema of the lower extremities and injection site reactions were the 2 most common events of interest and were noted in similar proportions of treated and placebo patients: 44% to 43% of each group for dependent edema and 24% to 29% for injection site reaction, respectively. There was no evidence for promotion of tumor growth, although some patients had a history of malignancy diagnosed and treated >5 years previously. Vision disorders occurred in 29% of treated patients but no placebo patients; however, none of the cases involved pathological choroidal neovascularization or new/active proliferative retinopathy. A variety of minor retinal findings, including drusen and A-V nicking, were reported. Flu-like symptoms occurred in 26% of the treated patients and no placebo patients. Of these 9 patients, only 2 experienced symptom onset within 72 hours of receiving Ad2/HIF-1
/VP16 (range, 2 to 180 days; median, 15 days). As expected, a rise in anti-adenoviral antibodies was observed in patients who had received Ad2/HIF-1
/VP16. Antibody titers peaked at day 60 and then declined (data not shown; no dose-dependent trends were identified given the considerable between- and within-patient variability). Because of the small group sizes for each dose, no dose effect could be noted across the study cohorts. Consequently, a maximum tolerated dose was not established. Further dose escalation is not supported by the sponsors preclinical toxicity data, however, and therefore is not planned.
Limb and Surrogate Outcomes
Figures 2 and 3
summarize the clinical outcomes for each patient at baseline, 6 months, 1 year, and final disposition. Only 2 of the 7 placebo patients remained at 1 year: 1 patient had return of rest pain after reporting resolution at 6 months and a toe amputation, and 1 patient had continuing rest pain. At 6 months, limb status observations in HIF-1
patients included complete rest pain resolution in 12 of 32 patients alive with index limb and complete ulcer healing in 3 of 18 patients (3 of 9 alive with index limb). At 1 year, there was complete rest pain resolution in 14 of 32 patients (14 of 21 alive with index limb), complete ulcer healing in 5 of 18 patients (5 of 8 alive with index limb), and 5 cases of ulcer healing accompanied with resolution of rest pain. Figure 4 shows an example of ulcer healing at 1 year in a patient originally randomized to the placebo group and subsequently adjudicated as a treatment failure who went on to receive active therapy at 1x1010 vp. None of the patients in the 2 highest-dosing groups (1x1011 and 2x1011 vp) experienced either death or amputation. Complete ulcer healing occurred in patients given doses ranging from 1x109 to 2x1011 vp; ulcer healing occurred in all 3 RC5 patients given the 1x1011- to 2x1011-vp doses.
|
ABI measurement was not available for all study patients because of arterial calcification, amputation, death, or early withdrawal. The median ABI for HIF-1
patients was 0.36 at baseline (n=24), 0.45 at 6 months (n=17), and 0.46 at 1 year (n=16). The median ABIs for placebo patients were 0.39 (n=5), 0.49 (n=3), and 0.48 (n=1), respectively. No correlation was seen between ABI and clinical outcomes, likely secondary because of the small number of patients.
Complete and comparable baseline and 6- and 12-month MRAs were available in 18 (n=14 HIF-1
, n=4 placebo) and 13 (n=11 HIF-1
, n=2 placebo) patients, respectively. At 6 months, 7 HIF-1
patients showed improvement, whereas at 12 months, 2 HIF-1
patients showed improvement, with the remainder either worsening or showing no change. In the placebo group, 2 patients showed improvement at 6 and 12 months, respectively; the remainder worsened or showed no change compared with baseline. Improvement in MRA score did not correlate with improved clinical status, ABI, or the study injection sites.
| Discussion |
|---|
|
|
|---|
, which regulates the expression of specific genes involved in the response to hypoxia and wound healing.9,20 Ad2/HIF-1
/VP16 appeared to be safe when delivered into skeletal muscle in the leg at doses ranging from 1x108 to 2x1011 in no-option CLI patients. An important point is that no safety problems emerged in this phase I program, including no evidence of malignancy or ocular neovascularization disorders related to the transgene, at least in the short term. Longer-term follow-up in a larger number of patients may be needed to firmly establish the safety of both vector and transgene. It is presumed that patients can receive only a single administration of adenoviral HIF-1
/VP16 because of the generation of an adenoviral serotype-specific immune response.21 Adenoviral vectors have now been used in multiple clinical trials in cardiovascular disease through intramuscular, intramyocardial, and intracoronary routes, with no indication of hepatic or other systemic toxicity at the doses used in this trial.5,2224 No dose-limiting side effects emerged in the present study at the maximum dose tested (2x1011 vp). Further dose escalations were not attempted because this was the highest dose supported by preclinical data and approved by the FDA for testing in humans. Additionally, further increases in dosing are associated with technical limitations with manufacturing adenoviruses at these high titers.
Overall, the present study confirmed the high rate of progression of disease in advanced CLI (whether treated or placebo), with amputation and mortality rates of 26% and 3% at 6 months and 26% and 13% at 1 year, respectively. Rates of amputation and mortality in patients treated with Ad2/HIF-1
/VP16 were 29% and 0% at 6 months and 29% and 9% at 1 year. There was a high rate of crossover to active drug among placebo patients in the present trial, with most continuing on placebo experiencing disease progression or death at 1 year, similar to rates from randomized trials and meta-analyses of studies conducted in similar patient populations.1,2527 In previous studies, 6-month rates of amputation ranged from 20% to 44%, and mortality rates ranged from 20% to 54%.1 Some of these differences are undoubtedly secondary to improvements in pharmacotherapy for atherosclerosis (eg, use of HMG CoA reductase inhibitors, angiotensin-converting enzyme inhibitors, and antiplatelet agents), the unique patient selection criteria in the trial, and improvements in interventional approaches.
Improvement in limb status (ie, complete ulcer healing and/or resolution of rest pain) was observed in some of the 34 CLI patients treated with Ad2/HIF-1
/VP16 in the 2 present studies. Complete resolution of rest pain occurred in 12 patients at 6 months (12 of 23 alive with index limb) and 14 patients at 1 year (14 of 21 alive with index limb). Complete ulcer healing occurred in 3 patients (3 of 9 alive with index limb) at 6 months and 5 patients (5 of 8 alive with index limb) at 1 year. Interpretation of the MRA results was confounded by the fact that a number of patients were unable to repeat the test either because they had experienced limb loss or for other reasons. Furthermore, we restricted evaluation to those patients who had comparable gadolinium contrastenhanced MRA or time-of-flight MRAs, further reducing the total evaluable scans. The lack of correlation between clinical improvement and MRA-derived scores may therefore relate to some of these issues. Additional considerations include the use of higher-resolution approaches (submillimeter), contrast-enhanced MRA below the level of the foot in an attempt to image collaterals, and standardization of "timing runs," gadolinium contrast dose, and delivery at study sites to ensure comparability of data.
| Conclusions |
|---|
|
|
|---|
in peripheral arterial disease. | Acknowledgments |
|---|
Source of Funding
The present study was funded by a grant from Genzyme Corp, the manufacturer of Ad2/HIF-1
/VP16.
Disclosures
Drs Rajagopalan, Olin, Deitcher, Laird, Grossman, Goldman, and Chronos were primary investigators on this Genzyme-sponsored clinical study. In addition, Drs Rajagopalan, Olin, and Chronos received grant support and are on the Steering Committee or other scientific advisory boards for this clinical program. Dr Kelly and K. McEllin are employees of Genzyme Corp. A. Pieczek reports no conflicts.
| References |
|---|
|
|
|---|
. Hypoxia-inducible factor-1
is a master switch gene that initiates and choreographs the expression of multiple proangiogenic genes in patients with critical limb ischemia. Although this phase I trial was not large enough to judge clinical efficacy in critical limb ischemia, the absence of major safety issues is reassuring. These data, along with the findings of clinical improvement in some patients, provided the rationale for a randomized, placebo-controlled trial of a hypoxia-inducible factor-1
transgene in a phase II study in subjects with advanced claudication symptoms that currently is ongoing in Europe and the United States. If this gene therapy approach were to prove safe and effective after subsequent, adequately powered phase III studies, it would provide clinicians with an alternative and/or an adjunctive approach to treating patients with refractory ischemic limb symptoms. This article has been cited by other articles:
![]() |
J. Tongers, J. G. Roncalli, and D. W. Losordo Therapeutic Angiogenesis for Critical Limb Ischemia: Microvascular Therapies Coming of Age Circulation, July 1, 2008; 118(1): 9 - 16. [Full Text] [PDF] |
||||
![]() |
R. J. Powell, M. Simons, F. O. Mendelsohn, G. Daniel, T. D. Henry, M. Koga, R. Morishita, and B. H. Annex Results of a Double-Blind, Placebo-Controlled Study to Assess the Safety of Intramuscular Injection of Hepatocyte Growth Factor Plasmid to Improve Limb Perfusion in Patients With Critical Limb Ischemia Circulation, July 1, 2008; 118(1): 58 - 65. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. Lei, S. Mason, D. Liu, Y. Huang, C. Marks, R. Hickey, I. S. Jovin, M. Pypaert, R. S. Johnson, and F. J. Giordano Hypoxia-Inducible Factor-Dependent Degeneration, Failure, and Malignant Transformation of the Heart in the Absence of the von Hippel-Lindau Protein Mol. Cell. Biol., June 1, 2008; 28(11): 3790 - 3803. [Abstract] [Full Text] [PDF] |
||||
![]() |
C Kalka and I. Baumgartner Gene and stem cell therapy in peripheral arterial occlusive disease Vascular Medicine, May 1, 2008; 13(2): 157 - 172. [Abstract] [PDF] |
||||
![]() |
C. M. Kramer Skeletal Muscle Perfusion in Peripheral Arterial Disease: A Novel End Point for Cardiovascular Imaging J. Am. Coll. Cardiol. Img., May 1, 2008; 1(3): 351 - 353. [Full Text] [PDF] |
||||
![]() |
A. O. Dokun, S. Keum, S. Hazarika, Y. Li, G. M. Lamonte, F. Wheeler, D. A. Marchuk, and B. H. Annex A Quantitative Trait Locus (LSq-1) on Mouse Chromosome 7 Is Linked to the Absence of Tissue Loss After Surgical Hindlimb Ischemia Circulation, March 4, 2008; 117(9): 1207 - 1215. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Bosch-Marce, H. Okuyama, J. B. Wesley, K. Sarkar, H. Kimura, Y. V. Liu, H. Zhang, M. Strazza, S. Rey, L. Savino, et al. Effects of Aging and Hypoxia-Inducible Factor-1 Activity on Angiogenic Cell Mobilization and Recovery of Perfusion After Limb Ischemia Circ. Res., December 7, 2007; 101(12): 1310 - 1318. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. L. Semenza Regulation of tissue perfusion in mammals by hypoxia-inducible factor 1 Exp Physiol, November 1, 2007; 92(6): 988 - 991. [Abstract] [Full Text] [PDF] |
||||
![]() |
F. J. Giordano Therapeutic Gene Regulation: Targeting Transcription Circulation, March 13, 2007; 115(10): 1180 - 1183. [Full Text] [PDF] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||