Published online before print February 21, 2005, doi: 10.1161/01.CIR.0000156332.56894.22
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Circulation. 2005;111:1033-1039.)
© 2005 American Heart Association, Inc.
Imaging |
Noninvasive Assessment of Blood Flow Based on Magnetic Resonance Global Coherent Free Precession
From Duke Cardiovascular Magnetic Resonance Center (I.K., J.F.H., A.W., T.A., M.A.P., E.-L.C., R.J.K., R.M.J.) and Siemens Medical Systems (W.G.R.).
Correspondence to Robert M. Judd, PhD, Duke Cardiovascular Magnetic Resonance Center, Duke University Health System, PO Box 3934, Durham, NC, 27710. E-mail robert.judd{at}dcmrc.mc.duke.edu
Received June 2, 2004; revision received November 14, 2004; accepted November 29, 2004.
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
Background— Magnetic resonance global coherent free precession (GCFP) is a new technique that produces cine projection angiograms directly analogous to those of x-ray angiography noninvasively and without a contrast agent. In this study, we compared GCFP blood flow with "gold standards" to determine the accuracy of noninvasive GCFP blood flow measurements.
Methods and Results— The relationship between GCFP blood flow and true blood flow defined by invasive ultrasonic flow probe and by phase contrast velocity encoded MRI (VENC) was studied in anesthetized dogs (n=6). Blood flow was controlled by use of a hydraulic occluder around the left iliac artery. GCFP images were acquired by selectively exciting the abdominal aorta and visualizing temporal blood flow into the iliac arteries. GCFP flow was similar to ultrasonic blood flow at baseline (131.3±44.8 versus 114.8±34.2 mL/min), during occlusion (10.8±5.1 versus 6.5±7.2 mL/min), during reactive hyperemia (191.4±100.7 versus 260.3±138.7 mL/min), during the new resting state (135.5±52.4 versus 117.8±24.1 mL/min), and during partial occlusion (61.4±36.4 versus 49.3±13.1 mL/min, P=NS for all). Results comparing GCFP flow with VENC were similar. Statistical analysis revealed that GCFP flow was related to mean blood flow assessed by the flow probe (P<0.0001) and by VENC (P<0.0001). In the control right iliac artery, conversely, GCFP measurements were unaffected throughout all left iliac interventions (P=NS).
Conclusions— GCFP blood flow is linearly related to true blood flow for a straight, cylindrical blood vessel without branches. Although more complex geometries imply a qualitative rather than a quantitative relationship, the data nevertheless suggest that GCFP may serve as the basis for a new form of noninvasive stress testing.
Key Words: imaging • angiography • atherosclerosis • stenosis
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Magnetic resonance global coherent free precession (GCFP)1 is a new technique that produces cine projection angiograms directly analogous to those of x-ray angiography noninvasively and without a contrast agent. GCFP is based on continuous MRI "tagging" of protons in flowing blood as they pass through a slice in space and the simultaneous acquisition of 2D projection images as blood flows into the distal vasculature. For example, Figure 1 shows GCFP images of the canine pulmonary arteries and their branches with the excitation slice placed at the level of the main pulmonary artery. A key feature of GCFP images is that body tissues outside the slice are never excited and therefore remain invisible, resulting in cine angiograms similar to those of invasive x-ray angiography. In principle, GCFP MRI provides both morphological information (vessel diameters) and functional information (pulsatile blood flow) in a single acquisition. Although the GCFP technique itself has been described previously,1 the potential ability of GCFP to noninvasively assess true physiological blood flow has not yet been investigated.
|
Noninvasive methods to assess blood flow are in general useful for detecting arterial stenoses. Hemodynamically significant stenoses result in reductions in resting blood flow and/or reductions in flow reserve during exercise and/or pharmacological stress.2,3 Detecting relative reductions in blood flow, for example, is the underlying physiological principle for the detection of coronary artery disease by SPECT,4–7 PET,8,9 and contrast-enhanced perfusion MRI.10 A new imaging technique capable of assessing absolute and/or relative arterial blood flows, therefore, theoretically represents a new approach to the noninvasive detection of arterial stenoses.
We hypothesized that GCFP MRI can accurately assess blood flow compared with "gold standards." The objective of the present study was to test this hypothesis by comparing in vivo GCFP images with true volumetric blood flow assessed by an invasive flow probe and by phase contrast velocity encoded MRI (VENC).
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Theory
A detailed mathematical analysis of blood flow assessed by GCGP is given in the Data Supplement. In brief, the first time derivative of GCFP filling distance, x(t), equals blood flow velocity, v(t) (see Figure 2). Volumetric blood flow Q(t) is related to both x(t), v(t), and the cross-sectional area of the study vessel (A). As shown in the Data Supplement, these equations can be combined to show that the time-averaged volumetric blood flow (
) during 1 cardiac cycle is linearly related to the maximum distance that GCFP-labeled blood protons move forward into the distal vascular tree during that cardiac cycle, xmax. Our experiments therefore were designed to compare xmax measured by GCFP with measured by the invasive flow probe and by VENC.
|
Experimental Preparation
Blood flow was examined in the iliac arteries of anesthetized dogs (n=6). The iliac arteries were chosen for these experiments because their relatively simple geometry allowed the GCFP excitation slice to be placed across the descending aorta immediately proximal to the aortic bifurcation such that blood flow could be manipulated in the left iliac artery while the right iliac artery served as control (see Figure 3). After anesthetization (brevitol), intubation, and mechanical ventilation with 1% to 3% isoflurane, the left iliac artery was dissected free and an invasive volumetric flow probe was placed around the vessel (Transonic Systems, Inc). A snare occluder was also placed around the artery to control blood flow. In pilot experiments, we found that the "MRI-compatible" flowmeter nevertheless contained small amounts of (nonferromagnetic) metal, which deleteriously affected the GCFP state because of disturbances in homogeneity of the static magnetic field. To address this, both the flow probe and occluder were placed approximately 15 cm distal to the aortic bifurcation (see Figure 3), and all branch vessels proximal to the flow probe were ligated.
|
MR Imaging
Images were acquired on a 1.5-T MR scanner (Siemens Sonata) by use of a 6-element phased-array surface radiofrequency receiver coil. After scout imaging to determine the location of the iliac arteries, ECG-gated GCFP images were acquired during repeated 15- to 20-second breathholds by use of a segmented k-space version of the GCFP pulse sequence described in detail elsewhere.1 Imaging parameters were as follows: field of view, 300x150 mm; matrix, 384x192; in-plane pixel size, 800x800 µm; TR, 4.4 ms; TE, 2.2 ms; bandwidth, 977 Hz/pixel; flip angle, 45°; and excitation slice thickness, 5 mm. For comparison with the GCFP images, VENC images were acquired by use of a standard clinical pulse sequence (ECG-gated, segmented k-space 2D-FLASH sequence; field of view, 250x172 mm; matrix, 256x176; repetition time, 41 ms; TE, 3.0 ms; bandwidth, 350 Hz/pixel; flip angle, 25°; and slice thickness, 10 mm), and images were acquired during repeated breathholds.
Experimental Protocol
Baseline images were acquired with the occluder fully open. To avoid image artifacts caused by radiofrequency noise emitted by the flow probe amplifier, the flow probe was turned off during GCFP image acquisition, and volumetric blood flow was recorded immediately before and after imaging. Baseline through-plane VENC images were acquired approximately 1 cm distal to the aortic bifurcation perpendicular to the axis of the iliac artery. As during GCFP imaging, the flow probe was turned off during VENC imaging.
After baseline imaging, images were acquired before, during, and after a 5-minute occlusion to examine GCFP blood flow during occlusion and reactive hyperemia. Immediately after occlusion of the iliac artery, GCFP images and flowmeter readings were acquired each minute for the first 2 minutes. At t=4 minutes, another set of GCFP images and flowmeter measurements were made. At t=5 minutes, the occluder was released, and the measurements were repeated each minute until a new resting state was established. The occlusion/reperfusion protocol was then repeated a second time to allow acquisition of GCFP and VENC images.
The effects of partial occlusions were investigated by pulling on the snare while monitoring the flowmeter and adjusting flow to approximately 50% of baseline flow. GCFP and VENC images were then acquired after flow had stabilized for at least 3 minutes.
Data Analysis
All MRI data were analyzed on a Leonardo workstation (Siemens Medical Solutions). The maximum GCFP filling distance (xmax in cm) for all GCFP images was measured in both the left and right iliac arteries at 5 different flow conditions: baseline, occlusion, reactive hyperemia, new resting state, and partial occlusion. GCFP flow ( by GCFP in mL/min) was calculated on the basis of the theoretical considerations described above. (See also Data Supplement, Equation 3.) The average blood flow velocity (in cm/s), the time-averaged blood flow ( by VENC) during 1 cardiac cycle (tf–t0 in ms), and the study vessel cross-sectional area (A in cm2) were assessed from the VENC images by use of ARGUS software. The theoretical maximum filling distances (xmax) were calculated both from the true blood flow data measured by the flow probe and VENC. (See Data Supplement for details, Equations 4 and 5.)
Statistical Analysis
Data are presented as mean±SD. The mean of the flow probe measurements before and after each image set was calculated for comparison with GCFP. Mixed-effects model analysis was used to evaluate the relationship between maximum filling distance and flow conditions; comparisons between flow conditions were made by use of the Tukey adjustment for multiple, pairwise comparisons. Mixed-effects models were also used to assess the relationship between GCFP filling distances and true blood flow. All statistical tests were 2 tailed; a value of P<0.05 was regarded as significant.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Comparison of GCFP Blood Flow and True Blood Flow
Mean GCFP blood flow measurements and true blood flow measured by the flow probe and VENC are shown in the Table. On the basis of all 6 animals and 5 experimental conditions (Figure 4), GCFP blood flow was linearly related to true blood flow measured by the flow probe (y=0.69x+30.8; P<0.0001) and VENC (y=0.64x+2.51; P<0.0001).
|
|
GCFP Filling Distances
At baseline, the maximum GCFP filling distance (xmax) in the left and right iliac arteries was 5.9±1.7 cm (3.9 to 8.8 cm) [mean±SD (range)] and 5.9±1.5 cm (3.6 to 8.0 cm), respectively (P=NS). In the left iliac artery, xmax fell to 0.5±0.2 cm (0.3 to 0.9 cm) during occlusion (P<0.05 compared with baseline), increased to 7.9±2.2 cm (4.1 to 9.7 cm) during reactive hyperemia (P<0.05 compared with occlusion), returned to a new resting steady state of 5.6±1.2 cm (4.4 to 7.5 cm) (P<0.05 compared with occlusion and reactive hyperemia), and decreased to 2.6±1.6 cm (1.1 to 5.3 cm) during partial occlusion (P<0.05 compared with baseline, occlusion, reactive hyperemia, and new resting steady state) (Figure 5).
|
In the right iliac artery, xmax during occlusion, reactive hyperemia, new resting steady state, and partial occlusion of the study vessel was 6.3±1.6 cm (3.7 to 7.9 cm), 5.5±1.6 cm (2.8 to 7.4 cm), 5.8±1.2 cm (4.2 to 7.7 cm), and 6.2±1.6 cm (4.7 to 9.1 cm), respectively. The differences were not statistically significant (P=NS).
Comparison of Maximum GCFP Filling Distance and True Blood Flow
Figure 6 shows the comparison of xmax with true blood flow measured by flow probe and VENC. On the basis of all 6 animals and all 5 experimental conditions, xmax was linearly related to mean blood flow assessed by the flowmeter (y=0.024x+1.89; P<0.0001). On the basis of all 6 animals and 3 flow conditions, xmax was linearly related to mean blood flow assessed by VENC (y=0.033x+0.2; P<0.0001).
|
Comparison of GCFP xmax With Calculated xmax on the Basis of the Flow Probe and VENC
Figure 7 shows the comparison of the calculated xmax from the flow probe data and VENC with the measured GCFP xmax. There is a linear relation between GCFP xmax and xmax calculated on the basis of the data from the flowmeter (y=0.63x+1.46; P<0.0001) and from VENC (y=0.91x–0.33; P<0.0001).
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
The main findings of this study were that blood flow assessed by GCFP systematically fell to zero during occlusion, increased beyond their baseline values during reactive hyperemia, and decreased below baseline as a result of partial occlusion. These data establish that GCFP blood flow measurements are directly related to physiological blood flow. We also found that, as predicted by theory (Data Supplement), GCFP maximum filling distance, xmax, is linearly related to average blood flow and therefore represents a rapidly assessable parameter, both visually and quantitatively, of blood flow in a given vessel.
Comparison of GCFP With Existing Techniques
"Luminographic" projection MRA techniques have been described in previous studies as projection angiograms,11,12 subtraction angiograms,12,13 arterial flow-tagging,14,15 spin labeling,16 and signal targeting with alternating radiofrequency.17 Although traditional MRI spin-labeling techniques also yield projection angiograms, the primary advantage of GCFP is its ability to produce multiframe movies depicting blood flow rather than a single frame angiogram because of an intrinsically higher imaging efficiency.1 Accordingly, GCFP is fundamentally different from traditional MRI spin labeling techniques in that GCFP images intrinsically contain a temporal component. Like GCFP, VENC images also contain a temporal component and can be used to noninvasively assess blood flow.18,19 VENC assessment of blood flow, however, is limited to a single spatial location within a vascular tree. For example, the assessment of blood flow in the pulmonary arteries shown in Figure 1 by use of VENC would require repeated prescriptions of imaging planes across each individual vessel of interest. Conversely, GCFP provides flow information throughout the entire vascular tree distal to the excitation slice based on a single acquisition. Accordingly, the information provided by GCFP differs from existing noninvasive techniques such as MRI spin-labeling and VENC and is more analogous to invasive techniques, such as x-ray angiography.
Semiquantitative blood flow information can be extracted from invasive x-ray cine angiograms by use of techniques such as TIMI frame counting. The underlying principle of TIMI frame counting is that blood flow can be characterized by determining the number of movie frames from the entrance of x-ray contrast dye into a vessel to the frame in which the dye enters the distal landmark branch.20 Because of the similarity of GCFP images to those of invasive angiography, frame counting of GCFP images might also be expected to provide blood flow information. A fundamental difference between x-ray angiograms and GCFP, however, is that whereas the "leading edge" of the x-ray contrast agent is a function of both blood flow and other variables, such as the rate of contrast injection, catheter size, degree of engagement, and phase of cardiac cycle,20 for GCFP, the "leading edge" of labeled blood is a function of flow alone, because the blood itself serves as the contrast agent and is effectively labeled instantaneously. Perhaps more importantly, x-ray angiograms require the injection of several milliliters of the contrast agent, which itself disturbs blood flow. The fact that contrast agent injection rates and volumes are irrelevant to GCFP imaging implies that GCFP may provide a more accurate measure of blood flow compared with TIMI frame counting of x-ray angiograms. Additional geometric considerations common to both GCFP images and x-ray angiograms, however, also affect the relationship to blood flow.
Effects of Blood Vessel Curvature, Branching, and Taper
Our data demonstrate that in the setting of a straight, cylindrical blood vessel without branches, such as the iliac artery, a linear relationship exists between maximum GCFP filling distance (xmax) and true volumetric blood flow (see Figures 4 and 6
). In general, however, blood vessels follow a tortuous 3D path through space, branch into smaller vessels, and decrease their diameters along their length. Each of these geometric changes will affect the relationship between blood flow and filling distances both in x-ray angiograms and in GCFP images.
The effects of vessel curvature on filling distance cannot be assessed in 3 dimensions on the basis of images from a single projection direction. Biplanar and, in particular, orthogonal projection images, however, can be used to fully describe the 3D orientation of blood vessels in space. In principle, therefore, GCFP filling distances will remain linearly related to true blood flow even for curved blood vessels, provided that there are no vessel branches or tapering and provided that the data analysis is based on 2 (orthogonal) sets of projection images.
The effects of vessel branching can in principle be described by accounting for the relative cross-sectional areas of the parent and child vessels. Conservation of mass can then be used as the basis for calculations of blood flows in child vessels relative to those of the parent. Noncircular blood vessel lumina, however, would add yet an additional unknown, which would be only partially addressed by 2 orthogonal projections. Vessel taper could also in principle be accounted for but requires consideration of lumen diameters along the length of each vessel, and, as for branching blood vessels, noncircular vessel lumina would add yet another source of nonlinearity to the relationship between GCFP xmax and true blood flow. Accordingly, fully accounting for the effects of vessel branching and taper would require significant additional analytical effort.
In practice, the acquisition of a second, orthogonal projection is straightforward and would simply require 1 additional 10- to 15-second breathhold. As for x-ray angiography, the resulting 2 cine angiograms would be sufficient to fully describe both the temporal and spatial distribution of blood flow throughout all blood vessels of a given vascular tree. Conversely, properly accounting for vessel branching and taper would require significant additional time and effort that would appear to be unrealistic in a routine clinical setting. Assuming that vessel branching and taper are not accounted for, the relationship between xmax and true blood flow will become nonlinear, precluding quantitative analyses. Importantly, however, although nonlinear, the relationship will remain monotonic (increases in blood flow will always increase xmax), such that qualitative assessments can be made on the basis of visual assessment.
Implications for Noninvasive Stress Testing
GCFP filling distance is directly related to true blood flow and can be rapidly assessed both visually and quantitatively. Side-by-side visual interpretation could therefore provide the basis for a new form of noninvasive stress testing. For example, under baseline conditions, the GCFP excitation slice could be placed immediately proximal to a vascular tree, and filling distances within every branch of every vessel in the tree could be visually inspected on the basis of 2 orthogonal views. Exercise or pharmacological stress could then be applied and the GCFP imaging procedure repeated. Visual inspection of the resulting orthogonal pair of stress images could then be made in comparison with the corresponding pair of orthogonal rest images to assess blood flow reserve within each individual blood vessel of the vascular tree. Failure of xmax to increase in response to stress would indicate a lack of flow reserve and the presence of a flow-limiting arterial stenosis. Further studies will be required to test the hypothesis that rest-stress GCFP imaging can be used to detect arterial stenoses.
Conclusions
GCFP blood flow measurements change systematically in response to variations in true blood flow. GCFP blood flow is linearly related to true blood flow for a straight, cylindrical blood vessel without branches. Curved blood vessels can be accounted for by biplanar imaging, but preservation of a linear relationship in the presence of branching and vessel taper require accounting for additional variables. Even without accounting for these variables, the relationship between GCFP filling distance and true blood flow will be monotonic and theoretically provides the basis for a new form of noninvasive stress testing.
|
Acknowledgments |
|---|
This work was supported in part by National Institutes of Health, National Heart, Lung, and Blood Institute grants RO1-HL-076489 and K02-HL-04394 (Dr Judd). The authors thank Ronnie Johnson for his help with the experimental preparation.
Disclosure
Drs Judd, Kim, and Chen are named as inventors on a pending US patent related to this study. Dr Rehwald is an employee of Siemens Medical Systems, Inc, which manufactures MRI scanners.
|
Footnotes |
|---|
The online-only Data Supplement can be found with this article at http://www.circulationaha.org.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
1. Rehwald WG, Chen EL, Kim RJ, Judd RM. Noninvasive cineangiography by magnetic resonance global coherent free precession. Nat Med. 2004; 10: 545–549.[CrossRef][Medline] [Order article via Infotrieve]
2. Wilson RF, Marcus ML, White CW. Prediction of the physiologic significance of coronary arterial lesions by quantitative lesion geometry in patients with limited coronary artery disease. Circulation. 1987; 75: 723–732.
3. Gould KL, Lipscomb K. Effects of coronary stenoses on coronary flow reserve and resistance. Am J Cardiol. 1974; 34: 48–55.[CrossRef][Medline] [Order article via Infotrieve]
4. Garcia EV, DePuey EG, Sonnemaker RE, Neely HR, DePasquale EE, Robbins WL, Moore WH, Heo J, Iskandrian AS, Campbell J. Quantification of the reversibility of stress-induced thallium-201 myocardial perfusion defects: a multicenter trial using bull’s-eye polar maps and standard normal limits. J Nucl Med. 1990; 31: 1761–1765.
5. Maddahi J, Van Train K, Prigent F, Garcia EV, Friedman J, Ostrzega E, Berman D. Quantitative single photon emission computed thallium-201 tomography for detection and localization of coronary artery disease: optimization and prospective validation of a new technique. J Am Coll Cardiol. 1989; 14: 1689–1699.[Abstract]
6. Mahmarian JJ, Boyce TM, Goldberg RK, Cocanougher MK, Roberts R, Verani MS. Quantitative exercise thallium-201 single photon emission computed tomography for the enhanced diagnosis of ischemic heart disease. J Am Coll Cardiol. 1990; 15: 318–329.[Abstract]
7. Van Train KF, Berman DS, Garcia EV, Berger HJ, Sands MJ, Friedman JD, Freeman MR, Pryzlak M, Ashburn WL, Norris SL. Quantitative analysis of stress thallium-201 myocardial scintigrams: a multicenter trial. J Nucl Med. 1986; 27: 17–25.
8. Uren NG, Melin JA, De Bruyne B, Wijns W, Baudhuin T, Camici PG. Relation between myocardial blood flow and the severity of coronary-artery stenosis. N Engl J Med. 1994; 330: 1782–1788.
9. Muzik O, Duvernoy C, Beanlands RS, Sawada S, Dayanikli F, Wolfe ER Jr, Schwaiger M. Assessment of diagnostic performance of quantitative flow measurements in normal subjects and patients with angiographically documented coronary artery disease by means of nitrogen-13 ammonia and positron emission tomography. J Am Coll Cardiol. 1998; 31: 534–540.
10. Klocke FJ, Simonetti OP, Judd RM, Kim RJ, Harris KR, Hedjbeli S, Fieno DS, Miller S, Chen V, Parker MA. Limits of detection of regional differences in vasodilated flow in viable myocardium by first-pass magnetic resonance perfusion imaging. Circulation. 2001; 104: 2412–2416.
11. Wedeen VJ, Meuli RA, Edelman RR, Geller SC, Frank LR, Brady TJ, Rosen BR. Projective imaging of pulsatile flow with magnetic resonance. Science. 1985; 230: 946–948.
12. Dixon WT, Du LN, Faul DD, Gado M, Rossnick S. Projection angiograms of blood labeled by adiabatic fast passage. Magn Reson Med. 1986; 3: 454–462.[Medline] [Order article via Infotrieve]
13. Singer JR, Crooks LE. Nuclear magnetic resonance blood flow measurements in the human brain. Science. 1983; 221: 654–656.
14. Feinberg DA, Crooks L, Hoenninger J III, Arakawa M, Watts J. Pulsatile blood velocity in human arteries displayed by magnetic resonance imaging. Radiology. 1984; 153: 177–180.
15. Burstein D. MR imaging of coronary artery flow in isolated and in vivo hearts. J Magn Reson Imaging. 1991; 1: 337–346.[Medline] [Order article via Infotrieve]
16. Reeder SB, Atalay MK, McVeigh ER, Zerhouni EA, Forder JR. Quantitative cardiac perfusion: a noninvasive spin-labeling method that exploits coronary vessel geometry. Radiology. 1996; 200: 177–184.
17. Edelman RR, Siewert B, Adamis M, Gaa J, Laub G, Wielopolski P. Signal targeting with alternating radiofrequency (STAR) sequences: application to MR angiography. Magn Reson Med. 1994; 31: 233–238.[Medline] [Order article via Infotrieve]
18. Mohiaddin RH, Pennell DJ. MR blood flow measurement: clinical application in the heart and circulation. Cardiol Clin. 1998; 16: 161–187.[CrossRef][Medline] [Order article via Infotrieve]
19. Firmin DN, Nayler GL, Klipstein RH, Underwood SR, Rees RS, Longmore DB. In vivo validation of MR velocity imaging. J Comput Assist Tomogr. 1987; 11: 751–756.[Medline] [Order article via Infotrieve]
20. Gibson CM, Cannon CP, Daley WL, Dodge JT Jr, Alexander B Jr, Marble SJ, McCabe CH, Raymond L, Fortin T, Poole WK, Braunwald E. TIMI frame count: a quantitative method of assessing coronary artery flow. Circulation. 1996; 93: 879–888.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||