Dynamic Holographic Imaging of the Beating Human Heart
Background—Currently, the reporting and archiving of echocardiographic data suffer from the difficulty of representing heart motion on printable 2-dimensional (2D) media.
Methods and Results—We studied the capability of holography to integrate motion into 2D echocardiographic prints. Images of normal human hearts and of a variety of mitral valve function abnormalities (mitral valve prolapse, systolic anterior motion of the mitral leaflets, and obstruction of the mitral valve by a myxoma) were acquired digitally on standard echocardiographic machines. Images were processed into a data format suitable for holographic printing. Angularly multiplexed holograms were then printed on a prototype holographic “laser” printer, with integration of time in vertical parallax, so that heart motion became visible when the hologram was tilted up and down. The resulting holograms displayed the anatomy with the same resolution as the original acquisition and allowed detailed study of valve motion with side-by-side comparison of normal and abnormal findings. Comparison of standard echocardiographic measurements in original echo frames and corresponding hologram views showed an excellent correlation of both methods (P<0.0001, r2=0.979, mean bias=2.76 mm). In this feasibility study, both 2D and 3D holographic images were produced. The equipment needed to view these holograms consists of only a simple point-light source.
Conclusions—Holographic representation of myocardial and valve motion from echocardiographic data is feasible and allows the printing on a 2D medium of the complete heart cycle. Combined with the recent development of online holographic printing, this novel technique has the potential to improve reporting, visualization, and archiving of echocardiographic imaging.
Since prehistoric times, humans have been able to capture and display reality in the form of static 2-dimensional (2D) images. Only a century ago, however, it became possible to capture the motion of objects in the form of a series of single frames and to play them back as an animation. The ability to display motion and structure has stimulated the evolution of echocardiography from M-mode echocardiography,1 which displays only 1 spatial dimension over time, to 2D echocardiography, allowing improved understanding of cardiac structure in vivo and new insights into cardiac function. However, to view cardiac motion in a compact display is more cumbersome with 2D compared with M-mode echocardiography. Review of 2D echocardiograms requires a time-consuming search on videotapes and is restricted to the resources of the echocardiography laboratory. Digital imaging has not substantially relieved this situation, with the requirement for computers, networks, and databases demanding a huge storage capacity. In addition, the information content of echocardiographic data has markedly increased, especially with the introduction of 3D and 4D echocardiography.
The technique of holography offers the possibility of integrating additional data dimensions into a 2D print. Although this is typically used to convey information about the 3D structure of objects, we explored holography as a means of integrating the temporal dimension (cardiac motion) onto a 2D print medium.
Data sets from normal subjects and from patients referred to our laboratory for echocardiograms for clinical indications were selected with the aim of gathering a representative series of mitral valve pathological conditions with sufficient image quality. We chose these conditions because visual assessment of valve motion is critical to the precise diagnosis of valve disease. For this feasibility study, we chose image loops of (1) 2 patients with normal mitral valves, (2) a patient with classic bileaflet mitral valve prolapse, (3) a patient with isolated prolapse of the posterior leaflet, (4) a patient with hypertrophic obstructive cardiomyopathy and systolic anterior motion of the mitral valve, and (5) a patient with diastolic obstruction of the mitral inflow by a large left atrial myxoma.
Image sequences of normal and abnormal hearts were acquired with a standard echocardiograph (Hewlett Packard Sonos 2500). A transthoracic 2.5-MHz transducer and a multiplane transesophageal probe with 6.5/5.0-MHz transducer frequencies were used. In 1 patient, a 3D echocardiogram was acquired with the multiplane transesophageal probe with a rotational acquisition in steps of 2°. Gated image acquisition was done with the ECG R wave and expiration as acquisition triggers. Image loops were stored in digital format on a magneto-optical disk in the DSR file format.
After transfer of the image files to a personal computer workstation (MEDIS Imaging Systems), the data were converted to standard tagged image file format (TIFF) using the DSR2TIFF software supplied by Hewlett Packard. The images were cropped to achieve the desired image size with GraphicsConverter 2.8 software (Lemke Software) on a Macintosh PowerPC 7100 Computer. To optimize the images, postprocessing was performed with the NIH Image software package version 1.61.
To allow side-by-side viewing of normal and abnormal valves, 4 different image sequences were then assembled into 1 compound data set containing 1 loop in each image quadrant (“quad-screen format”), again with NIH Image software. For the 3D echocardiogram, the image data set was surface-rendered on a 3D echo workstation (Tomtek GmbH) to produce consecutive 3D perspective views before holographic printing.
To display motion instead of 3D anatomy by holography, the conventionally used multiple perspective views in holography of 3D objects were substituted with multiple time-consecutive views as shown in Figure 1⇓. The data were transferred to a compact prototype “one-step ultragram printer” described elsewhere,2 and angularly multiplexed “vertical parallax only” holographic stereograms were printed. In brief, a workstation (Silicon Graphics Indigo) calculates the multiple required views from the original data set and drives a liquid crystal display. Collimated light from an Nd-YAG laser is modulated by the liquid crystal display, then reshaped by a customized holographic optical element to interfere with a reference beam. A light-sensitive holographic film (DuPont Corp) records the resulting interference pattern. A flow diagram of the process of hologram production is shown in Figure 2⇓.
Quantification and Statistical Analysis
To validate the accuracy of image representation on the resulting hologram, we chose to test the accuracy of simple length measurements, because the different holograms of the mitral valve were acquired in a number of different views and by both the transthoracic and the transesophageal approaches and because virtually all anatomic quantification methods in echocardiography are based on length measurements. Therefore, the diameters and lengths of several structures visible in each individual hologram, including ventricular, annular, and atrial diameters; wall thickness; leaflet length; and leaflet thickness, were measured by independent, blinded observers. Reader 1 (M.S.C.) measured the original digital image data sets, and reader 2 (P.R.H.) measured the resulting holograms. The results of measurements on both media were then compared by correlation techniques and the Bland-Altman method. A value of P<0.05 was considered significant.
After preliminary holographic prints were created to optimize machine settings and data formatting, vertical parallax holograms were made from image loops from 7 patients with normal and abnormal mitral valve anatomy or function. The standard-size hologram was 5×8 cm. A frame of a parasternal long-axis view of a normal heart and a video sequence of the resulting dynamic hologram are shown in Figure 3⇓.
For ease of comparison, images of 4 patients with different mitral valve pathological conditions were integrated into 1 holographic print of the same size. Frames from the original images and a video sequence of the hologram resulting from these image loops are displayed in Figure 4⇓.
The cross-sectional anatomy of the mitral valve can clearly be discerned in each loop. With tilting of the hologram (or the light source) up and down, mitral valve motion becomes apparent, and motion can be stopped in any desired phase of the heart cycle. In the quad-loop display, the viewer can easily see a slow-motion display for comparison of the normal and abnormal mitral valve motion patterns.
In the normal heart (upper left quadrant of each “quad print”), tilting of the hologram leads to perception first of mitral valve closure with onset of systole, followed by aortic valve opening. At the end of systole, the aortic valve closes and the mitral valve opens again.
In the lower left quadrant, the motion of a mitral valve with myxomatous degeneration is visible: With onset of systole, the thickened mitral valve moves across the mitral annular plane to form a convex shape toward the atrium, in contrast to the simultaneously visualized normal valve in the left upper quadrant, which does not cross the mitral plane.
In the upper right quadrant, which displays a heart with hypertrophic cardiomyopathy, the mitral valve closes normally in early systole. At this point, the redundancy of the mitral leaflet is clearly appreciated compared with the normal valve. Later in systole, it becomes apparent how the mitral valve moves toward the septum and finally obstructs the outflow tract. This anterior motion creates a visible gap in the mitral valve closure, thus demonstrating the mechanism of mitral regurgitation in this disease.
The lower right quadrant displays mitral valve function in a patient with left atrial myxoma: During diastole, the large myxoma protrudes into the mitral valve orifice, leading to functional mitral stenosis.
Figure 5⇓ shows 1 frame of a 3D surface-rendered echocardiogram in a patient with a localized prolapse of the posterior leaflet and a video recording of the resulting hologram. The mitral valve is viewed en face from the left atrium. In early systole, the mitral valve closes, and a localized prolapse of the posterior mitral leaflet (with a torn chorda tendinea at its anterior edge) becomes visible. With time progression during systole, it is easy to see how the extent of the prolapse increases to reach its maximum late in systole.
With the chosen resolution of 640×480 pixels, sharpness and noise in the holograms correspond to sharpness and noise present in the original frames. (However, the video recording used here for demonstration caused a reduction in resolution and sharpness not observed in the original holograms.) Printing in higher resolution is technically possible but requires future echocardiographs to deliver more highly resolved images. Contrast in the holograms is dependent primarily on the light source used for viewing; when a strong point-light source is used, contrast is excellent, but diffuse ambient light causes significant deterioration of image contrast.
All echocardiographic measurements obtained are shown as a composite in Figure 6A⇓. A total of 41 linear measurements was obtained. Measurements on the holograms correlate well with the same measurements performed on the original digital echo frames across the whole range of clinically important dimensions (P<0.0001, r2=0.979, SEE=0.514 cm). As seen in the Bland-Altman plot (Figure 6B⇓), the measurement bias was minimal (mean bias, 0.276 cm).
After development of the method, the time required to create a typical hologram after the image acquisition was ≈5 to 10 minutes for data conversion, 5 to 10 minutes for postprocessing, and after electronic transfer of the data, ≈10 minutes for hologram printing.
Conventional holography allows representation of a static 3D object. A change in viewing angle results in a change in perspective. Dynamic holography allows the representation of motion of a 2D or 3D object in a unified and convenient format.
This is the first study demonstrating the feasibility of dynamic holographic imaging of the beating human heart. This novel technique is based on conventional display holography. With this approach, a change in the viewing angle of the printed hologram leads to the perception of time progression (cardiac motion). With this new technique, it is possible to reduce a series of images of the beating human heart to a single printable image that can be used for storage, documentation, and reporting in echocardiography. There was no alteration of image size or dimensions with this technique.
The present report of the use of dynamic holographic imaging in patients with normal and abnormal mitral valve function demonstrates the feasibility, applicability, and usefulness of this technology in clinical cardiology, particularly in valve disease, for which the visual perception of motion is fundamental in making the correct diagnosis.
Display holography uses optical interference and diffraction to record and display multidimensional images.3 In optical holography, an object is illuminated with laser light, and the scattered light is combined with a nonscattered reference laser beam.4 The resulting interference pattern contains the information regarding the 3D nature of the object. A photosensitive substrate is then exposed to the interference pattern. In contrast, we used a holographic stereogram,5 which records the interference pattern of multiple perspective views. Thus, holograms can be used with digital information without the necessity to illuminate a real object with laser light,6 because the heart is obviously not accessible for laser illumination in vivo.
Holographic images allow viewing of different perspectives of an object with different viewing angles; this is most commonly used for representation of 3 spatial dimensions. A change of perspective with change of the viewing angle in the lateral direction is encoded in the horizontal parallax of these holograms, and a change of perspective with a vertical change in viewing angle has to be encoded in the vertical parallax. For reasons of computational efficiency, holographic stereograms often use horizontal parallax only to create the illusion of depth. In our work, a different approach was chosen: The images were limited to 2D frames readily obtained by standard 2D echocardiography, and the vertical parallax was used to represent time. The vertical instead of the horizontal parallax was chosen for encoding time because otherwise the 2 eyes, with their slightly different viewing angles, would have seen 2 different time points, leading to confusion. Combining temporal information on the vertical parallax and spatial information on the horizontal parallax is feasible but increases computational requirements by ≈2 orders of magnitude. As an intermediate step, we created a dynamic hologram of a surface-rendered 3D echo, which conveys information about 3D structure, although one is not able to view the object from all perspectives.
Holography in Biomedical Imaging
Different holographic techniques have been applied in medical imaging for ≈30 years: An early report describing holography to produce 3D representation of x-ray information appeared in 1968.7 By 1969, the combination of holographic techniques with static ultrasound information was used in an attempt to detect soft tissue tumors,8 but neither the display of motion nor holographic printing was possible at that time. In addition to applications in microscopy, holography has since been applied to produce static 3D images of the pelvis,9 the spine,10 and the brain11 by use of CT,12 MRI,13 and sonographic techniques14 to acquire image information. In these applications, holography was limited to display of static 3D anatomy but not motion.
For cardiac applications, the use of holographic interferometry for examination of explanted heart valves15 and for nondestructive testing of heart valve prostheses16 has been described. Our report expands these techniques by allowing holographic representation of the entire heart during the whole cardiac cycle in vivo.
Vannan et al17 applied optical, volumetric multiplexed transmission holography to excised animal hearts imaged by ultrasound and created images representing the static 3D anatomy of these hearts. Viewing of the resulting hologram necessitated a special viewing box incorporating Fresnell lenses and diffraction gratings. Because a “virtual image” (as opposed to a “real image” in optical terms) was produced, no measurements could be performed with images produced by this technique.
In contrast to all prior reports, the technique developed at our laboratories allows motion display in addition to 2D or 3D anatomy, is applicable to patients in vivo, yet does not require any special viewing devices; quantification is easily performed directly on the holographic print.
To date, due to the complexity of the production of holograms together with the difficulty of obtaining multidimensional digital data, holographic image representation has not found widespread clinical use. The technique described here is feasible because of both the advances in digital echocardiographic imaging, which is available in most current echocardiographic equipment, and the recent progress in 1-step printers for holography.
A major limitation for broad application of the described technique at the present time is the availability of a holographic printing device. We used a prototype laser printer developed at Massachusetts Institute of Technology.2 This device allows rapid hologram printing. It is expected that similar devices will be commercially available in the near future and can be integrated into a digital echocardiography laboratory. Although even at the present time, commercially available conventional, optical hologram production techniques might be used for our purposes, they are very time-consuming and expensive; in contrast, the technique described here is very fast, and after the introduction of desktop holographic printers, it is anticipated that the cost for printing a hologram will decrease, as in the development in current office laser printers.
The time requirement to generate reports is a critical factor for the use of a new modality in a busy echocardiography laboratory: in our experience, data conversion, the change of the computer platform, and the transfer of the image data from one to the other institution were the main determinants of time consumption. Holographic printing itself was done in a matter of minutes. Direct connection of a holographic printing device to an echocardiography machine delivering digital output (eventually over a local network) will eliminate the time-consuming data transfer between different machines and allow online printing similar to that by current office laser printers within ≤10 minutes after image acquisition if technology available today is applied.
All selected data sets in our study could be transformed into a readable hologram. For optimal results, a high signal-to-noise level was useful. Postprocessing of the images to optimize overall gray level and image contrast was useful, but overall, these changes were minor and could be done as automatic preprocessing in a dedicated device in the future within seconds.
Most of the examples shown were acquired by transesophageal echocardiography to achieve optimal resolution at the level of the mitral valve. Although the quality of the holographic result certainly depends on the quality of the acquired loops, we have also obtained successful results from transthoracic acquisitions of valvular and ventricular motion.
For the described holograms, image resolution of 640×480 pixels was chosen according to current echo screens after the borders not containing image information were cropped, but printing in much higher resolution is technically possible as soon as more higher resolved ultrasound images can be delivered by clinical echocardiographs.
Holographic printing is at present essentially a 1-way method: Although the printed holograms contain the exact quantitative data in terms of anatomy, brightness, and other image information, there is no easy way to reenter the printed image data into a computer for further measurements or postprocessing. However, accurate measurements of standardized heart dimensions can be performed directly on the holographic print.
The ability to print representations of cardiac structure and motion on a 2D medium may facilitate visualization and reporting of echocardiographic studies. This method also offers advantages for storing of images: in contrast to conventional videotapes, the space needed to store a series of loops of standard echocardiographic views is decreased to a few pages. Likewise, the technique offers advantages for comparison with old data by reducing the time-consuming searches of previously recorded data on videotapes.
Dynamic holographic imaging of the beating human heart is feasible. Combined with the recent development of online holographic printing, it has the potential to improve display, storage, and reporting of image loops of the beating human heart acquired by echocardiography.
This work was supported by a grant from the Swiss National Research Foundation (Dr Hunziker).
Presented in part at the 70th Scientific Sessions of the American Heart Association, Orlando, Fla, November 10–13, 1997, and published in abstract form (Circulation. 1997;96[suppl I]:I-698).
- Received July 17, 1998.
- Revision received November 17, 1998.
- Accepted December 4, 1998.
- Copyright © 1999 by American Heart Association
Edler I, Hertz CH: The use of ultrasonic reflectoscope for the continuous recording of movements of heart walls. Kungl Fzsiogr Sallsk i Lund Forhandl. 1954;24:5.
Benton SA. Display holography: a critical review of technology. In Huff L, ed. SPIE Proc 532, Holography. Bellingham, Wash: Society of Photographic and Instrument Engineers; 1985:8–13.
Hariharan P. Optical Holography: Principles, Techniques, and Applications. Cambridge, UK: Cambridge University Press; 1984.
Halle W. The Generalized Holographic Stereogram [master’s thesis]. Cambridge, Mass: Massachusetts Institute of Technology; 1991.
Benton SA. Principles of reflection holographic stereograms. In: Jeong TH, ed. Proceedings of the Third International Symposium on Display Holography. Lake Forest, Ill: Lake Forest College; 1988:593–608.
Gautsch TL, Johnson EE, Seeger LL. True three-dimensional stereographic display of 3D reconstructed CT scans of the pelvis and acetabulum. Clin Orthop. 1994;305:138–151.
Kikinis R, Jolesz F, Gerig G, Sandor T, Cline H, Lorensen W, Halle M, Benton S. 3-D morphometric and morphologic information derived from clinical brain MR images. NATO ASI Series. 1990;441–454.
Koivukangas J, Ylitalo J, Alasaarela E, Tauriainen A. Three-dimensional ultrasound imaging of brain for neurosurgery. Ann Clin Res. 1986;47(suppl 18):65–72.