Four-dimensional endocardial surface imaging with dynamic virtual reality rendering: a technical note

Introduction

Open heart surgery requires visualization of the internal surface of the heart (the endocardial surface) for surgical correction. High-resolution volume imaging technologies including ultrasound, magnetic resonance (MR), and computed tomography (CT) enable three-dimensional (3D) demonstration of complex cardiovascular anatomy both in static and dynamic modes (1). Dynamic 3D imaging is also known as four-dimensional (4D) imaging. During post-processing of the image data, the signal from the blood pool can be eliminated and the endocardial and endovascular surface anatomy can be selectively depicted (2,3). While post-processing of image data has traditionally been performed on a flat two-dimensional (2D) computer screen using a mouse and keyboard, it can also be performed in a 3D virtual space using hand-held devices or bare fingers (4).

In this technical note, we describe the imaging techniques, post-processing of the image data for dynamic visualization of the endocardial surface anatomy, and its application in two representative cases.

Imaging techniques

Modern imaging technologies assign signals within organs to small digitized volume units called voxels (volume elements). Voxels are the 3D analog of 2D pixels (picture elements). The size of the voxel (or pixel) sets a limit on the spatial resolution of an image. When the size of the voxel is sufficiently small relative to the object of interest, and assuming signal in the object is sufficiently different from its surroundings, the object can be confidently extracted from the surrounding structures and reconstructed three-dimensionally. The process is called 3D segmentation. Because segmentation is based on the difference in signal intensities between the object and surrounding structures, a contrast medium is typically used for both MR and CT. To image the beating heart, both MR and CT require electrocardiogram (ECG) gating to enable dynamic imaging and minimize artifacts from cardiac motion.

As advanced CT scanning requires only a fraction of a second to a few seconds, imaging is performed during dynamic passage of an intravenously injected iodine-based contrast medium during shallow breathing or a short breath-hold. As post-processing relies largely on thresholding, it is crucially important to time the image acquisition with the iodine contrast bolus, when the regions of interest are densely and homogeneously opacified. With CT, it is also important to avoid streak artifacts from undiluted contrast medium in the superior or inferior vena cava, and/or systemic venous atrium by using diluted contrast medium and saline chasers following the injection of undiluted contrast medium. Dynamic CT imaging can therefore be a complex procedure in congenital heart disease.

Image acquisition with MR is slower than with CT and the process of acquiring 4D images requires several minutes. With conventional MR contrast agents, this time window is not compatible with the desired dense and homogeneous enhancement of all cardiac chambers. However, ferumoxytol (Feraheme, Covis Pharma, Waltham, MA, USA) is an iron-based nanoparticle that has a long intravascular half-life on the order of 14–15 hours (2), such that once injected it enhances the entire blood pool and cardiac chambers for the duration of any practical MR acquisition. ECG-gated 4D images can be acquired over the span of several minutes without breath holding and with respiratory gating to minimize breathing motion artifacts. While the spatial resolution of conventional MR angiography is lower than that of CT angiography, MR angiography with ferumoxytol enables steady-state homogeneous opacification of all cardiac chambers and sizable vessels throughout the body, pushing the boundaries for contrast and resolution in children.

Dynamic MR imaging can also be performed using conventional multi-slice 2D cine-imaging. By acquiring the images without interslice gaps, cine-images can be reconstructed three-dimensionally. Although the current cine imaging technique is limited by non-isotropic voxel size, the results of dynamic 3D reconstruction are acceptable.

Ultrasound is also an excellent dynamic 3D imaging technology. However, ultrasound imaging is associated with abundant artifacts from tissue interfaces, bones, and air. Therefore, its use for 3D modeling is limited to the central areas of the heart including cardiac valves that are not associated with significant artifacts.

Post-processing (Figure 1, Video S1)

Figure 1 Post-processing steps for endocardial surface visualization (source images, ECG-gated CT angiograms in an 8-year-old child with a tortuous aortic arch). 3D, three-dimensional; ECG, electrocardiogram; CT, computed tomography.

A commercially available virtual reality (VR) software program (Elucis, Realize Medical, Ottawa, Canada) was set for dynamic 3D visualization. The VR system consisted of a desk-top or notebook computer (Intel Core i7-6700K, 32 GB RAM, NIVIDA RTX 3080), and Meta Quest 2 glasses (Meta, Menito Park, CA, USA). The regions of interest were segmented by carefully defining a range of signal intensity for thresholding. Once the ideal range of signal intensity for thresholding was identified, segmentation process propagates across other locations within the imaging volume and also to the imaging volumes of other phases of the cardiac cycle. This process was able to be facilitated by displaying the 2D images in a fast cine-mode and scrolling the images up and down while the segmentation tool is set active.

Demonstration of the endocardial surface was then achieved by adjusting the range of signal intensity such that the blood pool was eliminated and the endocardial and endovascular surface is directly visualized. Alternatively, the blood pool was segmented for demonstration of the surface of the cavity. Then, a surface shell with an even thickness was added to the segmented blood pool. Although the demonstrated internal anatomy was essentially the same, the latter method was preferred as the external contour looked cleaner and the subsequent editing steps were easier.

Visualization of the endocardial surface was further improved through the correction of inadequate representation of the anatomy due to image artifacts or heterogeneity of the signal intensity. The areas of inadequate representation were modified and sculpted based on the information shown in the images as well as the operator’s knowledge and experience.

The most difficult structures to visualize using threshold-based segmentation were the cardiac valve leaflets and the chordae tendinae of the atrioventricular valves. These structures were relatively small and/or thin, and continuously deformed and changed their positions during ventricular contraction and relaxation. They were only partially visualized because of the partial volume effect on the small and/or mobile structures. Although it was far from perfect, the valve leaflets and chordae were segmented separately by using a different range of signal intensity for the partially volumed structures of interest.

Once the segmented and edited endocardial surface of the heart was reviewed interactively by moving and rotating the reconstructed model and cutting the surface planes to reveal the internal structures. The views were then recorded as video clips or still frames for further discussion with clinicians and surgeons.

The post-processing steps described above were performed using either computer screen-based software (Mimics version 24 and 3-Matic version 16, Materialise, Leuven, Belgium) or primarily space-based software using VR. For dynamic 4D imaging, the space-based VR approach was advantageous over the traditional computer screen-based approach. VR enables extremely fast and efficient segmentation of the regions of interest during dynamic cine-display and unlimited interactive scrolling of the sectional imaging planes through the regions of interest. In addition, the segmented models were further edited interactively in either 2D or 3D mode using adding, removing, and thresholding tools.

Clinical case examples

Case 1 (Figure 2 and Video S2): perimembranous ventricular septal defect with major extension toward the outlet of the right ventricle in a 15-day-old newborn infant.

Figure 2 Case 1. Perimembranous VSD (source images, ECG-gated and respiration-navigated, ferumoxytol-enhanced, dynamic 3D MR angiograms). The sites of cardiac valvar attachment were traced using a computer-screen-based software (Mimics, Materialise NV, Leuven, Belgium), and the traces were overlain on the post-processed images. The TV, MV, aortic and pulmonary valvar annuli are colored deep blue, red, green and sky blue, respectively. To demonstrate the relationship between the margin of the VSD and tricuspid valve leaflets, the leaflets (shown in dark green color) were segmented separately using a different threshold range. The valve leaflets could not be visualized in their entirety because of the limited temporal resolution of the imaging technique. Unusually, the aorta is supported by muscular infundibulum with a gap between the aortic and mitral valve annuli (double-headed arrow). RV, right ventricle; Ao, aorta; ASD, atrial septal defect; VSD, ventricular septal defect; RA, right atrium; PA, pulmonary artery; RV, right ventricle; LV, left ventricle; TV, tricuspid valve; MV, mitral valve; ECG, electrocardiogram; 3D, three-dimensional; MR, magnetic resonance.

Case 2 (Figure 3 and Video S3): tricuspid valve regurgitation in a 3-year-old patient with hypoplastic left heart syndrome who underwent bidirectional cavopulmonary connection.

Figure 3 Case 2. Postoperative TR in hypoplastic left heart syndrome with previous bidirectional cavopulmonary anastomosis (source images, conventional cine MR images acquired without interslice gap). Cine images were obtained for the tricuspid valve without interslice gaps. The in-plane resolution was 1.1 mm × 1.1 mm and the slice thickness was 3 mm. The endocardial surface of the cardiac chambers and the tricuspid valve leaflets were reproduced as described above. The signal void area of the regurgitant jet in the RA was segmented using a different signal intensity range. The segmented regurgitant jet flows were overlain on the endocardial surface images. RA, right atrium; RV, right ventricle; PM-A, anterior papillary muscle; TR, tricuspid regurgitation; TV, tricuspid valve; MR, magnetic resonance.

Discussion

Dynamic imaging provides further insights into proper understanding of cardiovascular anatomy and function (2,3). While visualization of the endocardial surface is ideally suited for the planning of open-heart surgery, it is a time-consuming process to dynamically visualize the endocardial surface using conventional computer-screen-based software. VR enables extremely fast and reliable post-processing. The major advantage of VR is the use of two hand-held devices for post-processing. In a computer-screen-based approach, the operator’s hands and fingers are, in some sense, tied as they are used for handling a mouse and a keyboard. In the VR approach, the operator’s both hands are freed in an open space as multiple fingers of both hands can be used for multiple tasks at the same time. Therefore, VR is a powerful and efficient post-processing tool for interactive endocardial surface visualization. In the authors’ experience, the observers were able to develop acceptable skills for navigation and post-processing with hand-held devices after a few attempts.

Using currently available imaging and post-processing technologies, the visualization of the ventricular endocardial surface anatomy is sufficiently accurate (1). The atrial septum can also be visualized well but its thin thickness does not allow accurate assessment of its anatomy. Although visualization of the cardiac valves is far from satisfactory, the gross outline and orientation of the valve leaflets could be shown when needed. In our limited experience, the leaflets appear thick and irregular when they are moving. During a quiescent period, the leaflets are too thin to be properly segmented. Further improvement of spatial, temporal, and contrast resolutions of imaging techniques will eventually allow proper visualization of the valve leaflets and chords in the future.

Mixed reality (MixR) such as HoloLens (Microsoft, USA) is a blend of VR and real world that enables overlaying of the imaging object within or on the real objects such as the patient’s body and operating table (5). Although MixR enables a different level of reality simulation, there are limitations including a smaller field of view, less efficient and less precise processing capability resulting in less clear and effective visualization. MxR also requires more expensive hardware as compared to VR.

Conclusions

VR-based dynamic visualization of the cardiovascular structures is feasible and provides better functional insights into the pathologic lesions such as changes in size and shape of the ventricular septal defects and the location and mechanism of valvular regurgitation in comparison to conventional static imaging and visualization or 3D printing. As VR is based on the use of all fingers of both hands, the efficiency and speed of postprocessing are markedly enhanced. Despite such additional attributes of VR, however, the details and accuracy of VR-based segmentation process are limited as compared to conventional computer screen-based segmentation.

Acknowledgments

Funding: This study was supported by NIH (R01 HL127153-06 to J.P.F.).

Footnote

Provenance and Peer Review: This article was commissioned by the Guest Editors (Antonio F. Corno & Ali Dodge-Khatami) for the column “Pediatric Heart” published in Translational Pediatrics. The article has undergone external peer review.

Peer Review File: Available at https://tp.amegroups.com/article/view/10.21037/tp-24-140/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://tp.amegroups.com/article/view/10.21037/tp-24-140/coif). The column “Pediatric Heart” was commissioned by the editorial office without any funding or sponsorship. J.P.F. received support from NIH for development of the 4D Cardiac MRI technique used to generate the input image data for this work (No. R01 HL127153-06). He is also a consultant for Covid Pharma from May 2023 on an ad hoc basis, for which he has not yet performed any service or received any fees. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Written informed consent was obtained from the patients’ parents or guardians at the time of imaging for utilization of the acquired images for research purposes.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Nguyen KL, Ghosh RM, Griffin LM, et al. Four-dimensional Multiphase Steady-State MRI with Ferumoxytol Enhancement: Early Multicenter Feasibility in Pediatric Congenital Heart Disease. Radiology 2021;300:162-73. [Crossref] [PubMed]
2. Yoo SJ, Hussein N, Peel B, et al. 3D Modeling and Printing in Congenital Heart Surgery: Entering the Stage of Maturation. Front Pediatr 2021;9:621672. [Crossref] [PubMed]
3. Priya S, Nagpal P. Four-dimensional virtual reality cine cardiac models using free open-source software. Pediatr Radiol 2020;50:1617-23. [Crossref] [PubMed]
4. Chessa M, Van De Bruaene A, Farooqi K, et al. Three-dimensional printing, holograms, computational modelling, and artificial intelligence for adult congenital heart disease care: an exciting future. Eur Heart J 2022;43:2672-84. [PubMed]
5. Lau I, Gupta A, Ihdayhid A, et al. Clinical Applications of Mixed Reality and 3D Printing in Congenital Heart Disease. Biomolecules 2022;12:1548. [Crossref] [PubMed]