BrainVR: A Virtual Reality System for Neurology Education
Poster No:
1844
Submission Type:
Abstract Submission
Authors:
Gonzalo Rojas1, Jorge Fuentes1, Carlos Montoya1, Evelyng Faure1, Maria de la Iglesia-Vayá2
Institutions:
1Clinica las Condes, Santiago, RM, 2Join Unit FISABIO-CIPF, Valencia, Valencia
First Author:
Co-Author(s):
Introduction:
Visualization of complex neuroimaging images such as brain parcellation, functional brain connectivity, diffusion tensor imaging, functional imaging and combination of these techniques is a difficult task because they are 3D structures. Different advanced visualization solutions have been proposed in previous work (Stereoscopy: Rojas, 2014, virtual reality (VR)-based systems to view complex neuroimages: Rojas, 2015; 2016; 2017). Keiriz et al (2018) created a CAVE system to explore graph representations of functional connectivity data. The various visualization solutions have different costs and technical characteristics.
VR is a 3D environment of scenes or objects of real appearance, created by computer technology, which creates the feeling of being immersed in it. This environment is visualized by glasses or VR helmet. Oculus Quest (Oculus VR) is a VR headset, fully standalone, with two controllers.
Here we describe a fully controllable immersive 3D VR system for educational purposes (medical doctors or other healthcare professionals), that shows skull, pial cortex, main subcortical structures (Fischl, 2012), and main brain tracts (Oishi, 2010).
VR is a 3D environment of scenes or objects of real appearance, created by computer technology, which creates the feeling of being immersed in it. This environment is visualized by glasses or VR helmet. Oculus Quest (Oculus VR) is a VR headset, fully standalone, with two controllers.
Here we describe a fully controllable immersive 3D VR system for educational purposes (medical doctors or other healthcare professionals), that shows skull, pial cortex, main subcortical structures (Fischl, 2012), and main brain tracts (Oishi, 2010).
Methods:
The following medical images was acquired to a female 38 years old normal volunteer:
i) CT: 128-Slice SOMATOM Definition Flash (Siemens, Germany), protocol B70s (120 kVp, effective mAs 280, reference mAs 356, Gantry Tilt 0 rad, slice thickness 1.0 mm, voxel size: 1.0x1.0 mm; pitch, 0.9)
ii) 3D MPRAGE MRI: T1-weighted sequence, voxel dimensions: 1.0 x 1.0 x 1.0 mm3; image dimensions: 256 x 256 x 192, TE = 2.26 ms, TR = 2400 ms, flip angle = 8o . 3.0 T Skyra (Siemens, Germany)
iii) DTI MRI: single-shot diffusion-weighted spin-echo EPI sequence, TR = 10300 ms, TE = 95 ms, matrix = 122 × 122, FOV = 244 × 244 mm, slice thickness 2.0 mm, 50 contiguous sections, b = 1000 s/mm2, 64 non-collinear directions. 3.0 T Skyra.
Using CT volumetric image, a mesh of the skull was created using 3DSlicer 4.10.0 software (Segment Editor module, Threshold range 99 to 3068, other parameters default values).
Using 3D MPRAGE image and FreeSurfer 5.3 software, Pial mesh and cortical and subcortical structures segmentation was computed. 3D Slicer was used to do the corregistration between skull and pial meshes, and to create subcortical related mesh models (Model Maker module using marching cubes, Laplacian filter type and default values for other parameters).
Tractography of corpus callosum was computed using 3DSlicer Tractography Seeding module with default parameters and ROI in corpus callosum.
Quadric edge collapse decimation algorithm implemented in MeshLab 2016 software, was used to reduce the quantity of faces to 50.000 (in pial and skull meshes), with "preserve topology" option "on", and other algorithm options with default values (Fig 1).
3dsMax sofware (Autodesk Inc. www.autodesk.com) was used to combine all meshes in one FBX file to import in Unity 2017.4.10f1 LTS, and the software was created using C# programming language.
Oculus Quest 128GB VR glasses (www.oculus.com/quest) was used in this system and the complete software was loaded to the glasses memory (Fig 2).
i) CT: 128-Slice SOMATOM Definition Flash (Siemens, Germany), protocol B70s (120 kVp, effective mAs 280, reference mAs 356, Gantry Tilt 0 rad, slice thickness 1.0 mm, voxel size: 1.0x1.0 mm; pitch, 0.9)
ii) 3D MPRAGE MRI: T1-weighted sequence, voxel dimensions: 1.0 x 1.0 x 1.0 mm3; image dimensions: 256 x 256 x 192, TE = 2.26 ms, TR = 2400 ms, flip angle = 8o . 3.0 T Skyra (Siemens, Germany)
iii) DTI MRI: single-shot diffusion-weighted spin-echo EPI sequence, TR = 10300 ms, TE = 95 ms, matrix = 122 × 122, FOV = 244 × 244 mm, slice thickness 2.0 mm, 50 contiguous sections, b = 1000 s/mm2, 64 non-collinear directions. 3.0 T Skyra.
Using CT volumetric image, a mesh of the skull was created using 3DSlicer 4.10.0 software (Segment Editor module, Threshold range 99 to 3068, other parameters default values).
Using 3D MPRAGE image and FreeSurfer 5.3 software, Pial mesh and cortical and subcortical structures segmentation was computed. 3D Slicer was used to do the corregistration between skull and pial meshes, and to create subcortical related mesh models (Model Maker module using marching cubes, Laplacian filter type and default values for other parameters).
Tractography of corpus callosum was computed using 3DSlicer Tractography Seeding module with default parameters and ROI in corpus callosum.
Quadric edge collapse decimation algorithm implemented in MeshLab 2016 software, was used to reduce the quantity of faces to 50.000 (in pial and skull meshes), with "preserve topology" option "on", and other algorithm options with default values (Fig 1).
3dsMax sofware (Autodesk Inc. www.autodesk.com) was used to combine all meshes in one FBX file to import in Unity 2017.4.10f1 LTS, and the software was created using C# programming language.
Oculus Quest 128GB VR glasses (www.oculus.com/quest) was used in this system and the complete software was loaded to the glasses memory (Fig 2).
·Fig. 1. Brain pial cortex, subcortical structures, and corpus callosum tract models.
·Fig. 2. Oculus Quest 128GB VR glasses and both controllers.
Results:
Using VR glasses it is possible to view the skull, brain, corpus callosum tracts and some subcortical structures in his anatomical position in 3d with inmersion characteristics. It is possible to rotate the brain (using VR glasses controls and user head), and the user could immerse in the brain and other structures. The user can make each structure appear or disappear using the button of the VR glasses controls. Also a neurologic description of the structure will appear if the user select the respective structure.
Conclusions:
The software/hardware described here, shows the use of VR glasses for brain anatomy and physiology education for healthcare professionals.
More work must be done in the system, by adding more tracts to it (Oishi, 2010), and 7 functional connectivity networks (Yeo, 2011).
More work must be done in the system, by adding more tracts to it (Oishi, 2010), and 7 functional connectivity networks (Yeo, 2011).
Neuroanatomy, Physiology, Metabolism and Neurotransmission:
Anatomy and Functional Systems 1
Cortical Anatomy and Brain Mapping
Subcortical Structures
White Matter Anatomy, Fiber Pathways and Connectivity
Neuroinformatics and Data Sharing:
Informatics Other 2
Keywords:
Computational Neuroscience
Neurological
NORMAL HUMAN
Structures
Sub-Cortical
Tractography
Other - virtual reality
1|2Indicates the priority used for review
My abstract is being submitted as a Software Demonstration.
Please indicate below if your study was a "resting state" or "task-activation” study.
Healthy subjects only or patients (note that patient studies may also involve healthy subjects):
Was any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.
Was any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.
Please indicate which methods were used in your research:
For human MRI, what field strength scanner do you use?
Which processing packages did you use for your study?
Provide references using author date format
Keiriz, J.J.G., Zhan, L., Ajilore, O., Leow, A.D., Forbes, A.G., (2018), ‘NeuroCave: A web-based immersive visualization platform for exploring connectome datasets’, Netw Neurosci, vol. 2, no. 3, pp. 344-361.
Fischl, B., (2012), ‘FreeSurfer’, Neuroimage, vol. 62, no. 2, pp. 774-781.
Rojas, G.M., Gálvez, M., Vega Potler, N., Craddock, R., Margulies, D.S., Castellanos, F.X., Milham, M.P., (2014), ‘Stereoscopic Three-Dimensional Visualization Applied to Multimodal Brain Images: Clinical Applications and a Functional Connectivity Atlas’, Front. Neurosci, 8: pp. 328.
Rojas, G.M., Fuentes, J., Gálvez, M., Margulies, D.S., (2015), ‘Augmented Reality rsfc-MRI Visualization Application: ARiBraiN3T, iBraiN, iBraiNEEG: New Versions’, 21th Annual Meeting of the Organization for Human Brain Mapping, Honolulu, Hawaii, USA.
Rojas, G.M., Fuentes, J., Montoya, C., Galvez, M., (2016), ‘Virtual reality intrinsic functional connectivity visualization application for mobile devices: VRiBraiN’, 5th Biennial Conference on Resting State and Brain Connectivity, Vienna, Austria.
Rojas, G.M., Fuentes, J.A., Montoya, C., de la Iglesia-Vaya, M., Gálvez, M., (2017), ‘Visualization of Functional Connectivity Networks using VR glasses’, 22nd Annual Meeting of the Organization for Human Brain Mapping, Vancouver, Canada.
Yeo, B.T., Krienen, F.M., Sepulcre, J., Sabuncu, M.R., Lashkari, D., Hollinshead, M., Roffman, J.L., Smoller, J.W., Zöllei, L., Polimeni, J.R., Fischl, B., Liu, H., Buckner, R.L., (2011), ‘The organization of the human cerebral cortex estimated by intrinsic functional connectivity’, J Neurophysiol, vol. 106, no. 3, pp. 1125-1165.