Surface Reconstruction for the masses: PialGCN for Efficient Surface Reconstruction.
Poster No:
1936
Submission Type:
Abstract Submission
Authors:
William Ashbee1, Mohamed Masoud1, Satrajit Ghosh2, Sergey Plis1
Institutions:
1Trends center, Atlanta, GA, 2Harvard/MIT, Boston, MA
First Author:
Co-Author(s):
Introduction:
Brain surface reconstruction has evolved from the traditional Freesurfer's algorithm-based approach, renowned for its accuracy but hindered by lengthy processing times, to embracing deep learning methods [1,3-8]. These advanced methods enhance speeds by utilizing GPU resources, showing promise for web deployment. Our research specifically focuses on optimizing the Pial Surface generation in PialNN through the integration of Graph Convolutional Networks (GCNs). This innovative approach not only sharpens accuracy, as indicated by reduced Hausdorff distances but also lowers memory usage. Our method, differing from PialNN's standard pointwise deformations, employs graph-aware predictions using GCNs, streamlining both the accuracy and efficiency of the network.
Methods:
Our study utilized data from 897 patients, sourced from the Human Connectome Project [2]. The patients were divided into three categories: 732 in the training set, 58 in the validation set, and 107 in the test set. Each patient's data was processed using Freesurfer.
We aimed to enhance the PialNN model's vertex-independent predictions with GCNs [3]. This enhancement involved integrating Graph Convolutional Networks (GCNs) to incorporate neighborhood data into the prediction process, thereby improving the accuracy and feature recognition across vertices. The depth of the GCNs, critical to the model's performance, was a key aspect of our investigation.
In an effort to optimize memory usage, we reduced the number of deformation modules from three to two per forward pass.
Our implementation of the GCN component utilized PyTorch Geometric's framework within the Deformation module in figure 1 [9]. We also developed graph objects using PyTorch 3D [10] to mirror the mesh connectivity accurately. The inputs for our model included transforming Freesurfer-generated white surfaces into network-generated pial surfaces.
To evaluate the effectiveness of our methodology, we conducted both qualitative and quantitative assessments. Qualitatively, we examined the vertex-wise component distances of Hausdorff distances for a single patient across all models. This involved visualizing the differences on vertex-colored brain renderings from test set predictions. Quantitatively, we measured the Hausdorff distances, averaging them over the test set. These analyses allowed us to determine the impact of GCN depth on memory usage and prediction error.
We aimed to enhance the PialNN model's vertex-independent predictions with GCNs [3]. This enhancement involved integrating Graph Convolutional Networks (GCNs) to incorporate neighborhood data into the prediction process, thereby improving the accuracy and feature recognition across vertices. The depth of the GCNs, critical to the model's performance, was a key aspect of our investigation.
In an effort to optimize memory usage, we reduced the number of deformation modules from three to two per forward pass.
Our implementation of the GCN component utilized PyTorch Geometric's framework within the Deformation module in figure 1 [9]. We also developed graph objects using PyTorch 3D [10] to mirror the mesh connectivity accurately. The inputs for our model included transforming Freesurfer-generated white surfaces into network-generated pial surfaces.
To evaluate the effectiveness of our methodology, we conducted both qualitative and quantitative assessments. Qualitatively, we examined the vertex-wise component distances of Hausdorff distances for a single patient across all models. This involved visualizing the differences on vertex-colored brain renderings from test set predictions. Quantitatively, we measured the Hausdorff distances, averaging them over the test set. These analyses allowed us to determine the impact of GCN depth on memory usage and prediction error.
Results:
Our study revealed significant improvements in brain surface reconstruction using Graph Convolutional Networks (GCNs). We found that GCNs, particularly with 2 to 6 layers, reduced memory usage during training. Moreover, all GCN models outperformed PialNN's MLP methods in accuracy.
Additionally, we observed a direct relationship between the depth of GCN layers and memory consumption.
Additionally, we observed a direct relationship between the depth of GCN layers and memory consumption.
Conclusions:
Our study marks a significant leap in brain surface reconstruction by integrating Graph Convolutional Networks (GCNs) into the PialNN framework. GCN integration has led to substantial improvements in accuracy and error reduction. The adoption of GCNs enhances the model's accuracy, particularly in capturing the intricate details of brain structures, which is crucial for neurological research and clinical applications.
However, we also noted a trade-off between increased accuracy and higher memory usage with deeper GCN layers. This balance is particularly important for applications requiring real-time processing and web deployment.
Looking ahead, this research paves the way for advanced medical imaging techniques, potentially revolutionizing data processing and interpretation in neuroscience and beyond. Our findings contribute to the evolving role of deep learning in medical imaging, moving towards more accessible, high-precision imaging solutions in both research and clinical environments.
However, we also noted a trade-off between increased accuracy and higher memory usage with deeper GCN layers. This balance is particularly important for applications requiring real-time processing and web deployment.
Looking ahead, this research paves the way for advanced medical imaging techniques, potentially revolutionizing data processing and interpretation in neuroscience and beyond. Our findings contribute to the evolving role of deep learning in medical imaging, moving towards more accessible, high-precision imaging solutions in both research and clinical environments.
Modeling and Analysis Methods:
Methods Development 1
Other Methods 2
Keywords:
Computing
Cortex
Cortical Layers
Machine Learning
MRI
STRUCTURAL MRI
White Matter
1|2Indicates the priority used for review
Provide references using author date format
Glasser, Matthew F., et al. (2013), "The minimal preprocessing pipelines for the Human Connectome Project."
Neuroimage 80: 105-124.
Ma, Qiang, et al. (2021), "PialNN: A fast deep learning framework for cortical pial surface reconstruction."
International Workshop on Machine Learning in Clinical Neuroimaging. Springer, Cham.
Bongratz, Fabian, et al. (2022), "Vox2Cortex: Fast Explicit Reconstruction of Cortical Surfaces from 3D MRI
Scans with Geometric Deep Neural Networks." Proceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition.
Cruz, Rodrigo Santa, et al. (2021), "Deepcsr: A 3d deep learning approach for cortical surface reconstruction."
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.
Hoopes, Andrew, et al. (2021), "TopoFit: Rapid Reconstruction of Topologically-Correct Cortical Surfaces."
Medical Imaging with Deep Learning.
Lebrat, Leo, et al. (2021), "Corticalflow: A diffeomorphic mesh transformer network for cortical surface
reconstruction." Advances in Neural Information Processing Systems 34 : 29491-29505.
Ma, Qiang, et al. (2022), "CortexODE: Learning Cortical Surface Reconstruction by Neural ODEs." arXiv
preprint arXiv:2202.08329.
Fey, Matthias and Jan E. Lenssen. (2019), "Fast Graph Representation Learning with PyTorch Geometric." In Proceedings of the ICLR Workshop on Representation Learning on Graphs and Manifolds.
Ravi, Nikhila, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. (2020), "Accelerating 3D Deep Learning with PyTorch3D." arXiv preprint arXiv:2007.08501.