Skullstripping anatomical MRIs in AFNI using volumetric deep learning network: Preliminary results
Poster No:
1436
Submission Type:
Abstract Submission
Authors:
Yamuna Narayana Swamy1, Patrick McClure2, Richard Reynolds3, Francisco Pereira3, Paul Taylor3
Institutions:
1Scientific and Statistical Computing Core, NIMH, NIH, USA, 2Naval Postgraduate School, Monterey, CA, 3National Institute of Mental Health, Bethesda, MD
First Author:
Co-Author(s):
Introduction:
Identifying the brain within an anatomical MRI (i.e., skullstripping or brain extraction) is a standard step in many neuroimaging analysis pipelines. While it is generally easy to tell by eye which voxels belong to the brain and which do not, it is surprisingly difficult to automate. There are existing tools that do this well for good quality data with standard characteristics, but the challenge is to perform this task robustly across variations in physiology, cortical thickness, or other individual features, as well as the many types of noise or image distortions such as ringing, noise, dropout, and brightness inhomogeneity. In this work we demonstrate a new 3D skullstripping tool in AFNI [1] using a volumetric, convolutional neural network (V-net) that can estimate detailed brain masks for raw-to-minimally processed human anatomical datasets.
Methods:
A V-net is a volumetric neural network that is particularly suited to whole brain classification due to its in-built 3D convolutional kernel. Milletari et. al. [2] applied a V-net to MRI volumes for prostate segmentation, and Bontempi et al. [3] used a modified version for brain tissue classification. Here, we adopt this framework as a starting point for skullstripping, using a larger number of layers to perform a 2-channel classification problem: to identify "brain" and "non-brain". The architecture contains both a volumetric down-sampling path and a volumetric up-sampling path.
We implemented V-net in Python using PyTorch [4]. We trained it end to end to predict binary masks from T1-weighted (T1w) anatomical MRI volumes, using Sorensen-Dice loss [5]. The training dataset masks were created by using FreeSurfer [6] on good quality T1w volumes. To increase robustness and generalizability of the V-net, we augmented the training data by using AFNI to derive volumes with the following characteristics: Gibbs ringing, gain inhomogeneity, zipper noise, strong shading and affine transforms. The output of the V-net is a softmax layer, which provides a probability map of voxels belonging to foreground or background; specifically, it is a two channel volumetric segmentation corresponding to brain and the non-brain.
The training and testing data were T1w volumes (1mm isotropic voxels) from across 8 sites on 3 continents with a large age range (8-70 yrs) and different 3T scanner types [7]. Here, the network was trained for 100 epochs on 131 original plus 72 augmented datasets, and tested on 38 datasets.
We implemented V-net in Python using PyTorch [4]. We trained it end to end to predict binary masks from T1-weighted (T1w) anatomical MRI volumes, using Sorensen-Dice loss [5]. The training dataset masks were created by using FreeSurfer [6] on good quality T1w volumes. To increase robustness and generalizability of the V-net, we augmented the training data by using AFNI to derive volumes with the following characteristics: Gibbs ringing, gain inhomogeneity, zipper noise, strong shading and affine transforms. The output of the V-net is a softmax layer, which provides a probability map of voxels belonging to foreground or background; specifically, it is a two channel volumetric segmentation corresponding to brain and the non-brain.
The training and testing data were T1w volumes (1mm isotropic voxels) from across 8 sites on 3 continents with a large age range (8-70 yrs) and different 3T scanner types [7]. Here, the network was trained for 100 epochs on 131 original plus 72 augmented datasets, and tested on 38 datasets.
Results:
Most skullstripping algorithms perform well in the "core" brain region, i.e., away from the boundary. In our evaluations, we have computed the performance metrics around the brain edge where algorithms vary the most, defining an "inner" and "outer" rim, using AFNI's 3dDepthMap (Fig 1a). The true positive rate (TPR) was computed in the inner rim (Fig 1b), and the true negative rate (TNR) was computed for the outer rim (Fig 1c). In both cases, the typical V-net results closely match the initial mask datasets. Visually checking example subjects, we can see multiple cases where differences are due to the V-net providing more accurate masks to the underlying anatomy than FreeSurfer (Fig 2), e.g., see the arrows in Fig 2a. The V-net performs well in identifying the brain region in a dataset having considerable ringing artifacts (Fig 2b).
Conclusions:
The preliminary results are encouraging for this architecture to classify the brain from anatomical MRI volumes. Even with a small training dataset, the V-net shows high accuracy for standard datasets, and also shows some advantages over traditional skullstripping approaches in cases of lower data quality, such as ringing. In future work we plan to expand the training dataset size, as well as augmented sample size and variety. Finally, we also plan to expand the application to multiclass (e.g., tissue) classification[8].
Modeling and Analysis Methods:
Classification and Predictive Modeling 1
Methods Development 2
Keywords:
Computing
Data analysis
Machine Learning
Open-Source Software
1|2Indicates the priority used for review
Provide references using author date format
[2] Fausto Milletari et al.,(2016), “V-Net : Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation”, Fourth International Conference on 3D Vision (3DV)
[3] D. Bontempi et al.,(2020), “CEREBRUM: a fast and fully-volumetric convolutional Encoder-decodeR for weakly-supervised sEgmentation of BRain strUctures from out-of-the-scanner MRI” Med. Image Anal., 62 (2020)
[4] Paszke, A. et al.,(2019), PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., pp. 8024–8035.
[5] Dice L.,(1945), Measures of the amount of ecologic association between species. Ecology, 26:207–302
[6] Fischl B, et al. “Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain.” Neuron vol. 33,3 (2002): 341-55. doi:10.1016/s0896-6273(02)00569-x
[7] Taylor PA, Glen DR, Reynolds RC, Basavaraj A, Moraczewski D, Etzel JA. (May 2023), Editorial: Demonstrating quality control (QC) procedures in fMRI., Frontiers in Neuroscience.
[8] McClure P, Rho N, Lee JA, Kaczmarzyk JR, Zheng CY, Ghosh SS, Nielson DM, Thomas AG, Bandettini P, Pereira F (2019) “Knowing what you know in brain segmentation using Bayesian deep neural networks”. Frontiers in neuroinformatics.