Automated Pituitary and Pineal Gland Segmentation with Multi-Modal Input Channels
Poster No:
2012
Submission Type:
Abstract Submission
Authors:
Kathleen Larson1, Avnish Kumar1, Jean Augustinack1, Jocelyn Mora1, Devani Shahzade1, Bruce Fischl1, Douglas Greve1
Institutions:
1Athinoula A. Martinos Center for Biomedical Imaging, MGH, HMS, Boston, MA
First Author:
Co-Author(s):
Introduction:
The pituitary and pineal glands are two subcortical structures that play pivotal roles in the human endocrine system. The pituitary gland is a bilobular organ connected to the hypothalamus via the infundibulum and secretes hormones such as human growth hormone and oxytocin (Hall 2016, chap. 76). While much less is known about the pineal gland, it is thought to control melatonin secretion and various aspects of sexual function (Hall 2016, chap. 81). Accurate segmentation of both structures is critical for imaging-based analyses related to endocrine disorders. Here, we present a supervised, deep learning-based method for automatic segmentation of the pituitary and pineal glands. This work is based on a previous segmentation frameworks for the hypothalamus (Billot et al. 2020) and subcortical limbic system (Greve et al. 2021).
Methods:
Input data: Our dataset comprised n=32 subjects from the FreeSurfer Maintenance (FSM) Grant Data ("FsmData - Free Surfer Wiki" n.d.). Each subject was associated with four MRI modalities: T1-weighted (T1w), T2-weighted (T2w), quantitative T1 (qT1), and proton density (PD). The anterior and posterior pituitary lobes, the infundibulum, and the pineal gland were manually labeled by two labelers to provide 64 labeled data sets. (Figure 1)
Data augmentation: We applied a series of randomized, spatial and intensity augmentation functions to the input data during training. These include image cropping to a 100x100x100 window, an affine transformation, left-right flipping, a gamma transform for contrast alteration, bias field simulation, Gaussian white noise addition, and min-max normalization. Augmentation (aside from cropping and rescaling) began after 50 training epochs to allow proper initialization of network hyperparameters.
Network architecture: Based on the methods, we trained a U-Net to perform automatic segmentation of each label similar to the methods described in (Greve et al. 2021). Our network consisted of three layers, each with two convolution blocks composed of a 3x3x3 convolution and ELU activation. We began with 24 feature maps in the first layer; these were doubled during each decoding layer, and then halved during each encoding layer. At the end of each layer, we performed either max-pooling (encoding branch) or max-unpooling (decoding branch).
Experimental validation: We trained 12 different configurations of our U-Net, each with a different combination of MRI modalities as separate input channels. Training was performed using 80% of the data (n=52) for 2000 epochs using a combination of Dice and categorical cross entropy (CCE) loss functions.
Data augmentation: We applied a series of randomized, spatial and intensity augmentation functions to the input data during training. These include image cropping to a 100x100x100 window, an affine transformation, left-right flipping, a gamma transform for contrast alteration, bias field simulation, Gaussian white noise addition, and min-max normalization. Augmentation (aside from cropping and rescaling) began after 50 training epochs to allow proper initialization of network hyperparameters.
Network architecture: Based on the methods, we trained a U-Net to perform automatic segmentation of each label similar to the methods described in (Greve et al. 2021). Our network consisted of three layers, each with two convolution blocks composed of a 3x3x3 convolution and ELU activation. We began with 24 feature maps in the first layer; these were doubled during each decoding layer, and then halved during each encoding layer. At the end of each layer, we performed either max-pooling (encoding branch) or max-unpooling (decoding branch).
Experimental validation: We trained 12 different configurations of our U-Net, each with a different combination of MRI modalities as separate input channels. Training was performed using 80% of the data (n=52) for 2000 epochs using a combination of Dice and categorical cross entropy (CCE) loss functions.
·Figure 1: Input modalities and manual segmentations for an example subject.
Results:
Figure 2 displays the mean Dice scores for each label obtained using the different sets of input modalities. Overall, all models performed best in the anterior pituitary and worst in the infundibulum. We observed that when using a single input channel, the T1w image provided the best segmentations over the T2w, qT1, and PD modalities. Moreover, using a combination of T1w and PD, or T1w and at least two additional input channels, offered a slight improvement over T1w alone. In these cases, all network configurations achieved a mean Dice score in each label of greater than 83% in the anterior pituitary, and greater than 75% in the posterior pituitary, infundibulum, and pineal gland.
·Figure 2: Mean Dice scores within each label obtained using networks trained with different combinations of input modalities.
Conclusions:
We have presented an automated, U-Net-based framework for segmentation of the pituitary and pineal glands. We showed that utilizing multiple MRI modalities as separate input channels to our network improved segmentation results compared to use of a single input image.
Modeling and Analysis Methods:
Methods Development
Segmentation and Parcellation 1
Neuroanatomy, Physiology, Metabolism and Neurotransmission:
Subcortical Structures 2
Keywords:
Machine Learning
Segmentation
STRUCTURAL MRI
Sub-Cortical
1|2Indicates the priority used for review
Provide references using author date format
“FsmData - Free Surfer Wiki.” n.d. Accessed November 30, 2023. https://surfer.nmr.mgh.harvard.edu/fswiki/FsmData.
Greve, Douglas N., Benjamin Billot, Devani Cordero, Andrew Hoopes, Malte Hoffmann, Adrian V. Dalca, Bruce Fischl, Juan Eugenio Iglesias, and Jean C. Augustinack. 2021. “A Deep Learning Toolbox for Automatic Segmentation of Subcortical Limbic Structures from MRI Images.” NeuroImage 244 (December): 118610. https://doi.org/10.1016/j.neuroimage.2021.118610.
Hall, John E. 2016. Guyton and Hall Textbook of Medical Physiology. Philadelphia, PA: Elsevier.