Multi-task Learning framework for Brain Tumor Analysis with Uncertainty Estimation in MRI Images
Poster No:
2297
Submission Type:
Abstract Submission
Authors:
Maria Nazir1, Sadia Shakil2, khurram khurshid1
Institutions:
1Institute of space technology Islamabad, islamabad, Islamabad, 2The chinese University of Hong Kong, Hongkong, hongkong
First Author:
Co-Author(s):
Introduction:
Gliomas [1] are one of the deadliest brain tumors with extremely difficult diagnosis due to their complex behavior and irregular appearance. Magnetic Resonance Imaging (MRI) is the most famous imaging modality used for detecting tumors like gliomas. The manual segmentation of brain tumors in MRI images is a laborious task, which gives rise to subjective as well as objective errors [1]. To cater for all these issues, a lot of research has been done and is going on to develop AI-based solutions that can help doctors and radiologists in early and effective diagnosis of gliomas. However, an end-to-end system is still missing. In this research, an all-in-one Multi-Task Learning (MTL) [2] framework for complete analysis of gliomas with uncertainty estimation has been proposed.
Methods:
An end-to-end 3D MTL model with encoder as the shared backbone was developed that can give predictions for three tasks simultaneously. It can classify the tumor as High-grade glioma (HGG) / Low-grade glioma (LGG) , provides multi-class segmentation of the lesion area into three classes (Enhancing Tumor, Non-Enhancing Tumor and Whole Tumor), and predicts the overall survival of glioma patients in days by leveraging task relationships between similar tasks. Each task was executed using separate task specific layers that used the feature representation from the shared layers.
Loss optimization is the most critical factor in MTL frameworks [2]. Three different losses were used for three predictions and are combined to form one aggregated loss. RMSprop was used for loss optimization during back propagation by updating weights in each task specific layers and the shared layers. Loss weights are the most important hyper-parameters in MTL framework that can be tuned to minimize the overall loss and increase efficiency. Three weights for three different losses were used i.e., W1 = 0.01 for classification, W2=10 for segmentation and W3=0.001 for survival in days were found to be the best weight factors for the designed setup.
Pre-processed Brain Tumor Segmentation challenge [3] (BraTS 2019 and 2020) datasets with combination of 1, 2 and 4 MRI sequences (T1, T2, Flair and T1CE) were used for experimental purposes. We further did some more pre-processing i.e., cropping, re-sizing to reduce the size and normalization to standardize the data. The model was tested on three different batch sizes (32, 64 and 128) .. Prediction performance of all three tasks for each case was compared with state-of-the-art as well as MTL related studies.
Loss optimization is the most critical factor in MTL frameworks [2]. Three different losses were used for three predictions and are combined to form one aggregated loss. RMSprop was used for loss optimization during back propagation by updating weights in each task specific layers and the shared layers. Loss weights are the most important hyper-parameters in MTL framework that can be tuned to minimize the overall loss and increase efficiency. Three weights for three different losses were used i.e., W1 = 0.01 for classification, W2=10 for segmentation and W3=0.001 for survival in days were found to be the best weight factors for the designed setup.
Pre-processed Brain Tumor Segmentation challenge [3] (BraTS 2019 and 2020) datasets with combination of 1, 2 and 4 MRI sequences (T1, T2, Flair and T1CE) were used for experimental purposes. We further did some more pre-processing i.e., cropping, re-sizing to reduce the size and normalization to standardize the data. The model was tested on three different batch sizes (32, 64 and 128) .. Prediction performance of all three tasks for each case was compared with state-of-the-art as well as MTL related studies.
Results:
Fig. 1. shows the proposed 3D MTL architecture consisting of input block for pre-processing of MRI images, shared block for extracting features and finally the output block consisting of task specific layers that use the extracted features for multiple predictions. Fig. 2. shows the results obtained for multi-class segmentation with predicted mask, ground truths and uncertainty of the model for each case. Results show 95% accuracy, 86% dice and mean absolute error of 456.59 days for a combination of all four sequences. Uncertainty maps show some border pixels where the model is uncertain thereby enforcing the radiologists to pay more attention to lesion border for accurate prediction.
Conclusions:
We developed an all-in-one end-to-end framework that can provide multiple predictions for complete glioma analysis using the least resources (data, computation, inference time). Publicly available BraTS (2019 and 2020) datasets were used for experimentation. It is evident from the results that deep learning based multi-task learning frameworks have the potential to automate the whole brain tumor analysis process and give efficient results without human intervention with least resource utilization. The proposed solution can easily be installed in a clinical setup for initial screening of glioma patients after validation on local data.
Brain Stimulation:
Non-invasive Magnetic/TMS
Modeling and Analysis Methods:
Methods Development
Segmentation and Parcellation 2
Novel Imaging Acquisition Methods:
Anatomical MRI 1
Keywords:
Affective Disorders
Computational Neuroscience
Computing
Data analysis
MRI
STRUCTURAL MRI
1|2Indicates the priority used for review
Provide references using author date format
[2] M. Nazir et al., (2022), “Multi-task learning architecture for brain tumor detection and segmentation in MRI images,” J. Electron. Imaging, vol. 31, no. 05, pp. 035001–7, doi: 10.1117/1.JEI.31.5.051606.
[3] Multimodal Brain Tumor Segmentation Challenge Dataset, (2019 and 2020), https://www.med.upenn.edu/cbica/brats2020/data.html.