Multimodal Ensemble Deep Learning for Mental Disorder Classification Using Neuroimaging Data
Poster No:
1481
Submission Type:
Abstract Submission
Authors:
Hooman Rokham1, Haleh Falakshahi2, Vince Calhoun3
Institutions:
1Georgia Institue of Technology, Atlanta, GA, 2Georgia Institute of Technology, Atlanta, GA, 3GSU/GATech/Emory, Decatur, GA
First Author:
Co-Author(s):
Introduction:
The intricate exploration of brain functional mechanisms within mood and mental disorders remains a formidable undertaking. Existing challenges in the categorization of psychosis, characterized by issues like heterogeneity, lack of validity, and the constraints of unbalanced, small sample sizes, further compound the complexity of unraveling and recognizing essential biological features associated with these disorders. This study addresses these challenges by implementing a sophisticated multimodal ensemble deep learning approach. The primary objective is to perform a four-way classification task for the diagnosis of individual subjects, utilizing a comprehensive dataset that incorporates both structural and resting-state functional MRI data. This innovative methodology spans across three diverse datasets, fostering a more nuanced and insightful understanding of the intricate interplay between brain function and psychiatric disorders.
Methods:
In this investigation, we scrutinized the structural and resting-state functional MRI data of a cohort comprising 1520 subjects, encompassing 566 normal controls, 234 with bipolar disorder, 249 with schizoaffective disorder, and 473 with schizophrenia from three distinct datasets-FBIRN, B-SNIP1, and B-SNIP2. The preprocessing steps involved slice timing and rigid body head motion correction, followed by registration to the standard Montreal Neurological Institute (MNI) and subsequent smoothing. Subsequently, a fully automated Neuromark ICA pipeline (Du et al., 2020) was employed on the preprocessed data, extracting 53 intrinsic connectivity networks (ICNs) from seven different domain regions. Subject-specific functional networks and associated time-courses (TC) were then extracted.
Following this, we utilized a fully convolutional neural network (CNN) architecture as the foundational model for fMRI data and a 3D convolutional neural network for structural MRI (sMRI) data. Our proposed multimodal ensemble deep learning method was assessed for its ability to glean features from time courses and gray matter images using undersampling technique in conjunction with repeated cross-validation. By adopting this approach, each individual subject appeared in different test sets, facilitating evaluation and classification using models trained with distinct randomly generated training sets. The labeling assigned to each test sample was determined based on the maximum averaged probabilities obtained from the individual base models for each class.
Following this, we utilized a fully convolutional neural network (CNN) architecture as the foundational model for fMRI data and a 3D convolutional neural network for structural MRI (sMRI) data. Our proposed multimodal ensemble deep learning method was assessed for its ability to glean features from time courses and gray matter images using undersampling technique in conjunction with repeated cross-validation. By adopting this approach, each individual subject appeared in different test sets, facilitating evaluation and classification using models trained with distinct randomly generated training sets. The labeling assigned to each test sample was determined based on the maximum averaged probabilities obtained from the individual base models for each class.
Results:
Results demonstrates that the ensemble deep learning approach surpasses the performance of the base deep CNN models in handling both structural and functional MRI data. This enhancement is particularly evident in a performance boost of 0.11 for sMRI data and 0.13 for fMRI data. Furthermore, employing data fusion techniques and an ensemble approach elevates the overall accuracy to 0.61, accompanied by an area under the curve (AUC) of 0.83. Notably, statistical examinations of group differences in gray matter and functional connectivity data, coupled with post-classification analysis, reveal that misclassified subjects exhibit more pronounced distinctions within their respective groups compared to others predicted by the model.
·Unimodal and multimodal confision matrices of ensemble method.
Conclusions:
In summary, we have introduced a multimodal ensemble deep learning method for the classification of mood and psychosis disorders. The findings indicate that amalgamating individual base models as ensemble deep learning methods outperforms a single CNN. Additionally, the multimodal ensemble method further enhances performance. The results suggest that multimodal ensemble deep learning methods exhibit robustness to label noise and demonstrate superior performance in the presence of noisy and heterogeneous data, as compared to single deep learning models. Moreover, the ensemble deep learning method exhibits improved performance, especially in scenarios with small sample sizes, such as in small neuroimaging datasets within the field of psychiatry.
Modeling and Analysis Methods:
Classification and Predictive Modeling 1
Novel Imaging Acquisition Methods:
Multi-Modal Imaging 2
Keywords:
FUNCTIONAL MRI
Modeling
Psychiatric Disorders
STRUCTURAL MRI
1|2Indicates the priority used for review