Deep independent vector analysis learns linked and identifiable sources from multimodal data
Poster No:
1970
Submission Type:
Abstract Submission
Authors:
Xinhui Li1, Rogers Silva2, Vince Calhoun3
Institutions:
1Georgia Institute of Technology, Atlanta, GA, 2TReNDS Center, Atlanta, GA, 3GSU/GATech/Emory, Decatur, GA
First Author:
Co-Author(s):
Introduction:
To capture interpretable and multifaceted information about the brain, it is important to develop latent variable models that effectively analyze multimodal neuroimaging data. Recently, a multidataset independent subspace analysis (MISA) (Silva et al., 2020) framework has been developed encompassing multiple linear latent variable models to jointly analyze multimodal neuroimaging data. Meanwhile, identifiable variational autoencoder (iVAE) (Khemakhem et al., 2020) has been proposed to recover nonlinearly mixed sources by utilizing auxiliary variables. Built upon MISA and iVAE, we develop a nonlinear multivariate latent variable model, Deep Independent Vector Analysis (DeepIVA), to learn linked and identifiable sources that are nonlinearly mixed across multiple data modalities.
Methods:
DeepIVA Overview. We iteratively optimize the iVAE loss to recover identifiable sources per modality and the MISA loss to identify cross-modal source linkage until convergence (Fig 1).
Synthetic Data. We simulate multiple datasets with 2 modalities, {5, 10, 15} non-stationary multivariate Gaussian sources, {4, 8, 14} segments, and {2800, 5600} total observations.
Neuroimaging Data. We utilize the UK Biobank dataset (Miller et al., 2016) including two imaging modalities, T1-weighted structural MRI and resting-state functional MRI, from 2907 unaffected subjects. We uniformly partition subjects into 14 groups according to sex and age, and extract 15 sources.
Evaluation Metrics. We use two metrics to evaluate model performance: the trimmed mean correlation coefficient between the 25th percentile and the 75th percentile (MCC) and the minimum distance (MD) based on randomized dependence coefficient (RDC) matrix.
Synthetic Data. We simulate multiple datasets with 2 modalities, {5, 10, 15} non-stationary multivariate Gaussian sources, {4, 8, 14} segments, and {2800, 5600} total observations.
Neuroimaging Data. We utilize the UK Biobank dataset (Miller et al., 2016) including two imaging modalities, T1-weighted structural MRI and resting-state functional MRI, from 2907 unaffected subjects. We uniformly partition subjects into 14 groups according to sex and age, and extract 15 sources.
Evaluation Metrics. We use two metrics to evaluate model performance: the trimmed mean correlation coefficient between the 25th percentile and the 75th percentile (MCC) and the minimum distance (MD) based on randomized dependence coefficient (RDC) matrix.
Results:
Synthetic Data. According to the aggregated RDC matrices (Fig 2A), iVAE can identify unimodal sources (Row I, Columns I & II) but cannot capture cross-modal linkage (Row I, Columns III & IV). In contrast, MISA can learn cross-modal linkage (Row II, Columns III & IV) but cannot identify unimodal sources (Row II, Columns I & II). DeepIVA, which unifies iVAE and MISA, can not only recover sources per modality (Row III, Columns I & II), but also identify their linkage across modalities (Row III, Columns III & IV). Furthermore, DeepIVA shows the best aggregated performance (lowest MD, highest MCC) in all simulations (Fig 2B).
Neuroimaging Data. DeepIVA shows the strongest cross-modal dependence along the main diagonal (Fig 2C), suggesting that it can better capture linkage across two imaging modalities. We then color code the DeepIVA sources by sex and age groups (Fig 2D), and observe noticeable sex clusters (e.g. SCVs 12 and 15) and age clusters (e.g. SCVs 8 and 11), indicating that DeepIVA captures linked sources related to phenotype measures, with SCV 12 presenting nonlinear age effects. We next fit a separate linear line for observations from each segment. We note that slopes of fitted lines per segment are very consistent for most sources (e.g. SCVs 1-9), demonstrating that DeepIVA is capable of identifying consistent linked sources across segments. Reconstruction of DeepIVA sources also reveals linked brain biomarkers corresponding to sex and age groups (Fig 2E).
Neuroimaging Data. DeepIVA shows the strongest cross-modal dependence along the main diagonal (Fig 2C), suggesting that it can better capture linkage across two imaging modalities. We then color code the DeepIVA sources by sex and age groups (Fig 2D), and observe noticeable sex clusters (e.g. SCVs 12 and 15) and age clusters (e.g. SCVs 8 and 11), indicating that DeepIVA captures linked sources related to phenotype measures, with SCV 12 presenting nonlinear age effects. We next fit a separate linear line for observations from each segment. We note that slopes of fitted lines per segment are very consistent for most sources (e.g. SCVs 1-9), demonstrating that DeepIVA is capable of identifying consistent linked sources across segments. Reconstruction of DeepIVA sources also reveals linked brain biomarkers corresponding to sex and age groups (Fig 2E).
Conclusions:
DeepIVA can recover linked and identifiable sources that are nonlinearly mixed from synthetic data experiments and shows promising results from neuroimaging data experiments. Future work will quantify brain-phenotype relationships from the recovered neuroimaging sources.
Modeling and Analysis Methods:
Methods Development
Multivariate Approaches 1
Novel Imaging Acquisition Methods:
Anatomical MRI
BOLD fMRI
Multi-Modal Imaging 2
Keywords:
Aging
Data analysis
Design and Analysis
FUNCTIONAL MRI
Machine Learning
Modeling
MRI
Multivariate
STRUCTURAL MRI
1|2Indicates the priority used for review
Provide references using author date format
and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence
and Statistics, pp. 2207–2217. PMLR, 2020.
Karla L Miller, Fidel Alfaro-Almagro, Neal K Bangerter, David L Thomas, Essa Yacoub, Junqian
Xu, Andreas J Bartsch, Saad Jbabdi, Stamatios N Sotiropoulos, Jesper LR Andersson, et al.
Multimodal population brain imaging in the uk biobank prospective epidemiological study. Nature
neuroscience, 19(11):1523–1536, 2016.
Rogers F Silva, Sergey M Plis, Tulay Adalı, Marios S Pattichis, and Vince D Calhoun. Multidataset
independent subspace analysis with application to multimodal fusion. IEEE Transactions on Image
Processing, 30:588–602, 2020.