BrainVolCNN – A Volumetric CNN to Predict Task Contrasts from Resting-State Functional Connectivity

1885 

Abstract Submission 

Emin Serin¹, Andre Marquand², Kerstin Ritter³, Henrik Walter¹

¹Division of Mind and Brain Research, Department for Psychiatry, Charité–Universitätmedizin Berlin, Berlin, Germany, ²Radboud University Nijmegen, Nijmegen, Gelderland, ³Charité – Universitätsmedizin Berlin, Department of Psychiatry and Psychotherapy, Berlin, Germany

Emin Serin  
Division of Mind and Brain Research, Department for Psychiatry, Charité–Universitätmedizin Berlin
Berlin, Germany

Andre Marquand  
Radboud University Nijmegen
Nijmegen, Gelderland

Kerstin Ritter  
Charité – Universitätsmedizin Berlin, Department of Psychiatry and Psychotherapy
Berlin, Germany

Henrik Walter  
Division of Mind and Brain Research, Department for Psychiatry, Charité–Universitätmedizin Berlin
Berlin, Germany

It is increasingly recognized that large samples are essential for biomarker discovery, cohort stratification, and studying individual differences. While task-based fMRI (tb-fMRI) helps to understand neurocognitive mechanisms and outperforms resting-state fMRI (rs-fMRI) in predicting individuals' cognition (Tik et al., 2023), its cognitive demands present challenges when translating to large samples. However, recent studies have shown a strong correspondence between spontaneous and task-based brain activities (Cole et al., 2016; Tavor et al., 2016), indicating the potential predictability of task-evoked brain activity from resting-state brain activity. This suggests a promising approach to address the limitations of tb-fMRI in large samples. Nevertheless, previous efforts supporting this idea have been overly simplistic, focused only on cortical areas, or lacked transfer learning capabilities (Ngo et al., 2022; Tavor et al., 2016; Zheng et al., 2022). To overcome these limitations, we propose a volumetric-based neural network architecture (BrainVolCNN) that leverages resting-state connectivity of the entire cortex, including subcortical areas, to predict task-based brain activity maps (Fig. 1a).

Briefly, our study involved several steps. First, we predicted synthetic task contrast maps from resting-state connectivity and compared them to the actual contrast maps in the same training dataset. Second, we employed transfer learning by freezing the final layer of the network and fine-tuning the rest of the network on two held-out datasets that were not part of the training data. Third, we evaluated the performance of actual and predicted contrast maps in predicting age, sex, and fluid intelligence. Specifically, we trained our network using the HCP-Young Adults dataset (HCP-YA) (N = 953) (Van Essen et al., 2013) and evaluated it on HCP-Development (HCP-D) (N = 632) (Somerville et al., 2018) and UK Biobank (N = 20792) (Miller et al., 2016). To validate the predicted images, we assessed the usability of the predicted contrast maps in three machine-learning scenarios: predicting individuals' age, sex, and fluid intelligence based on their predicted and actual contrast maps. To establish a comprehensive evaluation, we compared the performance of our proposed model with various baselines, such as the linear model (Tavor et al., 2016), group-average contrast maps, and retest scans.

BrainVolCNN outperforms or achieves competitive reconstruction performance compared to baseline methods while preserving individual-specific information, leading to increased discriminability between subjects (Finn et al., 2015) (Fig. 1b). In addition, BrainVolCNN demonstrates superior reconstruction performance when applied to the HCP-D dataset after transfer learning, surpassing the linear model (Fig. 1c). Importantly, we also provide preliminary evidence of the potential value of our synthetic task images as biomarkers in that predicted contrast maps outperformed actual contrast maps in predicting sex and age while showing similar or slightly lower performance in predicting fluid intelligence (Fig. 2). This might be attributed to reduced noise during the synthetic image generation process, leading to an improved signal-to-noise ratio.

The results provide evidence of the partially superior performance of predicting task contrast maps from resting-state connectivity using BrainVolCNN compared to baseline models. This is evident in terms of both reconstruction accuracy and inter-subject discriminability. Our proposed framework facilitates transfer learning, enabling the prediction of contrast maps on datasets that have not been specifically acquired. Consequently, this enables the investigation of task-related biomarkers only requiring rs-fMRI scans.

Classification and Predictive Modeling ²

Methods Development ¹

Computational Neuroscience

FUNCTIONAL MRI

Machine Learning

Modeling