Predictive Coding Theory Inspired Brain Informer Model for Deciphering Mental States Across Subjects
Poster No:
1413
Submission Type:
Abstract Submission
Authors:
Zi-Han Wang1,2, Chao-Gan Yan3
Institutions:
1Institute of Psychology, Chinese Academy of Sciences, Beijing, Beijing, 2Department of Psychology, University of Chinese Academy of Sciences, Beijing, China, 3Institute of Psychology, Chinese Academy of Sciences, Beijing, Beijing
First Author:
Zi-Han Wang
Institute of Psychology, Chinese Academy of Sciences|Department of Psychology, University of Chinese Academy of Sciences
Beijing, Beijing|Beijing, China
Institute of Psychology, Chinese Academy of Sciences|Department of Psychology, University of Chinese Academy of Sciences
Beijing, Beijing|Beijing, China
Co-Author:
Introduction:
Decoding the mental states of the brain across diverse subjects through neural activity using non-invasive resting-state functional magnetic resonance imaging (fMRI) has always been a challenging task. Inspired by the predictive coding theory (Eslami 2018; Friston 2018; Ryskin 2023), which bears a resemblance to the way the large-scale language models, such as GPT-4, generate coherent text by predicting the next word when processing text, we proposed a mental state classification model based on the spatial-temporal Brain Informer. Our model simulates the brain's cognitive prediction process by continuously predicting fMRI trajectories and deciphers mental states across diverse subjects based on a multi-task learning model.
Methods:
In this work, we utilized 4 datasets. The CoRR dataset (Zuo 2014) comprises the rest fMRI data from 450 subjects. The Rumination dataset (Chen 2020) includes the fMRI of 41 healthy participants guided by a rumination paradigm in 4 mental states: resting, autobiographical memory, rumination, and distraction (Figure 1A). The Rumination_Suzhou dataset (Jia 2023) involves fMRI from 57 healthy subjects and 62 Major Depressive Disorder (MDD) patients during the rumination task. We also employed the DIRECT phase II dataset (Chen 2023), which contains data from 1216 healthy participants, 1502 MDD patients, 205 Bipolar Disorder (BD) patients, and 314 Schizophrenia (SCZ) patients.
Our Brain Informer model (Figure 1B), a Transformer-based time series forecaster using Prob Attention (Zhou 2021), learns time series information for multi-step predictions. We enhanced Informer with a spatial Attention Map to understand interactions between brain regions. The model was pretrained on the CoRR dataset for predicting fMRI time series. We then employed multi-task learning, fine-tuning the model with the Rumination dataset, concurrently handling fMRI time series prediction and mental state decoding as parallel tasks. Subsequently, we validated its cross-subject decoding capability with an independent Rumination_Suzhou dataset. Finally, we used the DIRECT phase II dataset to examine predictive patterns in different patient groups during resting states, aiming to understand the relationship between mental states and mental disorders.
Our Brain Informer model (Figure 1B), a Transformer-based time series forecaster using Prob Attention (Zhou 2021), learns time series information for multi-step predictions. We enhanced Informer with a spatial Attention Map to understand interactions between brain regions. The model was pretrained on the CoRR dataset for predicting fMRI time series. We then employed multi-task learning, fine-tuning the model with the Rumination dataset, concurrently handling fMRI time series prediction and mental state decoding as parallel tasks. Subsequently, we validated its cross-subject decoding capability with an independent Rumination_Suzhou dataset. Finally, we used the DIRECT phase II dataset to examine predictive patterns in different patient groups during resting states, aiming to understand the relationship between mental states and mental disorders.
·Figure 1: Rumination data acquisition and flow diagram of the Brain Informer model.
Results:
Experimental results indicate that our model competently predicts fMRI time series (Figure 2A). Incorporating a classification task into the model enables the differentiation of mental states, while also improving the prediction of fMRI time series (Figure 2D). Additionally, integrating spatial attention helps capture interactions between brain regions in the fMRI time series, benefiting both prediction and classification tasks (Figure 2C, 2D). To demonstrate the model's capability for cross-subject mental state classification, we tested on a wholly independent dataset (Figure 2E, 2F). The prediction correlation reached 69.10%, the accuracy for time segments classification reached 71.72%, and the cross-subject classification accuracy rose to 82.47%, which substantiates its generalizability and robustness. Under the validation of a big dataset, we found that patients with depression exhibit more autobiographical memory and rumination behaviors (Figure 2B). This discovery confirms previous research findings. Using our model to classify the mental states of participants can, to some extent, assist in the diagnosis of mental diseases.
·Figure 2: The figures of representative results.
Conclusions:
We developed a predictive coding-based fMRI classification model, akin to language models like GPT-4, for decoding brain mental states. It successfully predicts fMRI time series, and classifies mental states, aided by spatial attention mechanisms. Validated with the independent dataset, our model manifested appreciable generalization performance for practical application. Our study links rumination, memory, and MDD, showing more memory and rumination in depression. Future work aims to enhance its capabilities using larger datasets and detailed emotional and linguistic decoding.
Modeling and Analysis Methods:
Classification and Predictive Modeling 1
Connectivity (eg. functional, effective, structural)
fMRI Connectivity and Network Modeling 2
Methods Development
Task-Independent and Resting-State Analysis
Keywords:
Data analysis
FUNCTIONAL MRI
Machine Learning
Other - Mental States Decoding; fMRI Prediction; Multi-Task Deep Learning; Spatial-Temporal Informer; Mental Diseases Diagnosis
1|2Indicates the priority used for review
Provide references using author date format
Chen X et al. (2023). The Complexity of Functional Connectivity Profiles of the Subgenual Anterior Cingulate Cortex and Dorsal Lateral Prefrontal Cortex in Major Depressive Disorder: A DIRECT Consortium Study (p. 2023.03.09.531726). bioRxiv.
Eslami S A et al. (2018). Neural scene representation and rendering. Science, 360(6394), 1204–1210.
Friston K. (2018). Does predictive coding have a future? Nature Neuroscience, 21(8), 1019–1021.
Jia F-N et al. (2023). Aberrant degree centrality profiles during rumination in major depressive disorder. Human Brain Mapping.
Ryskin et al. (2023). Prediction during language comprehension: what is next? Trends in Cognitive Sciences, 27(11),1032-1052.
Zhou H et al. (2021). Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. In Proceedings of the AAAI conference on artificial intelligence 35(12), 11106-11115.
Zuo X-N et al. (2014). An open science resource for establishing reliability and reproducibility in functional connectomics. Scientific Data, 1(1), Article 1.