Unconscious processing networks by combining simultaneous EEG-fMRI data using optimal transport

1523 

Abstract Submission 

Julia Schräder^1,2, Damin Kühn³, Lennard Rompelberg³, Han-Gue Jo⁴, Thilo Kellermann^1,2, Michael Schaub³, Lisa Wagels^1,2

¹Department of Psychiatry, Psychotherapy and Psychosomatics, Faculty of Medicine, RWTH Aachen, Aachen, Germany, ²Institute of Neuroscience and Medicine: JARA-Institute Brain Structure Function Relationship (INM 10), Research Center Jülich, Jülich, Germany, ³Department of Computer Science, RWTH Aachen University, Aachen, Germany, ⁴Kunsan National University, Gunsan, Jeollabuk-do

Julia Schräder  
Department of Psychiatry, Psychotherapy and Psychosomatics, Faculty of Medicine, RWTH Aachen|Institute of Neuroscience and Medicine: JARA-Institute Brain Structure Function Relationship (INM 10), Research Center Jülich
Aachen, Germany|Jülich, Germany

Damin Kühn  
Department of Computer Science, RWTH Aachen University
Aachen, Germany

Lennard Rompelberg  
Department of Computer Science, RWTH Aachen University
Aachen, Germany

Han-Gue Jo  
Kunsan National University
Gunsan, Jeollabuk-do

Thilo Kellermann  
Department of Psychiatry, Psychotherapy and Psychosomatics, Faculty of Medicine, RWTH Aachen|Institute of Neuroscience and Medicine: JARA-Institute Brain Structure Function Relationship (INM 10), Research Center Jülich
Aachen, Germany|Jülich, Germany

Michael Schaub  
Department of Computer Science, RWTH Aachen University
Aachen, Germany

Lisa Wagels  
Department of Psychiatry, Psychotherapy and Psychosomatics, Faculty of Medicine, RWTH Aachen|Institute of Neuroscience and Medicine: JARA-Institute Brain Structure Function Relationship (INM 10), Research Center Jülich
Aachen, Germany|Jülich, Germany

MRI compatible EEG systems enable simultaneous EEG-fMRI data assessment, which provides high spatial and high temporal resolution of neural signaling data underlying experimentally defined cognitive processes. This allows us to determine (network) models for the same process based on two modalities using either EEG-informed fMRI analysis or dynamic causal modeling (DCM) (Friston et al., 2003). However, methods to construct dynamic network models using the information of both recording modalities jointly are still in their infancy. As a proof of principle, we use the mathematical framework of optimal transport (OT) to align multimodal DCMs in the form of multiplex networks. OT has shown promising results in many application domains including Neuroscience (Bazeille et al., 2019; Gramfort et al., 2015; Thual et al., 2022). Here, we interpret and formalize the unimodal DCMs as distributions over nodes of brain regions and compute mappings that minimize the distances between the distributions of different modalities. Subsequently, a shared multimodal representation (barycenter) is computed from these mappings.

We use simultaneously recorded EEG-fMRI data of 66 healthy participants (HC) and 60 patients with major depressive disorder (MDD) undergoing a priming task, in which an unconsciously presented prime stimulus may influence the participants' decision. This task ensured emotional conflict trials (primer emotion × target emotion) as well as non-conflict trials at conscious (150 ms primer presentation time) or unconscious level (16.7 ms primer presentation time). As a first step, we are using EEG-informed fMRI analysis in a multiple regression including the individual N170 EEG values for each condition (happy / sad / neutral primer × conscious / unconscious) as regressor. As a second step, we are using DCM to describe stimulus-related changes in neurophysiological networks and formalize these as multiplex networks, a special case of multilayer networks. Lastly, we apply OT to derive a multimodal barycenter that optimally matches to both neuronal networks inferred through DCM from EEG- and fMRI-data respectively. To obtain a suitable distance measure for such optimizations, we extend the definition of OT to directed multiplex networks and provide an efficient numerical solution.
To test for behavioral differences between groups, mean reaction times (RT) were compared between groups in incongruent trials via two-tailed t-tests.

MDD showed significantly higher mean RT compared to HC (t = 7.80, p < 0.001; HC mean RT = 0.62 s; MDD mean RT = 0.69 s). Multiple regression of N170 amplitudes and BOLD signal revealed significant negative correlations (pFWE-corr = 0.005) in several grey matter clusters. The larger the N170 amplitude, the higher the brain activity in cortical regions such as the fusiform gyrus (k = 137), middle/inferior temporal (k = 137), middle/superior frontal gyrus (k = 28 and 20), as well as middle cingulate & paracingulate gyri (k = 15) the cerebellum (k = 10) and the right amygdala (k = 18). Alignment of EEG-DCM and fMRI-DCM using OT showed group differences in the bottom-up processing of emotional facial expressions similar to behavioral differences (slower processing of happy facial expressions in MDD). We show that the barycenter is a more accurate model of neural connectivity than those inferred solely through either modality.

In this study, we demonstrated that performance in emotion detection is decreased in MDD compared to HC. Additionally, EEG-informed fMRI analysis showed correlation of face sensitive N170 values and face / emotion sensitive brain areas (e.g. fusiform gyrus and amygdala). We demonstrate that the optimal transport method enables us to align EEG and fMRI data effectively. This synchronization facilitates the development of more precise neural networks.

Psychiatric (eg. Depression, Anxiety, Schizophrenia) ²

Emotional Perception

Activation (eg. BOLD task-fMRI)

Connectivity (eg. functional, effective, structural) ¹

Multi-Modal Imaging

Affective Disorders

Computational Neuroscience

Consciousness

Electroencephaolography (EEG)

Emotions

FUNCTIONAL MRI

Modeling

Perception

Psychiatric Disorders