Brain Large-scale Networks and Language Modeling during Spoken Narrative Listening
Poster No:
1771
Submission Type:
Abstract Submission
Authors:
Marianna Chianese1, Federica Di Nardo1, Maria Agnese Pirozzi1, Mario Cirillo1, Francesco Di Salle2, Fabrizio Esposito1
Institutions:
1University of Campania "Luigi Vanvitelli", Naples, Italy, 2University of Salerno, Baronissi, Salerno, Italy
First Author:
Co-Author(s):
Introduction:
Understanding the spatio-temporal dynamics of large-scale brain networks during narrative listening might benefit from the application of language models. As different recurrent patterns of large-scale inter-network functional connectivity (FC) are usually linked to distinct brain states, a static (sFC) and dynamic (dFC) FC analysis of whole-brain fMRI time-series is presented where sFC and dFC changes are correlated to word suprisal, a linguistic metric previously used to link the narrative text to fMRI neural activity in language-related regions [1,2].
Methods:
Preprocessed fMRI data sets from two runs of a previous 3 Tesla MRI study [1,2], acquired in 31 healthy subjects during task-free listening to a 12-minute audiobook, played in both original (forward, FW) and reversed (backward, BW) versions, were re-analyzed via group independent component analysis (gICA) using the GIFT toolbox [3]. Seven gICA components were labelled to major large-scale networks and back-projected to individual time-courses (per subject, network, condition). These were entered into linear mixed-effects (LME) models to estimate the significance of each network temporal correlation to the word surprisal predictor after correction with false discovery rate (FDR) while accounting for the variance explained by the acoustic envelope of the sound and the frequency and duration of single words [1]. Clusters of recurring patterns of inter-network correlations across 660 consecutive time windows of 90 sec were extracted via K-means and silhouette methods and identified as dFC brain states. Frequency of occurrence and mean dwell time of each state were estimated across subjects, compared between playing conditions and correlated to mean word surprisal across time windows.
Results:
Somato-motor network (SMN) time-course was positively correlated (FDR<0.05) to the surprisal predictor during both BW and FW conditions while the default-mode network (DMN) time-course was positively correlated only during BW condition. Visual (VIS), dorsal (DAN) and ventral (VAN) attention and limbic (LN) network time-courses were all positively correlated, while fronto-parietal network (FPN) time-course was negatively correlated, to the surprisal predictor, during FW, but not BW, condition. Six dFC states were identified (Figure 1): States 1, 3 and 5 were dominated by DAN-VAN interactions, whereas states 2, 4 and 6 variably involved VIS, SMN, FPN and LN (mainly interacting with VAN and DAN). The frequency of occurrence and mean dwell time of state 1 were significantly higher during FW, compared to BW, while the opposite was observed for states 3 and 5 (Wilcoxon signed-rank test, FDR<0.05). No significant differences between conditions were observed for states 2, 4 and 6. The average percentage of subjects found in the same state across time windows was significantly correlated (Spearman correlation coefficient, FW: |ρ| > 0.21, P < 0.05, BW: |ρ| > 0.18, FDR < 0.05) to the average surprisal of the words within the same windows for four (1, 3, 4, 6) and five (1, 2, 3, 4, 6) states respectively during BW and FW conditions. For state 3 this correlation switched from positive (BW) to negative (FW) whereas the opposite was observed for state 4 (Figure 2).
·Figure 1. Inter-network correlations matrices for each cluster (dFC state).
·Figure 2. The average percentage of subjects found in the same state across time windows (blue), average surprisal of the words within the same time windows (red) in FW and BW conditions.
Conclusions:
These results illustrate the usefulness of surprisal for explaining sFC and dFC effects across large-scale brain networks during natural listening conditions. The interaction between DAN and VAN networks was found dominant in specific states for which (i) the number of occurrences and mean dwell time were significantly changed when the listened words were understandable to (and could be put in narrative context by) the same subjects and (ii) the co-occurrence of the same dFC state was correlated to the average surprisal across all words listened during intervals as long as 90s. These findings might have implications for future naturalistic fMRI studies and for probing language comprehension during narrative listening.
Language:
Language Other 2
Modeling and Analysis Methods:
Activation (eg. BOLD task-fMRI)
Connectivity (eg. functional, effective, structural)
fMRI Connectivity and Network Modeling 1
Keywords:
FUNCTIONAL MRI
Language
Modeling
Other - Task fMRI
1|2Indicates the priority used for review
Provide references using author date format
[2] A. G. Russo, A. Ciarlo, S. Ponticorvo, F. Di Salle, G. Tedeschi, e F. Esposito, «Explaining neural activity in human listeners with deep learning via natural language processing of narrative text.», Sci Rep, vol. 12, fasc. 1, p. 17838, ott. 2022, doi: 10.1038/s41598-022-21782-4.
[3] V. D. Calhoun, T. Adali, G. D. Pearlson, e J. J. Pekar, «A method for making group inferences from functional MRI data using independent component analysis.», Hum Brain Mapp, vol. 14, fasc. 3, pp. 140–151, nov. 2001, doi: 10.1002/hbm.1048.