A massive multimodal fMRI dataset bridging naturalistic processes and experimental cognitive tasks
Poster No:
1880
Submission Type:
Abstract Submission
Authors:
Heejung Jung1, Maryam Amini1, Bethany Hunt2, Eilis Murphy1, Patrick Sadil3, Yaroslav Halchenko1, Zizhuang Miao1, Philip Kragel4, Xiaochun Han5, Mickela Heilicher6, Bogdan Petre1, Owen Collins7, Martin Lindquist3, Tor Wager1
Institutions:
1Dartmouth College, Hanover, NH, 2University of Vermont, Burlington, VT, 3Johns Hopkins University, Baltimore, MD, 4Emory University, Atlanta, GA, 5Beijing Normal University, Beijing, China, 6University of Wisconsin-Madison, Madison, WI, 7University of California, Irvine, Irvine, CA
First Author:
Co-Author(s):
Introduction:
Cognitive neuroscience has advanced significantly due to the availability of openly shared datasets. Large sample sizes, large amounts of data per person, and diversity in tasks and data types are all desirable, but are difficult to achieve in a single dataset. Here, we present an open dataset with N = 101 participants and 7.5 hours of scanning per participant, with six multifaceted cognitive tasks including 1.5 hours of naturalistic movie viewing. This dataset's combination of ample sample size, extensive data per participant, and a wide range of experimental conditions – including cognitive, affective, social, and somatic/interoceptive tasks – positions it uniquely for probing important questions in cognitive neuroscience.
·Overview of acquired data
Methods:
Six tasks with 20 experimental conditions were administered to adult participants, scanned across four sessions (mean ± s.d. age: 24.7 ± 5.5 years; 69 females, 45 males, 2 other). The dataset includes: 1) a T1-weighted anatomical scan, 2) a multi-shell diffusion weighted MRI (dMRI) scan, 3) functional BOLD echo-planar imaging during cognitive tasks (TR = 460 msec, MB = 8), 4) behavioral and 5) physiological data collected during scanning, and 6) a battery of questionnaires prior to scanning.
Tasks completed in the scanner include: a) a naturalistic video viewing task, in which participants watch videos and report their evoked emotions; b) a naturalistic narratives task, in which participants listen/read a series of narratives; c) a dynamic faces task, in which participants view video clips of experimentally manipulated faces with difference age, race, gender, and facial expressions; d) a self-referential processing task, in which participants assess characters in terms of similarity, likability, and mental state attribution; e) a multimodal negative affect task, in which participants experience somatic pain, vicarious pain, and cognitive effort with different levels of predictive cues; and 6) a fractionated overlapping task, in which participants complete two out of four tasks, including text-based theory of mind (i.e. false-belief task; Dodell-Feder et al., 2011), image-based theory of mind (i.e. why/how task; Spunt & Adolphs, 2014), memory (e.g. old/new encoding/retrieval task), and attention reorienting tasks (Posner, 1980).
To ensure consistency and reproducibility, we published the experiment presentation code as a package prior to initiating data collection. Furthermore, each experimenter rigorously followed a predefined dialogue script when providing instructions and guidance to participants.
Tasks completed in the scanner include: a) a naturalistic video viewing task, in which participants watch videos and report their evoked emotions; b) a naturalistic narratives task, in which participants listen/read a series of narratives; c) a dynamic faces task, in which participants view video clips of experimentally manipulated faces with difference age, race, gender, and facial expressions; d) a self-referential processing task, in which participants assess characters in terms of similarity, likability, and mental state attribution; e) a multimodal negative affect task, in which participants experience somatic pain, vicarious pain, and cognitive effort with different levels of predictive cues; and 6) a fractionated overlapping task, in which participants complete two out of four tasks, including text-based theory of mind (i.e. false-belief task; Dodell-Feder et al., 2011), image-based theory of mind (i.e. why/how task; Spunt & Adolphs, 2014), memory (e.g. old/new encoding/retrieval task), and attention reorienting tasks (Posner, 1980).
To ensure consistency and reproducibility, we published the experiment presentation code as a package prior to initiating data collection. Furthermore, each experimenter rigorously followed a predefined dialogue script when providing instructions and guidance to participants.
Results:
All data conform to Brain Imaging Data Structure (BIDS) conventions and are managed by Datalad, and will be shared publicly on the OpenNeuro archive. The neuroimaging data were preprocessed using fmriprep ver 21.0.2 (Esteban et al., 2019). Regarding quality control, head movement was kept to a minimum, as demonstrated by framewise displacement lower than that of the UK biobank, in spite of the longer scan protocol (Wilcoxon Rank-Sum Test; W = 49313721, p < .001).
Preliminary results from the multimodal negative affect task indicate distinct activation for each modality. Somatic pain activates the canonical pain network, including the insular, somatosensory, and dorsal anterior cingulate cortices. Vicarious pain primarily engages the fusiform gyrus and superior temporal sulcus. Cognitive discomfort involves the attentional network, involving the intraparietal sulcus. Finally, button responses show significant activation in the primary motor cortex.
Preliminary results from the multimodal negative affect task indicate distinct activation for each modality. Somatic pain activates the canonical pain network, including the insular, somatosensory, and dorsal anterior cingulate cortices. Vicarious pain primarily engages the fusiform gyrus and superior temporal sulcus. Cognitive discomfort involves the attentional network, involving the intraparietal sulcus. Finally, button responses show significant activation in the primary motor cortex.
·Contrast of somatic pain, cognitive effort, vicarious pain, and button press. All maps (N=97) are compared against baseline and FDR corrected (q < .0001)
Conclusions:
We present a large-scale quality-controlled dataset with multiple cognitive tasks, which provides a testbed to explore novel research questions with a sufficiently large sample size for meaningful effect size. Utilizing this dataset, future studies can construct models of semantic decoding using the naturalistic movie viewing and narratives task or develop markers of affective systems, involving somatic pain, observed pain, and cognitive effort.
Modeling and Analysis Methods:
Methods Development 1
Novel Imaging Acquisition Methods:
BOLD fMRI 2
Perception, Attention and Motor Behavior:
Perception: Auditory/ Vestibular
Perception: Pain and Visceral
Perception: Visual
Keywords:
Cognition
MRI
Open Data
Pain
Perception
Somatosensory
Vision
1|2Indicates the priority used for review
Provide references using author date format
Dodell-Feder, D. (2013), ‘Using fiction to assess mental state understanding: a new task for assessing theory of mind in adults’, PloS one, 8(11), e81279.
Posner, M. I. (1980), ‘Orienting of attention’, Quarterly Journal of Experimental Psychology, 32(1), 3-25.
Spunt, R. P. (2014), ‘Validating the why/how contrast for functional MRI studies of theory of mind’, Neuroimage, 99, 301-311.