An fNIRS study Exploring Cognitive Processes in Narrative Engagement
Poster No:
2396
Submission Type:
Abstract Submission
Authors:
Leonhard Schreiner1, Christy Li2
Institutions:
1g.tec medical engineering GmbH, Schiedlberg, Austria, 2g.tec Neurotechnology Hong Kong Limited, Hong Kong, Hong Kong
First Author:
Co-Author:
Introduction:
Narrative engagement encompasses attention, immersion, and memory, which are interconnected cognitive processes. This study delves into the relationship between narrativity and cognitive processes by categorizing narratives into four complexity and semantic content-based levels. Our goal is to differentiate between these varying degrees of narrativity by utilizing functional Near-Infrared Spectroscopy (fNIRS) as a metric to assess brain activity while exposing individuals to narratives of different content. The chosen stimuli were paintings from 4-narrative levels, a subset defined by Ryan (2007). In this exploratory study, we investigate the relationship between different levels of narrativity and attention, engagement, and memory aspects using Oxygenated (HBO) and deoxygenated (HBR) hemoglobin concentration features from the fNIRS data and behavioral analysis.
Methods:
The wireless g.Nautilus fNIRS system (g.tec medical engineering GmbH) with 16 channels of EEG, combined with 8 channels of fNIRS, was used for data acquisition (Figure 1D). The 8 fNIRS channels were symmetrically placed bilaterally over the prefrontal cortex (PFC) (Figure 1B). To verify whether the general public would also categorize the paintings as per our pre-defined levels, we conducted an online validation test using the Clickworker platform (https://www.clickworker.com) with an independent panel of participants. Based on the participants' responses, we selected 10 pictures per category with the highest agreement percentage among the participants. Consequently, the stimuli used in this study comprised 40 paintings, with 10 paintings per narrativity level (Figure 1A). The procedure of the paradigm includes a short preparation phase, investigation of the pictures for 15 seconds followed by answering a questionnaire (Figure 1C). A support vector machine classifier with a 10x10 cross-fold (CV) approach was employed for the 4-class problem. A permutation test with shuffled labels was conducted 1000 times to identify statistically significant outcomes of the classifier.
Results:
Figure 2 illustrates the grand average in cortical oxygenation and deoxygenation across all subjects the prefrontal cortex. The graph displays an elevation in the HBO changes particularly in class 1 (abstract-narrative) and class 4 (high-narrative) during the visual presentation, with the most pronounced oxygenation increase in the abstract narrative class occurring approximately 5 seconds after the stimulus was presented. The concentrations of HBR exhibit only a minor decline around the 5-second mark, with no significant variations between classes. The drop observed across all channels and categories shortly after the task concluded can be attributed to movement artifacts resulting from subjects adjusting their positions to complete the questionnaires.The abstract-Narrative stimulus shows an high increase in accuracy, reaching the maximum of 75% at around 8 second after picture presentation.
Conclusions:
The findings from the behavioural analysis suggest that higher narrativity levels are associated with increased self-reported engagement while perceiving the stimulus. The relative HBO concentration changes show an increase in concentration across all levels, whereby the abstract-narrative and high-narrative classes have the highes concentration, which is in line with the outcome of the behavioural analysis. Therefore only the HBO feature channels were selected for classification. However, classification performance shows significant results only for the abstract-level class pictures. The brain might be processing abstract painting differently, such as analyzing colors, shapes, and textures, which could contribute to the classification outcome (Aviv, 2014).
Emotion, Motivation and Social Neuroscience:
Emotion and Motivation Other
Higher Cognitive Functions:
Higher Cognitive Functions Other 2
Learning and Memory:
Long-Term Memory (Episodic and Semantic)
Novel Imaging Acquisition Methods:
Multi-Modal Imaging 1
Perception, Attention and Motor Behavior:
Attention: Visual
Keywords:
Cognition
Data analysis
Electroencephaolography (EEG)
Memory
Modeling
Near Infra-Red Spectroscopy (NIRS)
Perception
Vision
Other - Narrative
1|2Indicates the priority used for review
