Inferring Affective States of Others Within Dynamic Social Interactions

811 

Abstract Submission 

Jisu Ro¹, Luke Chang², Ye Eun Seo³, WON MOK SHIM⁴

¹Sungkyunkwan University, Gangseo-gu, Seoul, ²Dartmouth College, Hanover, NH, ³Kaist, Yuseong-gu, Daejeon, ⁴CNIR/SKKU, Suwon-si, Kyonggi-do

Jisu Ro  
Sungkyunkwan University
Gangseo-gu, Seoul

Luke Chang, PhD  
Dartmouth College
Hanover, NH

Ye Eun Seo  
Kaist
Yuseong-gu, Daejeon

Won Mok Shim  
CNIR/SKKU
Suwon-si, Kyonggi-do

A key aspect of social cognition is inferring how another person is feeling. These inferences can be influenced by prior emotional states (Thornton et al, 2017), as well as by interactions with others (Houlihan et al, 2023). Although recent studies have attempted to characterize the predictive process of social cognition by examining prediction errors through simplified social actions within structured experimental context (Park et al, 2020), they have often dismissed the intricate nature of social interaction in a real life environment (Zaki et al, 2009). Therefore, in this study, we aim to elucidate how our social brain integrates social context information to dynamically update subjective predictions of others' state during naturalistic social interaction.

We developed several computational models outlining how participants might be generating their predictions of each characters' affective state for each scene. The reduced model recursively updates based on the valence prediction error (valence PE) - the difference between the observed and predicted valence ratings for a specific character. The full model further incorporates aspects of the social interaction - specifically the difference or mismatch between the two character's social intentions. We performed nested model comparisons to assess which model provided the best account of participant's valence rating data. To explore the neural dynamics of predictive computations, we collected brain imaging data with an independent group of participants (n=37) while they watched the movie inside the scanner without any task (3T, voxel size =3mm^3, TR =1s). We correlated the voxel-wise activity with each model component to investigate distinct neural responses associated with the predictive processes during social interaction.

For both characters in the movie, the full model incorporating the intention mismatch significantly outperformed the reduced model in explaining how participants' inferred each character's affective state. Univariate analysis revealed that the unsigned valence PE explains the striatal activity whereas its signed version elicited greater responses in the right temporal parietal junction (rTPJ). In contrast, the effect of intention mismatch was observed in the superior and middle temporal gyrus (STG, MTG) which expanded to more high-order cortical regions such as the angular gyrus (AG) and post cingulate cortex (PCC).

Our findings suggest that our social brain processes naturalistic social interactions in a predictive fashion. Participants appear to update their prediction about another person's affective state for each state based on prediction error. However, participants also appeared to simulate the consequences of the other social agent's intended action to generate more accurate predictions about how they might feel. These two distinct inference processes appear to be processed in distinct regions of the brain. Updating predictions based on overall errors is processed in the social cognition regions including the right TPJ, while tracking mismatches in intentions within the interaction appears to recruit the STG, MTG, AG and PCC. Together, this work provides a substantial advance towards understanding the complex psychological and neural processes that sustain our remarkable ability to infer another person's emotional state embedded in dynamic social interaction.

Emotional Perception

Social Cognition ¹

Social Interaction ²

Social Neuroscience Other

Cognition

Computational Neuroscience

Emotions

Modeling

Social Interactions

Other - naturalistic