Neural signatures of emotional inference and experience align during social consensus

793 

Abstract Submission 

Marianne Reddan¹, Desmond Ong², Tor Wager³, Sonny Mattek⁴, Isabella Kahhale⁵, Jamil Zaki⁶

¹Albert Einstein College of Medicine, The Bronx, NY, ²University of Texas at Austin, Austin, TX, ³Dartmouth College, Hanover, NH, ⁴University of Oregon, Eugene, OR, ⁵University of Pittsburgh, Pittsburgh, PA, ⁶Stanford University, Stanford, CA

Marianne Reddan, PhD  
Albert Einstein College of Medicine
The Bronx, NY

Desmond Ong, PhD  
University of Texas at Austin
Austin, TX

Tor Wager, PhD  
Dartmouth College
Hanover, NH

Sonny Mattek, PhD  
University of Oregon
Eugene, OR

Isabella Kahhale  
University of Pittsburgh
Pittsburgh, PA

Jamil Zaki, PhD  
Stanford University
Stanford, CA

Humans seamlessly transform dynamic social signals into inferences about the internal states of the people around them. The neural processes that lead to conscious inference are computationally complex and require integration across multiple sources of information such as an observer's internal homeostatic state, their past experiences, expectations, and social schemas [1,2]. Neuroimaging studies have implicated the medial prefrontal cortex, temporoparietal junction, and precuneus [3,4] in socioemotional inference; however, integrated models of how multiple brain regions interact to form an inference are lacking. Furthermore, prior research sometimes equates the signaller's intended emotion with the observer's inference. Here we seek to (1) disentangle the neural processes underlying the perception of signal intent from inference, and (2) test how concordance between these processes is related to inference accuracy.

We collected fMRI data from participants (N = 100) as they watched 24 videos of social targets describing real-life emotional stories of both positive and negative valence. Participants rated the emotional intensity of people (targets) describing significant life events. Targets rated themselves on the same scale to indicate the intended "ground truth" emotional intensity of their videos. We used LASSO-PCR, a multivariate regression technique, to train two models of observer brain activity via leave-one-subject-out cross-validation (Fig 1). 8 video trials comprised the training set, while the remaining 12 trials comprised the held-out validation set. Videos were selected from the Stanford Emotional Narratives Dataset (SENDv1) [5].
After we identified these two unique components that underlie socioemotional processing, we sought to test how they interact in relation to an individual person's empathic accuracy. To do this, we applied the "ground truth" and inference models to participant-level brain activity when participants made inaccurate inferences (low empathic accuracy) and accurate inferences (high empathic accuracy; Fig 2A). Then we correlated the predictions of the two models across all participants.

Both the "ground truth" emotional state of the target (r = 0.50, 5,000 bootstrap samples, P < 0.0001), and an observer's inference about the target (r = 0.53, 5,000 bootstrap samples, P < 0.0001) could be predicted from the observer's brain activity. These neural patterns predictive of "ground truth" and inference are dissociable (cosine similarity = 0.29).The alignment between the two models was significantly greater during high accuracy performance than low accuracy performance (z = -2.71, P = 0.003; Fig 2B). We verified this effect in the validation trials (low empathic accuracy alignment r = 0.58, P < 0.0001; high empathic accuracy alignment r = 0.79, P < 0.0001; two-tailed z-test of the correlation difference z = 2.83, P = 0.005; Fig 2C).

Using naturalistic socioemotional stimuli and machine learning, we developed reliable brain signatures that predict what an observer thinks about a target, what the target thinks about themselves, and investigated the correspondence between them. Interestingly, we found that greater concordance between the latent representation of the target's intended signal intensity and the observer's inference indicates greater empathic accuracy. These results give insight into the neural processes that transform social signals into conscious inference during social interactions. Future work will apply these signatures in clinical data to better our understanding of socioemotional dysfunction in people with mood disorders.

Emotional Perception ²

Social Cognition ¹

Activation (eg. BOLD task-fMRI)

Classification and Predictive Modeling

Multivariate Approaches

ADULTS

Cognition

Emotions

FUNCTIONAL MRI

Machine Learning

Multivariate