Structure in ongoing experience: extending Bayesian changepoint analysis for event segmentation

1370 

Abstract Submission 

Elizabeth DuPre¹, Scott Linderman¹, Russell Poldrack²

¹Stanford University, Stanford, CA, ²Stanford University, Palo Alto, CA

Elizabeth DuPre, PhD  
Stanford University
Stanford, CA

Scott Linderman  
Stanford University
Stanford, CA

Russell Poldrack  
Stanford University
Palo Alto, CA

Unconstrained or "naturalistic" stimuli have become increasingly popular in computational cognitive neuroscience, demanding new methods to analyze minimally annotated data. Event segmentation-in which task or stimulus changes are inferred directly from neural data-is an important tool in this space. Commonly, segmentations are labeled using left-right Hidden Markov Models (lrHMM), as in Baldassano et al. (2017). Importantly, lrHMMs have been shown to successfully segment functional magnetic resonance imaging (fMRI) data into events that agree with human annotations, such as individual scenes in a film. Researchers can then examine how these segmentation boundaries differ across brain regions and experimental contexts. However, these models are trained on averaged subject-level fMRI data, obscuring inter-individual variability in the location of event boundaries. They also produce a single segmentation, limiting insight into the model's relative confidence in event boundaries.

Here, we jointly identify individual as well as consensus group-level event segmentations by extending existing work on Bayesian changepoint analysis (Fearnhead, 2006) to a hierarchical framework. That is, we jointly estimate changepoints for each individual subject, constrained by hierarchically-estimated group changepoints. This framework also enables us to make additional improvements on existing lrHMMs, such as identifying the model's relative confidence in a given segmentation versus other segmentations with differing numbers of changepoints. We are additionally able to allow individual voxels to exhibit unique variances, better matching the underlying fMRI signal. We showcase this model in the Sherlock dataset shared during the OHBM Naturalistic Data educational course (Chang et al., 2020) and previously collected by Chen and colleagues (Chen et al., 2017).

Hierarchical Bayesian changepoint analysis reveals variability in event segmentations that is commonly missed in lrHMM analyses. In Figure 1, we show this variability at the group level, highlighting the relative confidence of the model for a range of changepoints (i.e., segmented events). We note that these segmentations still capture meaningful structure in the data, as shown in Figure 2 for an example segmentation. Our results also allow us to identify segmentation time courses for each individual in the training dataset, which has not been possible to date.

We expect this hierarchical Bayesian changepoint analysis implementation will be broadly useful in naturalistic neuroimaging, as well as in neural data recordings that show significant inter-session or inter-individual variability with limited annotations. This work will also be increasingly important in mapping individual differences in event segmentation and their changes in health and disease (Jafarpour et al., 2022).

Bayesian Modeling ¹

Classification and Predictive Modeling

Methods Development

Multivariate Approaches ²

Informatics Other

Cognition

Computational Neuroscience

Data analysis

Machine Learning

Open Data

Open-Source Code

Statistical Methods