Compositional representation of tasks in human multiple-demand cortex

972 

Abstract Submission 

Jinkang (Derrick) Xiang¹, Moataz Assem², Geoffrey Ngo¹, John Duncan², Marieke Mur^1,3,4

¹Western Institute of Neuroscience, University of Western Ontario, Canada, ²MRC Cognition and Brain Sciences Unit, University of Cambridge, UK, ³Department of Psychology, University of Western Ontario, Canada, ⁴Department of Computer Science, University of Western Ontario, Canada

Jinkang (Derrick) Xiang  
Western Institute of Neuroscience, University of Western Ontario
Canada

Moataz Assem  
MRC Cognition and Brain Sciences Unit, University of Cambridge
UK

Geoffrey Ngo  
Western Institute of Neuroscience, University of Western Ontario
Canada

John Duncan  
MRC Cognition and Brain Sciences Unit, University of Cambridge
UK

Marieke Mur  
Western Institute of Neuroscience, University of Western Ontario|Department of Psychology, University of Western Ontario|Department of Computer Science, University of Western Ontario
Canada|Canada|Canada

The execution of complex cognitive tasks activates an extensive network of frontal and parietal regions, known as the multiple-demand (MD) system, whose distributed activity patterns carry information about the task (Cole et al., 2011; Duncan, 2010; Woolgar et al., 2011). However, the functional organization of task representation remains unclear. Do certain tasks elicit more similar activity patterns than others? If so, what drives the functional organization? Computational work suggests that tasks may be represented in a compositional fashion in prefrontal cortex, where the representation of a task can be expressed as the algebraic sum of vectors representing the underlying sensory, cognitive and motor processes (Yang et al., 2019). Empirical evidence for compositional coding is limited (Cole et al., 2011; Reverberi et al., 2012). It remains to be tested if this principle generalizes to tasks that require context-dependent decisions.

We approach these questions by conceptualizing tasks as combinations of features (Cole et al., 2011; Mante et al., 2013; Rigotti et al., 2013). The feature dimensions define a task space, and points in this task space correspond to multivariate activity patterns elicited by tasks composed of different combinations of features. We designed a delayed-match-to-sample experiment where the task space was spanned by two feature dimensions: attended sensory modality and match rule (Fig. 1A). We measured brain activity of 32 healthy young adults while they performed the tasks in a 3T fMRI scanner (2D echo-planar imaging, 46 slices, voxel size: 2.5 mm³, TR: 1.53 s, TE: 30 ms).

To compare the neural response elicited by different tasks, we estimated representational dissimilarity matrices (RDMs) using cross-validated Mahalanobis distances for each participant. To probe into the representational content of the data RDMs, we first created model RDMs based on single task features and their interactions. Next, we fitted the model RDMs to the data RDMs using linear combinations of task features, then additionally included their interactions, using non-negative weights (Jozwik et al., 2016). Model performance was evaluated by the cosine similarity between data RDMs and predicted RDMs (Diedrichsen et al., 2021). We compared performance across models for different regions of interest (Assem et al., 2020) (Fig. 1C). If adding interaction terms to the model does not improve performance, this provides evidence for compositional coding (Fig. 1E, F).

Our tasks strongly engage the MD system and both the attended sensory modality and the match rule are decodable from cortical regions comprising the MD system (Fig. 1D). Attended sensory modality is more widely decodable than match rule, consistent with the presence of attentional effects in sensory regions. Furthermore, representations in the MD system and higher visual regions are better modeled by task dimensions than those in lower visual regions. Importantly, across regions of interest, the interaction between task dimensions does not explain the task representation over and above their linear combination (Fig. 1E, F). These results suggest that tasks are represented in the MD system in a compositional fashion.

The representation of tasks differs across the cortical hierarchy. Early processing regions show a representation that is more strongly dominated by attended sensory information and less by abstract task rules, while later processing regions such as the MD system show a representation that carries a broad array of task information. In addition, task representations can be modeled as a linear combination of the representations of task features, while their interactions do not contribute significantly, supporting the compositional coding strategy. Future work should test for compositional coding in a broader range of tasks and across spatial scales.

Higher Cognitive Functions Other ¹

Activation (eg. BOLD task-fMRI)

Multivariate Approaches ²

Cognition

Computational Neuroscience

FUNCTIONAL MRI

Modeling