Neural correlates of the perceptual spaces of low-level features and objects

2542 

Abstract Submission 

Suniyya Waraich¹, Keith Jamison², Amy Kuceyeski², Jonathan Victor³

¹Weill Cornell Graduate School, New York, NY, ²Weill Cornell Medicine, New York City, NY, ³Weill Cornell Medical College, New York, NY

Suniyya Waraich  
Weill Cornell Graduate School
New York, NY

Keith Jamison  
Weill Cornell Medicine
New York City, NY

Amy Kuceyeski  
Weill Cornell Medicine
New York City, NY

Jonathan Victor  
Weill Cornell Medical College
New York, NY

Low-level visual features and semantic information seem different: while low-level features, e.g., color, can vary continuously, objects often appear to be categorical. It has also been theorized that objects, unlike low-level features, are mentally represented in a tree-like structure (Kemp and Tenenbaum, 2008). To search for differences in the mental representations of low-level features and semantic information, we previously conducted psychophysical experiments to collect and analyze similarity judgments between different stimuli (Waraich and Victor, 2022) using a variant of multidimensional scaling (n=11-12 per domain, 8F). Five stimulus domains (Figure 1), which varied in their degree of semantic information were studied: a purely semantic domain of animal names, images of the animals, two intermediate domains of texturized images, and textures composed of colored checks. All five stimulus domains were derived from a common set of 37 animals. In separate experiments that probed the similarity structure of each stimulus domain individually, we found that indeed, even though all domains could be represented in a low-dimensional space, there was tree-like structure for the stimulus domains containing more semantic information: the image-like, image, and word domains (Waraich and Victor, 2023). We next sought to find the neural correlates of this difference.

9 of the participants (7F) who participated in the psychophysics experiments took part in an fMRI study. Over the course of four fMRI sessions, they were shown each of the stimuli (37 stimuli x 5 domains = 185) at least 7 times, while performing a one-back memory task to maintain central fixation and attention. Stimuli were presented in a slow event-related design (presentation time: 1s, ISI: 3.5s), but they appeared in blocks separated by 10s. Within a block, only stimuli from a single domain appeared. The data were acquired on a 3T Siemens scanner, with multiband acquisition and TR=0.8s. Data were preprocessed using fmriprep to perform motion correction, susceptibility distortion correction, and co-registration of functional scans with the anatomical scans. Two participants (2M) were excluded from further analysis due to excessive motion and poor performance on the one-back task. Response amplitudes to the five domains and individual images were separately obtained using two generalized linear models (Prince et al, 2022).

Voxel-wise responses to the five stimulus domains showed different spatial patterns, with the texture-like, image-like, and image domains eliciting the greatest activity in the occipital areas, and overall decreased activation relative to baseline and a different pattern of activation in response to the words. We quantified these differences by computing the dissimilarities between the patterns of responses (beta estimates) for each domain (using two metrics: cosine distance and correlation distance) for voxels that were explained well by the GLM: R^2 ≥ 0.1 (see Figure 2: fMRI) and R^2 ≥ 0.15 separately. These voxels comprise most of visual cortex. We found that the greatest similarity in responses was observed for the texture-like, image-like, and image domains, with the responses to textures sometimes similar to these 3 domains, and with responses to words always the most dissimilar across participants. Psychophysical experiments also showed a difference between domains but the main difference was between the more semantic domains (image-like stimuli, images, and words) and the less-semantic domains (textures and texture-like stimuli).

The psychophysical analysis compared within-domain similarities between stimuli and the fMRI analysis compared overall activation patterns between the domains. They both suggest a stepwise progression of representation in the brain, even though the correspondence between the representations based on similarity judgments and those based on neural activation patterns is not exact.

Language Comprehension and Semantics ²

Perception: Visual ¹

Cognition

Computational Neuroscience

FUNCTIONAL MRI

Vision