Distinct neural patterns for working memory and sensory representations in the early visual cortex
Poster No:
1122
Submission Type:
Abstract Submission
Authors:
Sungje Kim1, Joonwon Lee1, Hyunwoo Gu2, Dong-Gyu Yoo1, Heeseung Lee1, Hyang-Jung Lee1, Jaeseob Lim1, Minjin Choe1, Sang-Hun Lee1
Institutions:
1Seoul National University, Seoul, Korea, Republic of, 2Stanford University, Stanford, CA
First Author:
Co-Author(s):
Introduction:
Numerous studies have demonstrated the presence of working memory (WM) representations within the early visual cortex (EVC) (Harrison and Tong, 2009; Serences et al., 2009; Ester et al., 2013). The sensory recruitment hypothesis proposes that the EVC utilizes the same neural code for both WM and sensory representations (referred to as the 'sensory code'), enabling WM to benefit from the high fidelity of sensory coding (Adam et al., 2022; D'Esposito and Postle, 2015). However, a recent study challenges this view, suggesting that the population activity patterns in the EVC during sensory encoding differ from those observed during WM maintenance (Kwak and Curtis, 2022). In this study, we investigate whether the EVC uses a shared code for sensory and working memory information by designing a paradigm that sensory and mnemonic representations are sufficiently separated in time and compared them using several measures of neural codes.
Methods:
In a magnetic resonance imaging (MRI) scanner, 50 human observers participated in a task where they briefly viewed an oriented grating and later reproduced its orientation after a substantial delay period (16.5 s), which is long enough for the blood-oxygen-level-dependent (BOLD) responses to the grating to dissipate. During the delay period, there was a decision period where the subjects was asked whether the remembered orientation was clockwise or counter-clockwise than the presented stimulus orientation. Sensory codes were defined based on BOLD responses during the period of peak univariate BOLD activity, while working memory codes were defined using responses during the last moment of the delay period.
Results:
Both sensory code and working memory code could successfully read out the information in the EVC, but their informative period were different. Decoding performance by sensory code was peaked after the stimulus onset, decreased during delay and disappeared at the last period of delay. Conversely, the working memory code exhibited lower decoding performance than the sensory code during stimulus onset, but maintained higher performance in entire delay period. During the decision period, where the physical stimulus was present, the sensory code accurately decoded the orientation of the physical stimulus, while the working memory code successfully retrieved the remembered orientation. These findings underscore the differential temporal characteristics of sensory and working memory codes in the EVC, highlighting a unique representation of working memory distinct from sensory representation. Several measures were employed to compare sensory and working memory representations. First, when projected onto a low-dimensional state space (Panicello and Buschman, 2021), sensory and working memory orientations formed in two separate planes orthogonal to each other. Second, using the inverted encoding model (Brouwer and Heeger, 2009), decoding stimulus orientation revealed that working memory information could not be decoded using voxel weights trained on sensory representations, and vice versa. Third, voxel-level analysis demonstrated that the differences in codes were derived from distinct distributions of orientation preferences. Lastly, examination of retinotopic properties showed that orientation preferences in the sensory representation were correlated with radial positions in retinotopic space (Ryu and Lee, 2018; Lee and Ryu; 2023), while such correlation was absent in the working memory representation.
Conclusions:
Our findings suggest that EVC employs distinct neural codes for representing sensory and mnemonic orientations. This differentiation may enable the brain to perform tasks involving the comparison of both types of representations with less distraction from external interference. The observed separation in coding mechanisms supports the idea that EVC adapts to the unique demands of sensory and working memory processes, providing insights into the neural underpinnings of working memory within the early visual cortex.
Learning and Memory:
Working Memory 1
Modeling and Analysis Methods:
Multivariate Approaches 2
Perception, Attention and Motor Behavior:
Perception: Visual
Keywords:
Memory
Multivariate
Perception
Other - Neural code
1|2Indicates the priority used for review
Provide references using author date format
Serences, J. T., Ester, E. F., Vogel, E. K., & Awh, E. (2009). Stimulus-specific delay activity in human primary visual cortex. Psychological science, 20(2), 207-214.
Ester, E. F., Anderson, D. E., Serences, J. T., & Awh, E. (2013). A neural measure of precision in visual working memory. Journal of cognitive neuroscience, 25(5), 754-761.
Adam, K. C., Rademaker, R. L., & Serences, J. T. (2022). Evidence for, and challenges to, sensory recruitment models of visual working memory. Visual memory, 5-25.
D'Esposito, M., & Postle, B. R. (2015). The cognitive neuroscience of working memory. Annual review of psychology, 66, 115-142.
Kwak, Y., & Curtis, C. E. (2022). Unveiling the abstract format of mnemonic representations. Neuron, 110(11), 1822-1828.
Panichello, M. F., & Buschman, T. J. (2021). Shared mechanisms underlie the control of working memory and attention. Nature, 592(7855), 601-605.
Brouwer, G. J., & Heeger, D. J. (2009). Decoding and reconstructing color from responses in human visual cortex. Journal of Neuroscience, 29(44), 13992-14003.
Ryu, J., & Lee, S. H. (2018). Stimulus-tuned structure of correlated fMRI activity in human visual cortex. Cerebral Cortex, 28(2), 693-712.
Lee, S. H., & Ryu, J. (2023). A mismatch between human early visual cortex and perception in spatial extent representation: Radial bias shapes cortical representation while co-axial bias shapes perception. bioRxiv, 2023-02.