Grid-like representation in value-based decision-making
Poster No:
909
Submission Type:
Abstract Submission
Authors:
Mark Orloff1, Seongmin Park2, Jake Blumwald1, Philippe Domenech3, Erie Boorman1
Institutions:
1UC Davis, Davis, CA, 2CNRS, Lyon, Auvergne-Rhône-Alpes, 3Paris Brain Institute, Paris, France
First Author:
Co-Author(s):
Introduction:
Reward is encoded in the 'brain valuation system (BVS),' comprised of regions such as: dorsomedial prefrontal cortex (dmPFC) [1], posterior cingulate cortex (PCC) [2] and ventromedial prefrontal cortex (vmPFC) [3,4]. This value coding has been shown to reflect subjective value-how rewarding something is based on an individual's preference [2,5,6]. However, how a given rewarding option gets transformed into a subjective value (SV) signal is unclear.
One candidate for how this transformation could theoretically happen is via the brain's cognitive mapping system. A grid code, originally identified for its role in representing an animal's position and enabling path integration in physical space [7], encodes an individual's location in abstract task space (i.e., a non-physical 2D relational space) [8]. These grid-like signals have been found in entorhinal cortex (EC) and mPFC. Further work in rodents has shown that grid cell firing can be distorted with environmental deformations [9]. Here, we ask if humans use a grid-like representation to efficiently represent a 2D SV space and to infer decision vectors for risky decisions in this space.
One candidate for how this transformation could theoretically happen is via the brain's cognitive mapping system. A grid code, originally identified for its role in representing an animal's position and enabling path integration in physical space [7], encodes an individual's location in abstract task space (i.e., a non-physical 2D relational space) [8]. These grid-like signals have been found in entorhinal cortex (EC) and mPFC. Further work in rodents has shown that grid cell firing can be distorted with environmental deformations [9]. Here, we ask if humans use a grid-like representation to efficiently represent a 2D SV space and to infer decision vectors for risky decisions in this space.
Methods:
Participants (N = 35) were asked to make binary choices between two sequentially shown shape options drawn from two sets of shapes that vary along two continuous dimensions (such as width and orientation) corresponding to reward amount ($) and probability (%), respectively, while undergoing fMRI. We utilized the Cumulative Prospect Theory model to calculate the SV of each participant's choices [10].
We use two separate GLMs to test if BOLD activity is associated with 1) SV signal and 2) subjectively-weighted hexagonal modulation, characteristic of a grid-like representation. Specifically, we tested the following regressors in separate GLMs at the time that the second shape was shown:
SVshape = A^ρ × e^[-(-log(P))^α]
SV difference: Schosen - Sunchosen,
where SVshape represents the SV of a shape, A represents the reward amount, P represents the probability of winning the reward, ρ represents an individual's risk preference, and α represents an individual's probability weighting; and
hexagonal modulation of the decision vector over the attribute space (as predicted by grid cell firing fields): cos(6θ),
where θ represents the angle of the decision vector between the first and second shapes for each binary choice in a 2D SV space of subjective amount, norm[0,1](A^ρ), and subjective probability, norm[0,1](e^[-(-log(P))^α]), where norm[0,1](⋅) scales the values from 0 to 1. Threshold free cluster enhancement (TFCE) was used, with small-volume (S-V) correction in hypothesized regions-of-interest.
We use two separate GLMs to test if BOLD activity is associated with 1) SV signal and 2) subjectively-weighted hexagonal modulation, characteristic of a grid-like representation. Specifically, we tested the following regressors in separate GLMs at the time that the second shape was shown:
SVshape = A^ρ × e^[-(-log(P))^α]
SV difference: Schosen - Sunchosen,
where SVshape represents the SV of a shape, A represents the reward amount, P represents the probability of winning the reward, ρ represents an individual's risk preference, and α represents an individual's probability weighting; and
hexagonal modulation of the decision vector over the attribute space (as predicted by grid cell firing fields): cos(6θ),
where θ represents the angle of the decision vector between the first and second shapes for each binary choice in a 2D SV space of subjective amount, norm[0,1](A^ρ), and subjective probability, norm[0,1](e^[-(-log(P))^α]), where norm[0,1](⋅) scales the values from 0 to 1. Threshold free cluster enhancement (TFCE) was used, with small-volume (S-V) correction in hypothesized regions-of-interest.
Results:
We identified a subjective value comparison effect at decision time in PCC and vmPFC (PTFCE-S-V < 0.05), and an inverse effect in dmPFC, lateral prefrontal cortex (lPFC), and anterior insula (AI; PTFCE < 0.05) [2]. We also find a grid-like representation of decision vectors between options through an individual's distorted subjective value space in bilateral EC (PTFCE-S-V < 0.05), and, marginally, in mPFC (PTFCE-S-V = 0.07). This representation is specific to a six-fold periodicity (as opposed to 4-, 5-, 7-, or 8-fold control periodicities). Finally, we show that the strength of activation for subjective value difference in PCC and grid-like representation in bilateral EC is correlated across participants, controlling for decision consistency (inverse temperature).
·a: example task trials, red circle corresponds to time period of analyses; b: value difference signal
·a: two example SV spaces; b: grid-like representation signal; c: mean activity for each periodicity for bilateral EC and mPFC, d: correlation between PCC SV and EC grid-like representation signal
Conclusions:
In this study, we demonstrated that a grid-like representation of decision vectors in an SV space was utilized during risky choice. To our knowledge, this is the first demonstration of a grid-like representation of decision-vectors in a subjective 'value' space of probability and reward amount. Further, effects in the BVS and grid-like representation system were correlated, providing evidence that these systems are working collectively.
Higher Cognitive Functions:
Decision Making 1
Space, Time and Number Coding 2
Learning and Memory:
Learning and Memory Other
Keywords:
Cognition
Computational Neuroscience
FUNCTIONAL MRI
Learning
Memory
Other - cognitive maps; value-based decision-making, grid code
1|2Indicates the priority used for review
Provide references using author date format
2. Bartra O, McGuire JT, Kable JW. The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. NeuroImage. 2013;76: 412–427. doi:10.1016/j.neuroimage.2013.02.063
3. Lebreton M, Jorge S, Michel V, Thirion B, Pessiglione M. An Automatic Valuation System in the Human Brain: Evidence from Functional Neuroimaging. Neuron. 2009;64: 431–439. doi:10.1016/j.neuron.2009.09.040
4. Chib VS, Rangel A, Shimojo S, O’Doherty JP. Evidence for a Common Representation of Decision Values for Dissimilar Goods in Human Ventromedial Prefrontal Cortex. J Neurosci. 2009;29: 12315–12320. doi:10.1523/jneurosci.2575-09.2009
5. Padoa-Schioppa C, Assad JA. Neurons in the orbitofrontal cortex encode economic value. Nature. 2006;441: 223. doi:10.1038/nature04676
6. Boorman ED, Behrens TEJ, Woolrich MW, Rushworth MFS. How Green Is the Grass on the Other Side? Frontopolar Cortex and the Evidence in Favor of Alternative Courses of Action. Neuron. 2009;62: 733–743. doi:10.1016/j.neuron.2009.05.014
7. Hafting T, Fyhn M, Molden S, Moser M-B, Moser EI. Microstructure of a spatial map in the entorhinal cortex. Nature. 2005;436: 801–806. doi:10.1038/nature03721
8. Park SA, Miller DS, Boorman ED. Inferences on a multidimensional social hierarchy use a grid-like code. Nat Neurosci. 2021;24: 1292–1301. doi:10.1038/s41593-021-00916-3
9. Keinath AT, Epstein RA, Balasubramanian V. Environmental deformations dynamically shift the grid cell spatial metric. eLife. 2018;7: e38169. doi:10.7554/elife.38169
10. Tversky A, Kahneman D. Advances in prospect theory: Cumulative representation of uncertainty. J Risk Uncertain. 1992;5: 297–323. doi:10.1007/bf00122574