Neural representations of action-integrated reward in naturalistic foraging

974 

Abstract Submission 

Jaeyoung Jeon^1,2,3, WON MOK SHIM^1,2,3, Seng Bum Yoo^1,2,3

¹Ctr. for Neurosci. Imaging Res., Inst. for Basic Sci. (IBS), Suwon, Korea, Republic of, ²Dept. of Intelligent Precision Healthcare Convergence,Sungkyunkwan Univ.,, Suwon, Korea, Republic of, ³Dept. of Biomed. Engin., Sungkyunkwan Univ., Suwon, Korea, Republic of

Jaeyoung Jeon  
Ctr. for Neurosci. Imaging Res., Inst. for Basic Sci. (IBS)|Dept. of Intelligent Precision Healthcare Convergence,Sungkyunkwan Univ.,|Dept. of Biomed. Engin., Sungkyunkwan Univ.
Suwon, Korea, Republic of|Suwon, Korea, Republic of|Suwon, Korea, Republic of

Won Mok Shim  
Ctr. for Neurosci. Imaging Res., Inst. for Basic Sci. (IBS)|Dept. of Intelligent Precision Healthcare Convergence,Sungkyunkwan Univ.,|Dept. of Biomed. Engin., Sungkyunkwan Univ.
Suwon, Korea, Republic of|Suwon, Korea, Republic of|Suwon, Korea, Republic of

Seng Bum Yoo  
Ctr. for Neurosci. Imaging Res., Inst. for Basic Sci. (IBS)|Dept. of Intelligent Precision Healthcare Convergence,Sungkyunkwan Univ.,|Dept. of Biomed. Engin., Sungkyunkwan Univ.
Suwon, Korea, Republic of|Suwon, Korea, Republic of|Suwon, Korea, Republic of

Foraging is a continuous decision-making process aimed at maximizing long-term benefits. Such a process requires accurate beliefs about the spatial distribution of rewards, essential for constructing an efficient exploitation route. Furthermore, foraging behaviors can be understood in the context of evidence accumulation, where individuals strategically shift their preferences to accept or reject presented rewards, known as skipping behaviors (Hayden et al., 2018). However, previous studies that imitate foraging behaviors at an abstract level, such as patch-leaving, interpreted skipping behaviors predominantly in terms of reducing temporal cost (Constantino et al., 2015). Here, we introduced a two-step foraging task within an interactive 3D Minecraft-based platform, where participants are encouraged to exploit spatial regularities in a partially observable environment to maximize rewards.

We designed a spatial foraging task in a 3D-grid world featuring two different types and values of rewards (Fig. 1A). Participants were instructed to collect rewards as much as possible within a specified time, under a given type-to-value transition rule. Transition rules, determining the ratio between the requested types of harvested rewards (i.e. 1:1, 1:3), were implemented. The task consisted of two stages: 1) a regularity detection phase, during which participants reported whether each type of reward was spatially clustered (structured) or randomly distributed (random) (Fig. 1B), and 2) a reward collection phase, where participants either accepted or rejected offers at each step while navigating the environment based on their beliefs about the spatial regularity identified in the previous stage. Functional 7T MRI data (N=6, 12 maps) were collected after a practice session.

We first examined the effect of individuals' belief in spatial regularity on skipping frequency. Participants exhibited a significantly higher frequency of skipping behaviors in structured maps (t(5) = 2.19, p < .05) (Fig 2A), indicating their successful detection of reward clusters in structured maps. In addition, foraging scores were positively correlated (Pearson correlation coefficient, r = 0.41, p < .05) with skipping frequency (Fig 2B), emphasizing the role of skipping behavior as a key foraging strategy. Next, we examined neural activity in regions associated with reward processing in response to the presence of rewards and the selection of an action. Participants' actions as they approach the next reward position were categorized into three types: moving to a location without a reward (no reward), moving to a location with a reward but not collecting it later (skip), and moving to a location with a reward and collecting it (forage). We observed an increased BOLD response in the dACC when participants moved to a location with a present reward, regardless of whether they eventually harvested it or not, indicating its sensitivity to the observation of rewards (Fig 2C). In contrast, the vmPFC showed increased responses when a reward was foraged compared to when it was skipped, reflecting its involvement in anticipating significant future value linked to one's actions, rather than merely responding to the presence of a reward.

Our results demonstrated that humans strategically adopt the skipping strategy to enhance foraging efficiency based on the expected spatial distribution of rewards. This behavioral pattern was associated with distinct neural activity in the reward circuit (Juechems et al., 2019), suggesting a unique role for the dACC and vmPFC in representing the presence of rewards and the reward value that integrates future decision-making related to action selection. The observed skipping behavior serves as evidence that human subjects actively leverage information about spatial regularities to formulate optimal foraging routes, highlighting the integration of beliefs about reward regularities in the external environment for long-term reward maximization.

Decision Making ²

Higher Cognitive Functions Other ¹

BOLD fMRI

FUNCTIONAL MRI

Other - foraging; reward; 7T; decision making; Naturalistic task; 3D environment;