Dynamics of Decision-Consistent Bias in Human Visual Working Memory
Poster No:
1113
Submission Type:
Abstract Submission
Authors:
Hyunwoo Gu1,2, Joonwon Lee3, Sungje Kim3, Jaeseob Lim3, Hyang-Jung Lee3, Heeseung Lee3, Minjin Choe3, Dong-Gyu Yoo3, Jun Hwan (Joshua) Ryu1,2, Sukbin Lim4,5,6, Sang-Hun Lee3
Institutions:
1Stanford University, Stanford, CA, 2Wu Tsai Neurosciences Institute, Stanford, CA, 3Seoul National University, Seoul, Korea, Republic of, 4NYU Shanghai, Shanghai, China, 5Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning, Shanghai, China, 6NYU-ECNU Institute of Brain and Cognitive Science, Shanghai, China
First Author:
Co-Author(s):
Sukbin Lim
NYU Shanghai|Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning|NYU-ECNU Institute of Brain and Cognitive Science
Shanghai, China|Shanghai, China|Shanghai, China
NYU Shanghai|Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning|NYU-ECNU Institute of Brain and Cognitive Science
Shanghai, China|Shanghai, China|Shanghai, China
Introduction:
Humans tend to bias their ongoing actions toward their past decisions, a phenomenon dubbed decision-consistent bias [3,4]. Efforts to explain this seemingly irrational bias have been limited to the sensory level, despite the evidence that it is affected by post-perceptual processes [2,9]. Here, by combining psychophysical and cortical measurements with a class of dynamical models, we uncover a previously unidentified source of the decision-consistent bias: the interplay of decision-making with the drift dynamics of working memory.
Methods:
In a scanner, participants were asked to memorize the orientation of a briefly (1.5s) presented grating and estimate it after a prolonged (15s) delay, intervened by a discrimination task (1.5s) (Fig. 1a-b). In the discrimination task, upon the appearance of a reference, participants had to decide, whether the remembered orientation was tilted clockwise or counter-clockwise relative to the reference. We quantified the decision-consistent bias as the difference between choice-conditioned mean errors (Fig. 1c).
To characterize the dynamics of decision-consistent bias, we considered a model class of the memory process with or without drift dynamics (Fig. 1e-f). In these models, the initial point at the sensory encoding stage is constrained by the efficient encoding scheme [8], and the encoding and drift functions are constrained by the stimulus-specific bias inferred from the estimation errors (Fig. 1d) using the von Mises basis functions.
To probe the working memory dynamics during delay, we performed linear decoding from the early visual cortex, V1, V2, and V3, considering the availability of working memory information there [6,7]. We used the inverted encoding model [1] and projected the reconstructed responses onto the polar space to readout the orientation. To compare the BOLD decoding trajectories and the model predictions, we convolved the canonical hemodynamic response functions with the model predictions and the boxcar-modeled visual drives during the stimulus and reference epochs.
To characterize the dynamics of decision-consistent bias, we considered a model class of the memory process with or without drift dynamics (Fig. 1e-f). In these models, the initial point at the sensory encoding stage is constrained by the efficient encoding scheme [8], and the encoding and drift functions are constrained by the stimulus-specific bias inferred from the estimation errors (Fig. 1d) using the von Mises basis functions.
To probe the working memory dynamics during delay, we performed linear decoding from the early visual cortex, V1, V2, and V3, considering the availability of working memory information there [6,7]. We used the inverted encoding model [1] and projected the reconstructed responses onto the polar space to readout the orientation. To compare the BOLD decoding trajectories and the model predictions, we convolved the canonical hemodynamic response functions with the model predictions and the boxcar-modeled visual drives during the stimulus and reference epochs.
Results:
First, the drifting dynamics of the decision-consistent bias well explained behavioral responses. Across all participants, the full model with both drift and diffusion terms was superior to the reduced model without drift when the Bayesian information criterion was compared (Fig. 1g).
Next, the working memory signals in the visual cortex were consistent with the drifting dynamics. When conditioned on the discrimination decisions, the decoded trajectories showed a bifurcation away from the target orientation towards the direction consistent with the decision (Fig. 2b). Trajectories predicted by the full model closely resembled those observed in the BOLD signals, compared to the diffusion-only reduced model (Fig. 2c).
As predicted in the drifint time course (Fig. 2a), the post-decision component (b_post) decreases in late trials, whereas the pre-decision component (b_pre) increases. To quantify these delay-dependent changes, we estimated their differences in early and late trials, namely, Δb_pre and Δb_post, by inferring the conditional means during the discrimination and estimation tasks both from behavior (Fig. 2d) and BOLD signals (Fig. 2f). These results matched the full model prediction (Fig. 2h), both in the signs of Δb_post and in their negative correlation, while the reduced model failed to capture these signatures (Fig. 2j).
Furthermore, when conditioned on the converging (zero-crossing points in drift functions with negative slopes) and diverging (positive slopes) stimuli characterized for each participant, the full model correctly captured the larger absolute amounts of ∆b_pre and ∆b_post in diverging stimuli, whereas the reduced model could not (Fig. 2e,g,i,k).
Next, the working memory signals in the visual cortex were consistent with the drifting dynamics. When conditioned on the discrimination decisions, the decoded trajectories showed a bifurcation away from the target orientation towards the direction consistent with the decision (Fig. 2b). Trajectories predicted by the full model closely resembled those observed in the BOLD signals, compared to the diffusion-only reduced model (Fig. 2c).
As predicted in the drifint time course (Fig. 2a), the post-decision component (b_post) decreases in late trials, whereas the pre-decision component (b_pre) increases. To quantify these delay-dependent changes, we estimated their differences in early and late trials, namely, Δb_pre and Δb_post, by inferring the conditional means during the discrimination and estimation tasks both from behavior (Fig. 2d) and BOLD signals (Fig. 2f). These results matched the full model prediction (Fig. 2h), both in the signs of Δb_post and in their negative correlation, while the reduced model failed to capture these signatures (Fig. 2j).
Furthermore, when conditioned on the converging (zero-crossing points in drift functions with negative slopes) and diverging (positive slopes) stimuli characterized for each participant, the full model correctly captured the larger absolute amounts of ∆b_pre and ∆b_post in diverging stimuli, whereas the reduced model could not (Fig. 2e,g,i,k).
Conclusions:
Our findings provide empirical support for the drifting working memory and its interplay with decision-consistent bias in humans, accounting for how the working memory representations can continuously develop in a decision-consistent manner.
Higher Cognitive Functions:
Decision Making 2
Learning and Memory:
Working Memory 1
Modeling and Analysis Methods:
Multivariate Approaches
Perception, Attention and Motor Behavior:
Perception: Visual
Keywords:
Other - Working memory, Decision-making
1|2Indicates the priority used for review
Provide references using author date format
[2] Fritsche, M. (2019), 'Reference repulsion is not a perceptual illusion', Cognition 184, 107–118.
[3] Jazayeri, M. (2007), 'A new perceptual illusion reveals mechanisms of sensory decoding', Nature 446, 912–915
[4] Luu, L. (2018), 'Post-decision biases reveal a self-consistency principle in perceptual inference', Elife 7, 1–24
[5] Panichello, M.F. (2019), 'Error-correcting dynamics in visual working memory', Nature Communications 10, 1–11
[6] Rademaker, R.L. (2019), 'Coexisting representations of sensory and mnemonic information in human visual cortex', Nature Neuroscience 22, 1336–1344.
[7] Serences, J.T. (2009), 'Stimulus-specific delay activity in human primary visual cortex', Psychological Science 20, 207–214.
[8] Wei, X.X. (2015), 'A Bayesian observer model constrained by efficient coding can explain ‘anti-Bayesian’ percepts', Nature Neuroscience 18, 1509–1517
[9] Zamboni, E. (2016), 'Do perceptual biases emerge early or late in visual processing? Decision-biases in motion perception', Proceedings of the Royal Society B: Biological Sciences 283.