Training Increases Reward Learning and Decreases Depression Symptoms Across Repeated Sessions
Poster No:
762
Submission Type:
Abstract Submission
Authors:
Shivani Goyal1,2, John Wang1, Vanessa Brown3, Jacob Lee1, Nanda Sankarasubramanian1, Brooks Casas1,2, Pearl Chiu1,2
Institutions:
1Fralin Biomedical Research Institute at Virginia Tech Carilion, Roanoke, VA, 2Department of Psychology, Virginia Tech, Blacksburg, VA, 3Department of Psychiatry, University of Pittsburg, Pittsburg, PA
First Author:
Shivani Goyal
Fralin Biomedical Research Institute at Virginia Tech Carilion|Department of Psychology, Virginia Tech
Roanoke, VA|Blacksburg, VA
Fralin Biomedical Research Institute at Virginia Tech Carilion|Department of Psychology, Virginia Tech
Roanoke, VA|Blacksburg, VA
Co-Author(s):
Brooks Casas
Fralin Biomedical Research Institute at Virginia Tech Carilion|Department of Psychology, Virginia Tech
Roanoke, VA|Blacksburg, VA
Fralin Biomedical Research Institute at Virginia Tech Carilion|Department of Psychology, Virginia Tech
Roanoke, VA|Blacksburg, VA
Pearl Chiu
Fralin Biomedical Research Institute at Virginia Tech Carilion|Department of Psychology, Virginia Tech
Roanoke, VA|Blacksburg, VA
Fralin Biomedical Research Institute at Virginia Tech Carilion|Department of Psychology, Virginia Tech
Roanoke, VA|Blacksburg, VA
Introduction:
Disrupted neural and behavioral indices of reward learning have been suggested to play a mechanistic role in depression [1,2]. If this mechanism were supported, we would expect improving reward learning to decrease depression symptoms across time. Here, we address this question with a longitudinal design, investigating if repeated sessions of behavioral training change reward learning and decrease depression symptoms across time.
Methods:
Participants: We recruited 929 online participants from Amazon's Mechanical Turk platform and 40 in-person participants from regions of southwest Virginia to complete our longitudinal study.
Behavioral Training: Participants completed a probabilistic reward learning task with repeated queries about a feature of the task environment (learning queries; Figure 1a) or control queries. Learning queries trained participants on one of four computational-based learning targets known to affect reinforcement learning (Figure 1b). For up to 12 total study visits, participants repeated the task and completed a depression symptom questionnaire two to three times per week. Depression questionnaire scores at baseline were used to split participants into no-depression and depression subgroups, where individuals in the depression subgroup met threshold for at least mild depression severity.
Computational Modeling and Symptom Analysis: A Q-learning model was fit to behavioral responses using hierarchical Bayesian estimation to provide estimates of reward sensitivity and learning rate for each participant on each visit. Reward sensitivity captured participants' value dissociation between high versus low outcome values, while learning rate informed how much participants learned from previously experienced outcomes. Mixed linear models assessed relationships between learning parameters, depression symptoms, and study progression.
fMRI Analysis: Participants that completed the study in-person underwent functional magnetic resonance imaging in a 3T Siemens Prisma Fit on the first and last study visits. Preprocessing of echo-planar images included slice timing correction, realignment, co-registration to a T1 structural image, normalization, and smoothing with a 6x6x6 mm kernel in SPM12. On the individual level, hemodynamic responses to outcome events were assessed. General linear models were used to perform individual and group level analyses.
Behavioral Training: Participants completed a probabilistic reward learning task with repeated queries about a feature of the task environment (learning queries; Figure 1a) or control queries. Learning queries trained participants on one of four computational-based learning targets known to affect reinforcement learning (Figure 1b). For up to 12 total study visits, participants repeated the task and completed a depression symptom questionnaire two to three times per week. Depression questionnaire scores at baseline were used to split participants into no-depression and depression subgroups, where individuals in the depression subgroup met threshold for at least mild depression severity.
Computational Modeling and Symptom Analysis: A Q-learning model was fit to behavioral responses using hierarchical Bayesian estimation to provide estimates of reward sensitivity and learning rate for each participant on each visit. Reward sensitivity captured participants' value dissociation between high versus low outcome values, while learning rate informed how much participants learned from previously experienced outcomes. Mixed linear models assessed relationships between learning parameters, depression symptoms, and study progression.
fMRI Analysis: Participants that completed the study in-person underwent functional magnetic resonance imaging in a 3T Siemens Prisma Fit on the first and last study visits. Preprocessing of echo-planar images included slice timing correction, realignment, co-registration to a T1 structural image, normalization, and smoothing with a 6x6x6 mm kernel in SPM12. On the individual level, hemodynamic responses to outcome events were assessed. General linear models were used to perform individual and group level analyses.
Results:
Across time, learning queries increased reward sensitivities in no-depression participants (β = 0.036, p =< 0.001, 95% CI (0.022, 0.049)). In contrast, control queries did not change reward sensitivities in no-depression participants across time ((β = 0.016, p = 0.303, 95% CI (-0.015, 0.048)). Of the learning queries, those targeting value comparison processes improved depression symptoms (β = -0.509, p = 0.015, 95% CI (-0.912, -0.106)) and increased reward sensitivities across time (β = 0.052, p =< 0.001, 95% CI (0.030, 0.075)) in depression participants. Increased reward sensitivities related to decreased depression symptoms across time in these participants (β = -2.905, p = 0.002, 95% CI (-4.75, -1.114)). In the neuroimaging sample, regardless of time, individuals receiving value comparison queries showed increased outcome activity in the left nucleus accumbens, medial orbitofrontal cortex, and right caudate compared to those who received control queries (Figure 2).
Conclusions:
Behavioral reward learning was improved through repeated sessions of targeted training for participants with a range of clinical symptoms. Improved behavioral reward learning was associated with improved clinical symptoms with time. Neural results showed increased outcome activity in reward circuitry brain regions with targeted training. These results support disrupted reward learning as a mechanism in depression and suggest the potential of behavioral training to target neurobehavioral deficits in reward learning and evoke symptom change.
Disorders of the Nervous System:
Psychiatric (eg. Depression, Anxiety, Schizophrenia) 2
Emotion, Motivation and Social Neuroscience:
Reward and Punishment 1
Higher Cognitive Functions:
Decision Making
Modeling and Analysis Methods:
Activation (eg. BOLD task-fMRI)
Bayesian Modeling
Keywords:
Affective Disorders
Behavioral Therapy
Cognition
Computational Neuroscience
FUNCTIONAL MRI
Learning
Modeling
Psychiatric
Other - Reward Learning
1|2Indicates the priority used for review
Provide references using author date format
2. Pike, A. C., & Robinson, O. J. (2022), ‘Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals: A Systematic Review and Meta-analysis’, JAMA Psychiatry, vol. 79, no. 4, pp. 313–322.
3. Liu, X., Hairston, J., Schrier, M., & Fan, J. (2011), ‘Common and distinct networks underlying reward valence and processing stages: a meta-analysis of functional neuroimaging studies’, Neuroscience and biobehavioral reviews, vol. 35, no. 5, pp. 1219–1236.
4. Diekhof, E. K., Kaps, L., Falkai, P., & Gruber, O. (2012), ‘The role of the human ventral striatum and the medial orbitofrontal cortex in the representation of reward magnitude - an activation likelihood estimation meta-analysis of neuroimaging studies of passive reward expectancy and outcome processing’, Neuropsychologia, vol. 50, no.7, pp. 1252–1266.