Inhibiting Mesolimbic Dopamine Neurons Reduces the Initiation and Maintenance of Instrumental Responding
Introduction
Considerable evidence indicates that brief phasic dopamine (DA) neuron activity can serve as a reward prediction error (RPE) signal capable of supporting certain forms of associative learning. Neural response profiles congruent with this view have been observed in non-human primates and rodents (Cohen et al., 2012, Day et al., 2007, Eshel et al., 2015, Flagel et al., 2011, Hart et al., 2014, Ljungberg et al., 1992, Matsumoto and Hikosaka, 2009, Owesson-White et al., 2008, Roesch et al., 2007, Schultz et al., 1997, Schultz et al., 2015, Waelti et al., 2001), and optogenetic manipulation of VTA DA neuron activity to mimic a positive or negative RPE at the time of reward receipt, or immediately following an action, predictably changes future behavior (Chang et al., 2016, Hamid et al., 2016; Parker et al., 2016; Steinberg et al., 2013, Tsai et al., 2009).
In addition to learning, a role for VTA DA in ongoing motivation, or behavioral activation, has long been appreciated (Berridge, 2007, Berridge and Robinson, 2003, Robbins and Everitt, 2007, Salamone and Correa, 2002, Salamone and Correa, 2012), and has been proposed to vary in relation to current subjective effort/energy utilization requirements (Beeler et al., 2012, Cagniard et al., 2006, Niv, 2007, Ostlund et al., 2012, Salamone and Correa, 2012). For example, DA antagonists and lesions of DA terminals in the nucleus accumbens (NAc) reduce effortful instrumental responding and promote behavioral switching from more to less effortful means to gain access to food reward (Aberman and Salamone, 1999, Hosking et al., 2015, Ostlund et al., 2012, Salamone et al., 1991).
Observation of two modes of DA neuron activity, phasic firing and tonic firing (Goto et al., 2007, Grace, 2000, Grace, 2016, Niv, 2007, Sulzer et al., 2016, Wanat et al., 2009), have led to the assignment of distinct contributions of phasic and tonic DA neuron activity to learning and motivation, respectively, especially when considering DA actions in the NAc (Niv et al., 2007, Parker et al., 2010, Schultz, 2007, Zweifel et al., 2009), with phasic DA changes mediating learning and changes in tonic levels over tens of seconds to minutes impacting motivational variables. However, multiple studies suggest that motivation may be related to changes in VTA/NAc DA levels over relatively short time scales that would typically not be considered ‘tonic’ (Collins et al., 2016, Hamid et al., 2016, Ko and Wanat, 2016, Phillips et al., 2003, Roitman et al., 2004, Wassum et al., 2012a), and have been referred to as ‘slow phasic’, representing changes occurring on the time scale of behavior, yet longer than the time required for a DA neuron burst (Salamone and Correa, 2012). For example, in a recent set of studies using fast-scan cyclic voltammetry (FSCV), Collins and colleagues (2016) directly measured DA in NAc of rats trained to lever press one lever for access to a second lever whose depression led to reward. These authors found that DA concentration increases were sometimes observed to begin prior to lever approach, and to last on the order of 5–10 s (Collins et al., 2016). Features of these increases were correlated with latency to approach the first lever, suggesting that these briefer DA signals alter motivated responding directly (Collins et al., 2016). While these responses changed over days with learning, they also were predictive of behavioral response characteristics, in line with roles of DA in both motivation and learning (Collins et al., 2016, Wassum et al., 2012a). Similar conclusions emerge from a detailed analysis of NAc FSCV DA signals within a reinforcement learning framework; Hamid et al. (2016) found that the dynamics of the DA signal encode both immediate value that acutely impacts effort expenditure, as well as changes in value, or RPEs, that affect future response allocation. Interestingly, the authors found that optogenetic activation of VTA DA neurons time-locked to a trial initiation cue did not affect future choice, i.e., learning, but did decrease response latencies on that same trial. Together these findings suggest that VTA DA neuron activity can impact ongoing responding, congruent with a motivational interpretation, perhaps even on shorter time scales than typically considered to reflect ‘tonic’ DA.
While these and other prior studies indicate contributions to motivated behavior, they generally did not distinguish between a requirement for DA for initiation or maintenance of responding. In a careful series of studies, Nicola and colleagues provide evidence that DA levels in the NAc are critical for ongoing modulation of behavior, specifically for facilitation of approach behavior (Nicola, 2010, Nicola, 2016), an idea that would place the requirement for DA at the initiation phase of action sequences, rather than the expression. On the other hand, findings of selective effects of DA lesions on responding under larger ratio requirements as compared with continuous reinforcement (Aberman and Salamone, 1999, Salamone et al., 2001), as well as the observation of DA increases from the initiation to the full execution of a response sequence (Collins et al., 2016) might suggest that DA also is important for maintenance of longer sequences of responses.
Here we tested whether suppression of DA neuron activity when mice were required to execute a sequence of responses to obtain reward would impair performance, using optogenetic inhibition of TH+ neurons in the midbrain. We trained TH-IRES-Cre mice on appetitive instrumental procedures for food reward then triggered DA neuron inhibition during different behavioral epochs. Based on the prior findings discussed above, we chose to inhibit DA neurons at two distinct time points during these instrumental procedures. In the first condition, we triggered inhibition while mice were engaged in off-task behavior and measured the probability that mice would reinitiate instrumental responding while DA neurons were inhibited. In the second condition, we triggered inhibition just after mice initiated an instrumental response and were therefore actively engaged in a bout of responding. We found that TH+ neuron inhibition applied both prior to or during a sequence of instrumental actions made to obtain reward reduces the probability that reward-seeking actions will be elicited during the time of inhibition; this is an acute effect that fully recovers when the inhibition ceases. Thus inhibition can acutely decrease the probability of reward-seeking actions, in agreement with contributions of DA to ongoing responding during both the initiation and maintenance phases of instrumental behavior.
Section snippets
Subjects
Male tyrosine hydroxylase (TH)-IRES-Cre mice aged 8–12 weeks were housed individually on a reverse 12-h light/dark cycle (lights off at 10:00). All mice began behavioral training with standard rodent chow and water available ad libitum. Mice were tested under food restriction and were brought down to 90% of their free-fed body weight over 5 days before the start of the behavioral test. All experimental procedures were approved by the Institutional Animal Care and Use Committee of the Ernest
Results
We used in vivo optogenetic techniques with the aim of selectively inhibiting midbrain DA neurons of mice during behavior. To permit inhibition of TH+ neurons in the VTA, TH-IRES-Cre mice received injections of Cre-dependent AAV virus (AAV5-Ef1a-DIO-eNpHR3.0-eYFP) expressing the light-sensitive inhibitory channel, halorhodopsin 3.0 (NpHR), in the VTA; chronic optical fiber implants were targeted dorsal to this region to allow for selective bilateral optogenetic DA neuron inhibition (Fig. 1).
Discussion
Here we demonstrate that inhibition of midbrain DA neurons impairs performance of motivated behavior. We first show that DA neuron activity at the time when mice are not engaged in instrumental behavior is required to promote initiation of an instrumental response. In addition, DA neuron activity when mice are engaged in an instrumental response promotes the continuation of responding until completion of the response requirement. Finally, because in most cases mice readily compensate for nose
Acknowledgments
This work was supported by National Institutes of Health Grant R01 DA035943 and funds from the State of California through UCSF. Author contributions: Experimental conception, design and interpretation: SFW, PHJ; data collection: SFW, RMR; data analysis: SFW; writing and editing: SFW, PHJ.
References (64)
- et al.
Nucleus accumbens dopamine depletions make rats more sensitive to high ratio requirements but do not impair primary food reinforcement
Neuroscience
(1999) - et al.
Effects of dopamine antagonists and accumbens dopamine depletions on time-constrained progressive-ratio performance
Pharmacol Biochem Behav
(1998) - et al.
Parsing reward
Trends Neurosci
(2003) - et al.
Dopamine scales performance in the absence of new learning
Neuron
(2006) - et al.
The Yin and Yang of dopamine release: a new perspective
Neuropharmacology
(2007) - et al.
Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine
Behav Brain Res.
(2002) - et al.
The mysterious motivational functions of mesolimbic dopamine
Neuron
(2012) - et al.
Nucleus accumbens dopamine depletions make animals highly sensitive to high fixed ratio requirements but do not impair primary food reinforcement
Neuroscience
(2001) - et al.
Phasic dopamine signals: from subjective reward value to formal economic utility
Curr Opin Behav Sci
(2015) - et al.
Establishing causality for dopamine in neural function and behavior with optogenetics
Brain Res
(2013)
Extinction of cocaine self-administration reveals functionally and temporally distinct dopaminergic signals in the nucleus accumbens
Neuron
Striatal dopamine neurotransmission: regulation of release and uptake
Basal Ganglia
GABA neurons of the VTA drive conditioned place aversion
Neuron
Phasic mesolimbic dopamine signaling precedes and predicts performance of a self-initiated action sequence task
Biol Psychiatry
Putting desire on a budget: dopamine and energy expenditure, reconciling reward and resources
Front Integr Neurosci
The debate over dopamine’s role in reward: the case for incentive salience
Psychopharmacology
Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors
Nat Neurosci
Rapid regulation of depression-related behaviours by control of midbrain dopamine neurons
Nature
Neuron-type-specific signals for reward and punishment in the ventral tegmental area
Nature
Dynamic mesolimbic dopamine signaling during action sequence learning and expectation violation
Sci Rep
Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens
Nat Neurosci
Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approach behavior
J Neurosci
Arithmetic and local circuitry underlying dopamine prediction errors
Nature
A selective role for dopamine in stimulus-reward learning
Nature
Long-lasting contribution of dopamine in the nucleus accumbens core, but not dorsal lateral striatum, to sign-tracking
Eur J Neurosci
The tonic/phasic model of dopamine system regulation and its implications for understanding alcohol and psychostimulant craving
Addict Abingdon Engl
Dysregulation of the dopamine system in the pathophysiology of schizophrenia and depression
Nat Rev Neurosci
Mesolimbic dopamine signals the value of work
Nat Neurosci
Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term
J Neurosci
Dopamine antagonism decreases willingness to expend physical, but not cognitive, effort: a comparison of two rodent cost/benefit decision-making tasks
Neuropsychopharmacology
Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion
J Neurosci
Start/stop signals emerge in nigrostriatal circuits during sequence learning
Nature
Cited by (29)
Deconstructing and reconstructing behaviour relevant to mental health disorders: The benefits of a psychological approach, with a focus on addiction
2022, Neuroscience and Biobehavioral Reviews