Elsevier

Neuroscience

Volume 372, 21 February 2018, Pages 306-315
Neuroscience

Inhibiting Mesolimbic Dopamine Neurons Reduces the Initiation and Maintenance of Instrumental Responding

https://doi.org/10.1016/j.neuroscience.2017.12.003Get rights and content

Highlights

  • Is midbrain dopamine neuron activity required for responding on fixed ratio and progressive ratio tasks in mice?

  • Optogenetic inhibition of dopamine neurons decreases the likelihood of initiation a response bout.

  • Optogenetic inhibition of dopamine neurons after bout initiation decreases the likelihood of bout completion.

  • Mice counteract depressant effects of dopamine inhibition during no-inhibition periods to maintain overall reward levels.

Abstract

Mesolimbic dopamine perturbations modulate performance of reward-seeking behavior, with tasks requiring high effort being especially vulnerable to disruption of dopamine signaling. Previous work primarily investigated long-term perturbations such as receptor antagonism and dopamine depletion, which constrain the ability to assess dopamine contributions to effort expenditure in isolation from other behavior events, such as reward consumption. Also unclear is if dopamine is required for both initiation and maintenance when a sequence of multiple instrumental responses is required. Here we used optogenetic inhibition of midbrain TH+  neurons to probe the role of dopamine neuron activity during instrumental responding for reward by varying the time epoch of neural inhibition relative to the time of response initiation. Within a fixed-ratio procedure, requiring eight nosepoke responses per reinforcer delivery, or a progressive ratio (PR) procedure, in which within-session response requirements increased exponentially, inhibiting dopamine neurons while mice were engaged in response bouts decreased the probability of continued responding. If inhibition occurred during each attempted bout, the effect was to decrease total responses, and thus amount of rewards earned, over a session. In contrast, if inhibition was applied only during some bouts, mice increased the number of bouts initiated to earn control levels of reward. Inhibiting dopamine neurons while mice were not responding decreased the probability of initiating an instrumental response but had no effect on the amount of effort exerted over the entire session. We conclude that midbrain dopamine signaling promotes initiation of instrumental responding and maintains motivation to continue ongoing bouts of effortful responses.

Introduction

Considerable evidence indicates that brief phasic dopamine (DA) neuron activity can serve as a reward prediction error (RPE) signal capable of supporting certain forms of associative learning. Neural response profiles congruent with this view have been observed in non-human primates and rodents (Cohen et al., 2012, Day et al., 2007, Eshel et al., 2015, Flagel et al., 2011, Hart et al., 2014, Ljungberg et al., 1992, Matsumoto and Hikosaka, 2009, Owesson-White et al., 2008, Roesch et al., 2007, Schultz et al., 1997, Schultz et al., 2015, Waelti et al., 2001), and optogenetic manipulation of VTA DA neuron activity to mimic a positive or negative RPE at the time of reward receipt, or immediately following an action, predictably changes future behavior (Chang et al., 2016, Hamid et al., 2016; Parker et al., 2016; Steinberg et al., 2013, Tsai et al., 2009).

In addition to learning, a role for VTA DA in ongoing motivation, or behavioral activation, has long been appreciated (Berridge, 2007, Berridge and Robinson, 2003, Robbins and Everitt, 2007, Salamone and Correa, 2002, Salamone and Correa, 2012), and has been proposed to vary in relation to current subjective effort/energy utilization requirements (Beeler et al., 2012, Cagniard et al., 2006, Niv, 2007, Ostlund et al., 2012, Salamone and Correa, 2012). For example, DA antagonists and lesions of DA terminals in the nucleus accumbens (NAc) reduce effortful instrumental responding and promote behavioral switching from more to less effortful means to gain access to food reward (Aberman and Salamone, 1999, Hosking et al., 2015, Ostlund et al., 2012, Salamone et al., 1991).

Observation of two modes of DA neuron activity, phasic firing and tonic firing (Goto et al., 2007, Grace, 2000, Grace, 2016, Niv, 2007, Sulzer et al., 2016, Wanat et al., 2009), have led to the assignment of distinct contributions of phasic and tonic DA neuron activity to learning and motivation, respectively, especially when considering DA actions in the NAc (Niv et al., 2007, Parker et al., 2010, Schultz, 2007, Zweifel et al., 2009), with phasic DA changes mediating learning and changes in tonic levels over tens of seconds to minutes impacting motivational variables. However, multiple studies suggest that motivation may be related to changes in VTA/NAc DA levels over relatively short time scales that would typically not be considered ‘tonic’ (Collins et al., 2016, Hamid et al., 2016, Ko and Wanat, 2016, Phillips et al., 2003, Roitman et al., 2004, Wassum et al., 2012a), and have been referred to as ‘slow phasic’, representing changes occurring on the time scale of behavior, yet longer than the time required for a DA neuron burst (Salamone and Correa, 2012). For example, in a recent set of studies using fast-scan cyclic voltammetry (FSCV), Collins and colleagues (2016) directly measured DA in NAc of rats trained to lever press one lever for access to a second lever whose depression led to reward. These authors found that DA concentration increases were sometimes observed to begin prior to lever approach, and to last on the order of 5–10 s (Collins et al., 2016). Features of these increases were correlated with latency to approach the first lever, suggesting that these briefer DA signals alter motivated responding directly (Collins et al., 2016). While these responses changed over days with learning, they also were predictive of behavioral response characteristics, in line with roles of DA in both motivation and learning (Collins et al., 2016, Wassum et al., 2012a). Similar conclusions emerge from a detailed analysis of NAc FSCV DA signals within a reinforcement learning framework; Hamid et al. (2016) found that the dynamics of the DA signal encode both immediate value that acutely impacts effort expenditure, as well as changes in value, or RPEs, that affect future response allocation. Interestingly, the authors found that optogenetic activation of VTA DA neurons time-locked to a trial initiation cue did not affect future choice, i.e., learning, but did decrease response latencies on that same trial. Together these findings suggest that VTA DA neuron activity can impact ongoing responding, congruent with a motivational interpretation, perhaps even on shorter time scales than typically considered to reflect ‘tonic’ DA.

While these and other prior studies indicate contributions to motivated behavior, they generally did not distinguish between a requirement for DA for initiation or maintenance of responding. In a careful series of studies, Nicola and colleagues provide evidence that DA levels in the NAc are critical for ongoing modulation of behavior, specifically for facilitation of approach behavior (Nicola, 2010, Nicola, 2016), an idea that would place the requirement for DA at the initiation phase of action sequences, rather than the expression. On the other hand, findings of selective effects of DA lesions on responding under larger ratio requirements as compared with continuous reinforcement (Aberman and Salamone, 1999, Salamone et al., 2001), as well as the observation of DA increases from the initiation to the full execution of a response sequence (Collins et al., 2016) might suggest that DA also is important for maintenance of longer sequences of responses.

Here we tested whether suppression of DA neuron activity when mice were required to execute a sequence of responses to obtain reward would impair performance, using optogenetic inhibition of TH+ neurons in the midbrain. We trained TH-IRES-Cre mice on appetitive instrumental procedures for food reward then triggered DA neuron inhibition during different behavioral epochs. Based on the prior findings discussed above, we chose to inhibit DA neurons at two distinct time points during these instrumental procedures. In the first condition, we triggered inhibition while mice were engaged in off-task behavior and measured the probability that mice would reinitiate instrumental responding while DA neurons were inhibited. In the second condition, we triggered inhibition just after mice initiated an instrumental response and were therefore actively engaged in a bout of responding. We found that TH+ neuron inhibition applied both prior to or during a sequence of instrumental actions made to obtain reward reduces the probability that reward-seeking actions will be elicited during the time of inhibition; this is an acute effect that fully recovers when the inhibition ceases. Thus inhibition can acutely decrease the probability of reward-seeking actions, in agreement with contributions of DA to ongoing responding during both the initiation and maintenance phases of instrumental behavior.

Section snippets

Subjects

Male tyrosine hydroxylase (TH)-IRES-Cre mice aged 8–12 weeks were housed individually on a reverse 12-h light/dark cycle (lights off at 10:00). All mice began behavioral training with standard rodent chow and water available ad libitum. Mice were tested under food restriction and were brought down to 90% of their free-fed body weight over 5 days before the start of the behavioral test. All experimental procedures were approved by the Institutional Animal Care and Use Committee of the Ernest

Results

We used in vivo optogenetic techniques with the aim of selectively inhibiting midbrain DA neurons of mice during behavior. To permit inhibition of TH+ neurons in the VTA, TH-IRES-Cre mice received injections of Cre-dependent AAV virus (AAV5-Ef1a-DIO-eNpHR3.0-eYFP) expressing the light-sensitive inhibitory channel, halorhodopsin 3.0 (NpHR), in the VTA; chronic optical fiber implants were targeted dorsal to this region to allow for selective bilateral optogenetic DA neuron inhibition (Fig. 1).

Discussion

Here we demonstrate that inhibition of midbrain DA neurons impairs performance of motivated behavior. We first show that DA neuron activity at the time when mice are not engaged in instrumental behavior is required to promote initiation of an instrumental response. In addition, DA neuron activity when mice are engaged in an instrumental response promotes the continuation of responding until completion of the response requirement. Finally, because in most cases mice readily compensate for nose

Acknowledgments

This work was supported by National Institutes of Health Grant R01 DA035943 and funds from the State of California through UCSF. Author contributions: Experimental conception, design and interpretation: SFW, PHJ; data collection: SFW, RMR; data analysis: SFW; writing and editing: SFW, PHJ.

References (64)

  • G.D. Stuber et al.

    Extinction of cocaine self-administration reveals functionally and temporally distinct dopaminergic signals in the nucleus accumbens

    Neuron

    (2005)
  • D. Sulzer et al.

    Striatal dopamine neurotransmission: regulation of release and uptake

    Basal Ganglia

    (2016)
  • K.R. Tan et al.

    GABA neurons of the VTA drive conditioned place aversion

    Neuron

    (2012)
  • K.M. Wassum et al.

    Phasic mesolimbic dopamine signaling precedes and predicts performance of a self-initiated action sequence task

    Biol Psychiatry

    (2012)
  • J.A. Beeler et al.

    Putting desire on a budget: dopamine and energy expenditure, reconciling reward and resources

    Front Integr Neurosci

    (2012)
  • K.C. Berridge

    The debate over dopamine’s role in reward: the case for incentive salience

    Psychopharmacology

    (2007)
  • C.Y. Chang et al.

    Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors

    Nat Neurosci

    (2016)
  • D. Chaudhury et al.

    Rapid regulation of depression-related behaviours by control of midbrain dopamine neurons

    Nature

    (2013)
  • J.Y. Cohen et al.

    Neuron-type-specific signals for reward and punishment in the ventral tegmental area

    Nature

    (2012)
  • A.L. Collins et al.

    Dynamic mesolimbic dopamine signaling during action sequence learning and expectation violation

    Sci Rep

    (2016)
  • J.J. Day et al.

    Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens

    Nat Neurosci

    (2007)
  • P. Di Ciano et al.

    Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approach behavior

    J Neurosci

    (2001)
  • N. Eshel et al.

    Arithmetic and local circuitry underlying dopamine prediction errors

    Nature

    (2015)
  • S.B. Flagel et al.

    A selective role for dopamine in stimulus-reward learning

    Nature

    (2011)
  • K.M. Fraser et al.

    Long-lasting contribution of dopamine in the nucleus accumbens core, but not dorsal lateral striatum, to sign-tracking

    Eur J Neurosci

    (2017)
  • A.A. Grace

    The tonic/phasic model of dopamine system regulation and its implications for understanding alcohol and psychostimulant craving

    Addict Abingdon Engl

    (2000)
  • A.A. Grace

    Dysregulation of the dopamine system in the pathophysiology of schizophrenia and depression

    Nat Rev Neurosci

    (2016)
  • A.A. Hamid et al.

    Mesolimbic dopamine signals the value of work

    Nat Neurosci

    (2016)
  • A.S. Hart et al.

    Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term

    J Neurosci

    (2014)
  • J.G. Hosking et al.

    Dopamine antagonism decreases willingness to expend physical, but not cognitive, effort: a comparison of two rodent cost/benefit decision-making tasks

    Neuropsychopharmacology

    (2015)
  • A. Ilango et al.

    Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion

    J Neurosci

    (2014)
  • X. Jin et al.

    Start/stop signals emerge in nigrostriatal circuits during sequence learning

    Nature

    (2010)
  • Cited by (29)

    View all citing articles on Scopus
    View full text