II. Neural Circuits and Transmitters Mediating Effort-Based Choice: Mesolimbic Dopamine

Basic research with rodents has demonstrated that effort-based decision making is regulated by a distributed forebrain circuity that involves many brain areas and neurotransmitters, with mesolimbic dopamine (DA) playing a major role (Salamone et al., 2016a,b,c). Typically, effort-based decision making is assessed using tasks that offer animals a choice between a relatively preferred reinforcer that can only be obtained by a high exertion of effort versus a lower effort/lower value option (Fig. 1). Some tasks use operant behavior methods to study response allocation under conditions in which working for a preferred food by lever pressing is the high-effort option and simply approaching and consuming a less preferred laboratory chow is the low-effort option. Using this type of procedure, several studies have employed tasks in which the lever-pressing option is a type of fixed ratio (FR) or progressive ratio (PROG) schedule that is reinforced by relatively preferred high carbohydrate operant pellets, and the low-effort/low-preference alternative is standard laboratory chow that is freely available in the chamber. (Salamone et al., 1991; Randall et al., 2012; Sommer et al., 2014; Fig. 1, left). With the concurrent FR5/chow-feeding choice task (Salamone et al., 1991; Yohn et al., 2016a,b,c,d,e), both options are available concurrently, and animals are free to allocate their behavior across the different choices. In contrast, with the most commonly used version of the PROG/chow-feeding choice procedure (Randall et al., 2012, 2015a,b), if an animal fails to obtain a reinforcer within a 2-minute period, the lever-pressing option will end, and the only option is to approach and consume chow. A discrete trial T-maze barrier choice task also has been developed (Salamone et al., 1994; Pardo et al., 2012; Yohn et al., 2015a,b; Fig. 1, right), and with this procedure rodents have a choice across multiple trials between one arm that contains a high magnitude of food and the other arm that contains a low density of food (e.g., four pellets versus two pellets). An effort-related challenge is provided under some conditions, in which the high density of reinforcement can only be obtained by climbing a vertical barrier. Furthermore, there are effort-discounting procedures that have been used, which employ either different FR lever-pressing requirements or a T-maze with a barrier to control the effort component (Floresco et al., 2008; Bardgett et al., 2009).

• Download figure
• Open in new tab
• Download powerpoint

Fig. 1.

Effort-related choice tasks conducted in rodent operant boxes and mazes (see cited references in text for details). Left: The concurrent lever-pressing/chow-feeding choice tasks give animals the option of lever pressing on a fixed or progressive ratio schedule to obtain access to preferred high-carbohydrate pellets versus approaching and consuming freely available chow. Right: The T-maze barrier choice task presents trials that allow animals to select one arm that requires them to climb a vertical barrier to obtain a large magnitude of reinforcement versus the other arm, which has no barrier and a lower magnitude of reinforcement.

Considerable evidence implicates brain DA systems, particularly mesolimbic DA, in regulating behavioral activation, the exertion of effort, and effort-based decision making. Motivational conditions such as periodic presentation of small amounts of food (e.g., single 45 mg food pellets) to food-restricted rats can generate intense activities such as polydipsia, wheel running, and high levels of motor activity, and evidence indicates that DA antagonism and depletion of mesolimbic DA blunt these activities (Robbins and Koob, 1980; Wallace et al., 1983; Salamone, 1986, 1988; McCullough and Salamone, 1992). One way of controlling the work requirement of an operant schedule is to vary the ratio requirement (i.e., the number of lever presses required for reinforcement). Several studies have shown that the effects of DA antagonism and nucleus accumbens DA depletions interact with the ratio requirements of an operant procedure, and that impairments are greater on schedules with a larger ratio requirement (Aberman and Salamone, 1999; Caul and Brindle, 2001; Salamone et al., 2001; Ishiwari et al., 2004). If ratio requirements are attached to the time interval requirements of an operant schedule (i.e., a variable interval schedule with a ratio requirement attached in tandem), the effects of nucleus accumbens DA depletions interact with the ratio requirements of the schedule, but not the time interval requirement (Correa et al., 2002; Mingote et al., 2005). This observation is consistent with the results of studies showing that interference with accumbens DA transmission did not affect delay discounting (Winstanley et al., 2005; Li et al., 2015) or progressive interval lever pressing (Wakabayashi et al., 2004).

Across a wide variety of tasks, administration of low doses of DA antagonists and accumbens DA depletions has been shown to alter effort-based decision making, producing a low-effort bias that shifts animals away from the high-effort option and toward the low-effort choice. In studies that have used the concurrent FR5/chow-feeding choice procedure, a low-effort bias can be reliably induced by systemic administration of low doses of DA D1 or D2 family antagonists such as SCH23390, ecopipam, haloperidol, raclopride, and eticlopride (Salamone et al., 1991, 2002; Cousins and Salamone, 1994; Koch et al., 2000; Farrar et al., 2007; Sink et al., 2008), as well as by intra-accumbens DA antagonism (Nowend et al., 2001; Farrar et al., 2010) and accumbens DA depletions produced by local injections of 6-hydroxydopamine (Salamone et al., 1991; Cousins et al., 1993; Sokolowski and Salamone, 1998). This shift in choice is marked by a substantial reduction in lever pressing and a concomitant increase in chow intake. PROG/chow-feeding choice performance also has been used to assess the effort-related effects of DA antagonism. Administration of DA D1 or D2 family antagonists reduced PROG lever pressing, but, on this task, which generates relatively high baseline levels of chow intake, DA antagonism did not suppress intake of the concurrently available chow, in fact tending to increase it still further (Randall et al., 2012, 2015a). Several papers have reported that T-maze barrier choice performance also is affected by DA antagonism and depletion; these manipulations reduce climbing of the maze arm with the barrier (the high e-effort option) and increase selection of the arm with no barrier and the lower magnitude of reward (Salamone et al., 1994; Cousins et al., 1996; Mott et al., 2009; Pardo et al., 2012; Yohn et al., 2015a). Furthermore, tests of effort discounting using both lever-pressing and maze procedures have shown that DA D1 and D2 antagonism shifted choice in the direction of a low-effort preference (Floresco et al., 2008; Bardgett et al., 2009; Hosking et al., 2015).

Recent studies have involved administration of the pharmacological DA-depleting agent tetrabenazine. Tetrabenazine is a potent and reversible inhibitor of the vesicular monoamine transporter type-2, and blockade of this protein inhibits vesicular storage of monoamines, thereby leading to neurotransmitter depletion. At low doses, tetrabenazine is relatively selective at reducing levels of DA compared with norepinephrine (NE) and 5-HT (Pettibone et al., 1984; Tanra et al., 1995). Low doses of tetrabenazine that reduced extracellular DA in nucleus accumbens by 75% and altered DA-related signal transduction as measured by alterations in phosphorylated DARPP-32 (DA and cAMP-related phophoprotein, 32 kDa) were shown to shift choice behavior and induce a low-effort bias in rats tested on the FR5/chow-feeding choice task (Nunes et al., 2013b). The shift from lever pressing to chow intake was seen after local injections of tetrabenazine into nucleus accumbens core, but not into overlying dorsomedial neostriatum (Nunes et al., 2013b). Tetrabenazine also alters effort-based choice in rats tested on a concurrent FR7/choice procedure that used different sucrose concentrations as the reinforcer (Pardo et al., 2015), the PROG/chow-feeding choice task (Randall et al., 2015a), and the T-maze barrier choice task (Yohn et al., 2015a,b).

III. Behavioral Effects of Dopaminergic Manipulations on Effort-Related Tasks Are Not Due to Broad or General Changes in Reward, Temporal Processing, or Motor Incapacity

As described above, across diverse tasks conducted in multiple laboratories, using a number of different pharmacological and neurotoxic tools, it has been shown that interference with DA transmission shifted choice behavior from task components that require a high degree of effort to those that involve a lower exertion of effort. Of course, to interpret these effects as being dependent upon the effort-related features of the task, it is important to demonstrate that they are not due to other potential manifestations of reduced DA transmission. For example, whenever one studies any aspect of the behavioral functions of DA, a commonplace comment may be that a global or general deficit in reward underlies the effect being reported. The DA hypothesis of reward is a ubiquitous feature of the scientific literature, as well as popular media, the internet, and film. Yet, despite the almost automatic tendency of some to explain virtually any aspect of DA function as somehow being dependent on reward, there are critical theoretical and empirical problems with this, many of which have been reviewed in detail elsewhere (Salamone et al., 1997, 2007; Salamone and Correa, 2002, 2012; Floresco, 2015; Nicola, 2016). First and foremost, the term reward has no consistent scientific meaning (Salamone et al., 2005; Salamone and Correa, 2012), and, depending upon the paper, or even the paragraph, this term is used variously to refer to subjective pleasure or hedonic reactivity, appetite, preference, and even reinforcement learning. Given the slippery and imprecise nature of this term, it is wholly inadequate to attribute specific effects in experiments simply to reward without any qualification or explication. Nevertheless, it is useful, even critical, to determine whether impairments in specific aspects of appetitive motivation such as reinforcer preference, appetite, or hedonic reactivity, as well as other factors such as delay of reinforcement or motoric capacity, underlie the behavioral effects of dopaminergic manipulations on effort-related performance.

The most common reinforcer used in studies of effort-based choice is food, and an enormous body of evidence has demonstrated that the dopaminergic conditions that produce a low-effort bias in rodents do not alter food intake, preference for the different foods used, or discrimination of reinforcer magnitude, and do not reduce sucrose preference or hedonic reactivity. Although the concurrent FR5/ and PROG/chow-feeding choice procedures involve lever pressing for one type of food (high-carbohydrate pellets) versus approaching and consuming a different food (standard laboratory chow), free feeding preference studies have shown that the conditions that produce the shift in choice behavior (low doses of DA antagonists, DA depletions) do not reduce total food intake or the preference for high-carbohydrate pellets over chow (Salamone et al., 1991; Koch et al., 2000; Nunes et al., 2013b). With animals tested in the T-maze barrier task, DA antagonism and depletion shift arm choice away from the arm with the barrier that has the high density of food reinforcement; however, these manipulations have no effect on choice of the high-reinforcement versus low-reinforcement arm when no barrier is present (Salamone et al., 1994; Yohn et al., 2015a,b). Although tetrabenazine was shown to decrease lever pressing for a high concentration of sucrose and increase intake of a concurrently available lower concentration, it did not alter intake of or preference for the different concentrations of sucrose used (Correa et al., 2016). Moreover, tetrabenazine did not alter markers of hedonic reactivity to sucrose (Correa et al., 2016), which is consistent with the large body of research from Berridge, Robinson, and colleagues indicating that interference with DA transmission does not alter hedonic reactivity to sucrose (Berridge, 2007; Berridge and Robinson, 1998, 2003). Thus, dopaminergic manipulations that affect effort-based choice are not altering choice based upon the quality or quantity of food; they are altering the choice of instrumental behavior that the animal uses to obtain food, and altering it in such a way as to bias the animal toward the lower effort response. Furthermore, studies have shown repeatedly that the effects of interference with DA transmission on effort-based choice do not resemble the effects of reinforcer devaluation by prefeeding (Salamone et al., 1991; Randall et al., 2012; Pardo et al., 2015) or appetite-suppressant drugs (Salamone et al., 2002; Sink et al., 2008; Randall et al., 2012, 2015a). Taken together, these results demonstrate that the induction of a low-effort bias in animals treated with DA antagonists or depleting agents is not due to impairments in the primary or unconditioned reinforcing characteristics of food (Kelley et al., 2005; Salamone et al., 2007, 2016a,b,c).

As noted above, choosing to lever press for one type of food versus approaching and consuming another, or selecting one maze arm versus another, naturally involves a cost/benefit analysis. There are response costs related to effort and time that are involved in any action, and thus opportunity costs involving effort and time when one chooses one action over another. Because lever pressing on a ratio schedule or running in a maze and climbing a barrier both involve time, it is useful to consider the potential role of intermittence or delay of reinforcement as a possible mediating variable in studies of effort-based choice. A described above, there is evidence that intermittence of reinforcement per se is not an important determinant of the effects of accumbens DA depletions or antagonism on instrumental response output (Correa et al., 2002; Wakabayashi et al., 2004; Mingote et al., 2005). Moreover, studies have reported that delay discounting is not affected by accumbens DA depletions (Winstanley et al., 2005) or intra-accumbens DA antagonism (Li et al., 2015). Furthermore, Floresco et al. (2008) developed an equivalent delay procedure that was designed to control for the effects of the delays involved in ratio lever pressing on delay discounting performance. They observed that the DA antagonist flupenthixol produced a low-effort bias in tests of ratio discounting even when controlling for response-related delays (Floresco et al., 2008). Thus, it does not appear that time delays in themselves can provide the sole explanation for the pattern of effects induced by DA antagonism or depletion in studies of effort-based choice. Nevertheless, it is possible that time becomes more relevant when sustained effort over very large units of time is required. For example, although animals with accumbens DA depletions can emit normal numbers of responses when ratio requirements are low, they show a catastrophic loss of responding when very large ratios (e.g., FR300) are required in the absence of primary reinforcement, even if the overall magnitude of reinforcement is the same across ratio schedules (Salamone et al., 2001). In that case, these results suggest that the deficit induced by DA depletion is not simply one of reducing the total number of responses, but instead appears to interact with how the response requirement is organized.

It is generally recognized that DA systems are involved in aspects of motor function, so it is natural to ask whether the effects of dopaminergic manipulations on effort-based choice are simply due to a deficit in motor capacity, or an absolute ceiling on the number of times an animal can press a lever or climb a barrier. On the one hand, there is evidence demonstrating that the changes in effort-related performance that follow DA depletions do not depend simply on an impairment in the absolute number of responses that can be emitted by the animal. Cousins and Salamone (1994) reported that in rats with moderate depletions of accumbens DA depletions, deficits were seen in FR5 responding when the animals were tested on the concurrent FR5/chow-feeding choice task (i.e., when they had an alternative food source available), but the same animals did not show significant reductions in FR5 lever pressing when no other food source was available, and the only way to obtain food was through lever pressing. Mice tested on the T-maze barrier choice after injections of haloperidol showed no deficit in arm selection when both arms had a barrier (Pardo et al., 2012). Similarly, DA-depleted rats tested on the T-maze barrier choice task showed deficits in barrier climbing when the no-barrier arm contained some food, but did not show significant reductions in barrier climbing when that was the only way to obtain food (Cousins et al., 1996; Yohn et al., 2015b). Furthermore, analyses of latency data in animals tested on the T-maze task have shown that run latency is not a statistical mediator of the effect of dopaminergic manipulations on arm choice (Yohn et al., 2015a,b). Thus, it appears that the animals with impaired DA transmission are capable of responding, but they are selecting the path to food reinforcement that involves less effort.

IV. What Fundamental Processes Underlie the Low-Effort Bias Induced by Interference with Dopamine Transmission?

As described above, an enormous body of evidence demonstrates that one can induce profound alterations in the allocation of instrumental responses in animals responding on choice tasks by interfering with DA transmission, either by low doses of DA antagonists, pharmacological depletion of DA, or local DA antagonism or depletion in nucleus accumbens. By a process of elimination, these behavioral effects are not easily attributable to a reduction in primary food motivation, appetite, reinforcer preference, reference memory, delay of reinforcement, or a severe motor impairment that renders the organism incapable of responding. In that case, it is worthwhile to consider what the fundamental neurobehavioral processes are that mediate the effects of dopaminergic manipulations on effort-based choice. As alluded to in a previous paper, the motivational functions of mesolimbic DA appear to be somewhat mysterious (Salamone and Correa, 2012), and there continues to be debate about the specific role that DA systems play in aspects of motivation (Nicola, 2016). The section below will provide a brief theoretical perspective on concepts that help to explain how some of the diverse motivational functions of mesolimbic DA are particularly relevant for understanding the impact of dopaminergic manipulations on effort-related choice.

Perhaps the most important contribution of behavioral sciences to neuroscience is the idea that complex psychologic processes such as motivation, cognition, motor function, and emotion are not unitary in nature, but in fact can be deconstructed into dissociable parts by manipulating the nervous system (Salamone and Correa, 2002; Berridge and Robinson, 2003; Salamone et al., 2007). Furthermore, it is necessary to recognize that some of these dissociated component functions fall into areas of overlap between traditional constructs (Salamone and Correa, 2017). For example, the behavioral activation functions of mesolimbic DA are not strictly motivational or motoric in nature because there is actually a fundamental overlap between activational aspects of motivation and movement control that has been recognized and discussed for decades (Mogenson et al., 1980; Salamone, 1992; Salamone and Correa, 2002, 2012; Salamone et al., 2017). Thus, it is critical to consider not only what is impaired by interference with DA transmission but also what is intact, and to decipher the meaning behind this pattern of results.

Animals with impaired mesolimbic DA transmission remain directed toward the acquisition and consumption of food reinforcement (Koob et al., 1978; Salamone et al., 1991, 1993, 1994; Kelley et al., 2005; Nunes et al., 2013b; Yohn et al., 2015b), and, when tested on instrumental tasks that involve minimal work, they show little or no impairment (McCullough et al., 1993; Salamone et al., 2001, 2017; Correa et al., 2002; Ishiwari et al., 2004; Mingote et al., 2005). However, motivational stimuli have several distinct behavioral effects, and it appears that interference with DA transmission can impair some of them, despite leaving aspects of primary or unconditioned reinforcement intact. Motivational stimuli that act as positive reinforcers can have behaviorally activating effects and can support the instigation and maintenance of highly vigorous instrumental behaviors. Associative processes that underlie primary reinforcement can be intertwined with Pavlovian processes, and conditioned stimuli allow organisms to predict and anticipate reinforcement, which is necessary for promoting and sustaining instrumental behavior.

The behaviorally selective effects of DA antagonism or depletion on specific aspects of motivational processes can lead to interesting dissociations. Systemic administration of haloperidol can dramatically reduce the locomotor activation induced by scheduled food presentation, while leaving performance of a very simple instrumental response intact (Salamone, 1986, 1988). Low doses of DA antagonists and accumbens DA depletions can leave a relatively simple instrumental behavior intact (e.g., FR1 lever pressing), but, with larger and larger ratio requirements (FR5 up to FR300), a clear and ultimately catastrophic impairment emerges (McCullough et al., 1993; Aberman and Salamone, 1999; Salamone et al., 2001, 2017; Ishiwari et al., 2004). In behavioral economic terms, interference with DA transmission increases elasticity of demand for food reinforcement and reduces the willingness to pay for food reinforcement while leaving preference-related aspects of value intact (Aberman and Salamone, 1999; Salamone et al., 2009, 2016a,c, 2017). Furthermore, considerable evidence indicates that accumbens DA regulates Pavlovian approach behavior and the activating effects of Pavlovian-conditioned stimuli on instrumental behavior (Wyvell and Berridge, 2000; Parkinson et al., 2002; Lex and Hauber, 2008).

Taking all this together, it is reasonable to suggest that many of the behaviors that are highly sensitive to interference with mesolimbic DA transmission are vigorous instrumental activities that are instigated or sustained by conditioned stimuli (Salamone and Correa, 2012). This view of DA function resonates with classic ideas about the role of conditioned stimuli in motivated behavior. Cofer and Appley (1964) suggested that there was an anticipation–invigoration mechanism, which is turned on by conditioned stimuli and then serves to activate or invigorate instrumental behavior. Because pursuit of reinforcement in the laboratory or the wild often requires considerable response vigor, and actions must be instigated in the absence of primary reinforcement, a major function of mesolimbic DA involves bridging the psychologic distance that separates organisms from motivationally significant stimuli (Salamone and Correa, 2012). In the context of effort-related decision-making tasks, in which behavior is supported by primary reinforcement but also by discrete cues or contextual stimuli, the net result of interfering with accumbens DA transmission is to reduce the tendency to work vigorously for food reinforcement. But given that these animals are still directed toward the acquisition and consumption of food, their behavior becomes redirected and they choose the lower-effort path to food procurement.

It also appears that these effects of impaired DA transmission are related to decision making based upon physical effort as opposed to other types of factors. Hosking et al. (2015) reported that DA antagonism impaired discounting based upon physical effort, but not cognitive effort. Although local injection of DA D2 antagonists into nucleus accumbens core has been shown to alter effort-related decision making (Nowend et al., 2001; Farrar et al., 2010), the same manipulation did not affect decision making based upon risk (Stopper et al., 2013). A recent study reported on the effects of the DA-depleting agent tetrabenazine on a T-maze choice task in which mice could choose between running in a running wheel versus approaching and consuming sucrose pellets. Tetrabenazine shifted choice behavior, decreasing selection of running wheel activity but actually increasing consumption of sucrose (Correa et al., 2016). This finding illustrates that under some circumstances interfering with DA transmission can actually increase the relative reinforcing value of sucrose, and also serves to underscore the role of physical effort as a factor in determining sensitivity to dopaminergic impairments.

Although this study is focused on the pharmacology of effort-based choice, it is important to consider the relation between the behavioral pharmacology findings reviewed above and what is known about the activity of DA neurons in the context of instrumental conditioning. Studies employing electrophysiology and neurochemistry methods demonstrate that DA neuron activity is not simply tied to the delivery of primary positive reinforcers across a broad range of conditions (Marinelli and McCutcheon, 2014; Salamone et al., 2016a). Rather, it appears that the phasic DA signaling responses in accumbens as measured in animal experiments are context dependent, they vary across fast and slow timescales, and what is being responded to depends upon the details of the behavioral procedures being used. A substantial literature demonstrates that fast phasic DA neuron responses represent a reward prediction error, which could be related to the expected utility of rewards (Stauffer et al., 2016). However, the specific relation between reward prediction error signals and exertion of effort remains uncertain. For example, knockout of ventral tegmental N-methyl-D-aspartate receptors, which dramatically reduces phasic DA signaling, was reported to have no effect on PROG responding (Zweifel et al., 2009). There also is evidence that prolonged DA signaling (i.e., ramps) in response to stimuli that are distant in time and space from the reinforcer is associated with a sustained motivational drive that maintains instrumental behavior during maze learning (Howe et al., 2013). Hamid et al. (2016) studied DA signaling as measured by fast cyclic voltammetry responses in rats that were behaving a flexible decision-making task, which had distinct phases that ultimately led to reinforcement. They observed that phasic DA responses increased in magnitude as animals progressively passed through the task phases and moved toward the increasing likelihood of reinforcement at the end. These DA signals were correlated with important aspects of behavioral output, such as latency to instigate the behavioral response. These DA-related signals increased as the animals progressed through the phases of the task even when reward was predicted, and thus did not represent a reward prediction error response. This led Hamid et al. (2016) to suggest that mesolimbic DA helps to translate estimates of reinforcer availability into decisions to work for reinforcers, and that mesolimbic DA release could be providing a motivational signal regulating behavioral activation and the decision of whether or not to engage in effortful activity while working for rewards. Overall, several studies have reported an association between instrumental response output and pre- or postsynaptic markers of DA-related signaling (Sokolowski et al., 1998; Segovia et al., 2011, 2012; Howe et al., 2013; Saddoris et al., 2015; Hamid et al., 2016; Ko and Wanat, 2016; Wood et al., 2017). Furthermore, selective chemogenetic activation of mesolimbic DA neurons was recently reported to increase responding for sucrose in rats responding on a PROG schedule (Boekhoudt et al., 2018). This effect was characterized by increased initiation of bouts of instrumental behavior and was not seen after activation of nigrostriatal DA neurons. Optogenetic inhibition of ventral tegmental area DA neurons suppressed the initiation and the maintenance of effortful operant responding on FR8 and PROG schedules (Fischbach-Weiss et al., 2018). Taken together, these studies provide additional evidence of the importance of DA neuron activity in the regulation of response vigor and work output.

V. Neural Circuits and Transmitters Mediating Effort-Based Choice: Adenosine, GABA, and Nondopaminergic Components of the Circuit

Despite the clear importance of DA in regulating effort-based choice, it is nevertheless evident that no single neurotransmitter or brain area participates in a complex behavioral process on its own. Thus, it is important to review the nondopaminergic components of the neural circuitry that has been implicated in effort-related decision making. Over the last several years, a number of laboratories have begun to characterize the role that various brain structures and neurotransmitters play in effort-related choice behavior. Multiple studies have shown that anterior cingulate/prefrontal cortex is an important part of the circuitry regulating effort-based choice. Large lesions of prefrontal cortex produce a low-effort bias in rats tested on the T-maze choice task (Walton et al., 2002), and subsequent work showed that these effects were largely attributable to lesions targeting anterior cingulate cortex (Walton et al., 2003). Schweimer and Hauber (2006) reported that depletions of anterior cingulate DA by injections of 6-hydroxydopamine also were able to shift T-maze barrier choice performance. Hart et al. (2017) found that excitotoxic lesions of anterior cingulate cortex decreased PROG lever pressing for sucrose pellets while leaving intake of concurrently available chow unaffected. The effects of anterior cingulate lesions in this study were not mediated by decreased appetite, a change in food preference, or a failure to update reinforcement value. Floresco and Ghods-Sharifi (2007) found that the basolateral amygdala and its connections with the anterior cingulate cortex form part of a serial circuitry regulating effort-related decision making.

Considerable evidence indicates that DA interacts with the neuromodulator adenosine in striatal areas, including neostriatum and nucleus accumbens, and that drugs that act on adenosine receptors can exert substantial behavioral effects. Caffeine and other methylxanthines such as theophylline and theobromine act as minor stimulants via their actions as nonselective antagonists of adenosine receptors (Ferré et al., 2008; Randall et al., 2011; Pardo et al., 2013; López-Cruz et al., 2014, 2016). The A₁ and A_2A subtypes are the major adenosine receptors in the brain, and nucleus accumbens and neostriatum have a very high expression of adenosine A_2A receptors (Schiffmann et al., 1991; DeMet and Chicz-DeMet, 2002; Ferré et al., 2004). Several studies have reported that there are cellular interactions of DA D2 and adenosine A_2A receptors that are colocalized on the same striatal and accumbens medium spiny neurons, which include the ability to form heteromeric complexes and convergence onto the same cAMP/protein kinase A signal transduction cascade (Schiffmann et al., 1991; Ferré, 1997; Svenningsson et al., 1999; Fuxe et al., 2003). Adenosine A_2A receptor antagonists have been investigated and developed as potential antiparkinsonian agents (Ferré et al., 1997, 2001; Morelli and Pinna, 2001; Correa et al., 2004; Pinna et al., 2005; Salamone et al., 2008a,b), and istradefylline is currently used in Japan for the treatment of Parkinson’s disease (Kondo et al., 2015). Furthermore, drugs that stimulate or inhibit adenosine A_2A receptors induce substantial effects on operant behavior and effort-related choice. Intra-accumbens injections of the adenosine A_2A agonist CGS 21680 attenuated lever pressing in rats responding on a tandem VI 60 second/FR10 schedule, but did not impair performance on a standard VI 60 second schedule (Mingote et al., 2008). Infusion of CGS 21680 directly into nucleus accumbens shifted effort-related choice, decreasing FR5 lever pressing and increasing chow intake (Font et al., 2008), whereas control infusions into overlying dorsomedial neostriatum were ineffective. In general, stimulation of adenosine A_2A receptors in nucleus accumbens appears to induce similar effects to those produced by accumbens DA depletions or antagonism.

Several different adenosine A_2A receptor antagonists, including MSX-3, MSX-4, Lu AA47070, and istradefylline, have been shown to reverse the low-effort bias induced by systemically administered DA D2 antagonists in rats tested on the FR5/chow-feeding choice task (Farrar et al., 2007; Salamone et al., 2009; Worden et al., 2009; Nunes et al., 2010; Collins et al., 2012; Santerre et al., 2012). Intra-accumbens injections of MSX-3 reversed the effects of intra-accumbens injections of the D2 antagonist eticlopride in rats responding on the FR5/chow concurrent choice task (Farrar et al., 2010). Antagonism of adenosine A_2A receptors also reverses the effects of D2 antagonism in rats (Mott et al., 2009) and mice (Pardo et al., 2012) tested on the T-maze barrier choice task, and adenosine A_2A receptor knockout mice are resistant to the effects of haloperidol on selection of the barrier arm of the T-maze (Pardo et al., 2012). These effects of adenosine A_2A receptor antagonists on effort-related choice are subtype specific, as several papers have reported that blockade of adenosine A₁ receptors does not attenuate the effects of either DA D1 or D2 antagonists (Mott et al., 2009; Salamone et al., 2009; Nunes et al., 2010; Pardo et al., 2012). Studies using signal transduction markers of DA D2 transmission, such as cFos and phosphorylated DARPP-32, indicate that adenosine A_2A receptor antagonists are so effective in reversing the effort-related effects of D2 antagonists because A_2A blockade directly attenuates the signal transduction effects of D2 antagonism in medium spiny neurons that colocalize both receptors (Farrar et al., 2010; Santerre et al., 2012).

Nucleus accumbens core medium spiny neurons that colocalize DA D2 and adenosine A_2A receptors use GABA as their neurotransmitter and project to lateral ventral pallidum (Farrar et al., 2008; Mingote et al., 2008). GABA release in lateral ventral pallidum is increased by blockade of accumbens core D2 receptors (Salamone et al., 2010), and also by stimulation of accumbens adenosine A_2A receptors (Mingote et al., 2008). Consistent with these findings, injections of the GABA_A receptor agonist muscimol into lateral ventral pallidum produced a low-effort bias in rats tested on the FR5/chow-feeding choice task (Farrar et al., 2008). Moreover, an experiment using disconnection methods demonstrated that unilateral injections of the adenosine A_2A agonist CGS21680 into nucleus accumbens on one side of the brain, combined with contralateral injections of muscimol into lateral ventral pallidum, produced a strong impairment in high-effort instrumental responding that was not seen after various control procedures (Mingote et al., 2008). These results indicate that nucleus accumbens and lateral ventral pallidum form part of a serial circuit-mediating effort-based choice. Interestingly, other disconnection experiments have been performed on other parts of the forebrain circuitry, and these studies indicate that there is a series of connections involving basolateral amygdala, anterior cingulate cortex, nucleus accumbens, and ventral pallidum, which together participate in the regulation of effort-related decision making (Fig. 2). Several studies have identified that nucleus accumbens core as being a critical striatal component of this circuitry (Sokolowski et al., 1998; Nowend et al., 2001; Font et al., 2008; Mingote et al., 2008; Hauber and Sommer, 2009; Farrar et al., 2010; Ghods-Sharifi and Floresco, 2010; Randall et al., 2012; Nunes et al., 2013b), although there is some evidence that nucleus accumbens shell also participates (Nowend et al., 2001). To date, evidence is mixed on the role of DA in medial neostriatum (Cousins et al., 1993; Farrar et al., 2010), although this area appears to be important for DA/5-HT interactions regulating response vigor and exertion of effort (Bailey et al., 2018).

• Download figure
• Open in new tab
• Download powerpoint

Fig. 2.

Anatomic circuit diagrams depicting some of the brain structures and neurotransmitters involved in studies of effort-related decision making in rodents. Left: Schematic of the rat brain showing mesostriatal DA systems. Mesolimbic DA projection is shown with the solid arrow, whereas nigrostriatal DA projections are illustrated with dotted line. Right: This figure is a circuit diagram illustrating the limbic, cortical, striatal, and pallidal connections that are involved in effort-based choice. Furthermore, this figure illustrates the results from a group of studies employing disconnection methods. With these procedures, researchers produce combined contralateral damage to interconnected brain structures to determine whether they form part of a serial circuit that participates in a particular behavioral function (Floresco and Ghods-Sharifi, 2007: basolateral amygdala and anterior cingulate cortex; Hauber and Sommer, 2009: anterior cingulate cortex and nucleus accumbens core; Mingote et al., 2008: nucleus accumbens core and lateral ventral pallidum).

Several other manipulations that affect brain function have been identified as producing effects on effort-based choice. Injections of the muscarinic acetylcholine agonist pilocarpine directly into nucleus accumbens shifted effort-related choice in rats tested on the FR5/chow-feeding choice task, and these actions were reversed by coadministration of scopolamine (Nunes et al., 2013a). Genetic deletion of the neuronal calcium sensor-1, which reduces accumbens DA transmission, made mice less willing to work for food, but left food preference, responding for conditioned reinforcement, and the ability to represent changes in reward value intact (Ng et al., 2016). Inactivation of the lateral hypothalamus produced a low-effort bias as marked by a reduction in barrier climbing in the T-maze (Karimi et al., 2017). Glycine acts as both an inhibitory neurotransmitter and a co-agonist at the N-methyl-D-aspartate receptor, and recent studies indicate that the glycine uptake inhibitor bitopertin can reverse the effort-related effects of haloperidol in rats tested on the FR5/chow-feeding and T-maze barrier choice tasks (Yohn et al., 2017a). The nutritional supplement curcumin, when ingested orally or injected directly into the ventricles of the brain, was able to reverse the effects of tetrabenazine in rats tested on the concurrent FR5/chow-feeding choice task (Yohn et al., 2017b).

VI. Clinical Implications: Targeting Drugs for the Potential Treatment of Effort-Based Dysfunctions

As described above, behavioral activation and effort-related processes are critically important aspects of normal motivation. In addition, the clinical literature demonstrates that deficits in these processes can be a disabling feature of human psychopathology. Terms such as fatigue, apathy, anergia, psychomotor retardation, amotivation, and negative symptoms are employed to describe psychiatric symptoms characterized by a lack of behavioral activation that can be seen across several different disorders, including depression and schizophrenia (Salamone et al., 2006, 2015; Treadway and Zald, 2011; Barch et al., 2014; Dantzer et al., 2014; Whitton et al., 2015; Le Heron et al., 2017; De Doncker et al., 2018). These motivational symptoms are highly debilitating in depressed people (Tylee et al., 1999), and factor analysis studies showed that there is an anergia/fatigue factor that is correlated with the overall severity of depression (Gullion and Rush, 1998). Patients with Parkinson’s disease also show motivational dysfunctions such as fatigue and apathy (Friedman et al., 2007). Although 5-HT selective uptake inhibitors (SSRIs) are the most commonly used antidepressants, motivational symptoms related to behavioral activation and exertion of effort are relatively resistant to treatment with SSRIs (Stahl, 2002; Cooper et al., 2014; Fava et al., 2014; Rothschild et al., 2014). Research on effort-related motivational functions and dysfunctions involving human participants and animal models is expanding our understanding of the neurochemical basis of motivational symptoms and could ultimately lead to the development of drug treatments for motivational symptoms. Such an approach is validated by human studies showing that alterations in effort-based decision making are associated with depression (Treadway et al., 2012; Yang et al., 2014, 2016; Culbreth et al., 2018), schizophrenia (Gold et al., 2013; Culbreth et al., 2018), and Parkinson’s disease (Chong et al., 2015).

The recursive interaction between human and animal studies has led to the development of formal animal models of motivational pathology that employ tasks assessing effort-based decision making (Salamone et al., 2006, 2015, 2016a,b,c; Simpson et al., 2011, 2012; Markou et al., 2013; Bailey et al., 2016; Der-Avakian et al., 2016). This research has involved an assessment of the effort-related effects in rodents of conditions associated with depression, anergia, fatigue, and apathy in humans, as well as the evaluation of potential drug treatments. One such line of inquiry has involved studies related to stress, because it is such an important factor in psychopathology. Restraint stress has been shown to induce a low-effort bias as measured with an effort-discounting task in rats (Shafiei et al., 2012), and the effort-related effects of stress involve the actions of corticotropin-releasing hormone (Bryce and Floresco, 2016).

As described above, several recent papers have used the vesicular monoamine transporter type-2 inhibitor tetrabenazine to alter effort-related choice in rodents. Tetrabenazine is used clinically to treat Huntington’s disease, but it induces side effects in humans that include depression and fatigue (Frank, 2009, 2010, 2014; Guay, 2010). Moreover, tetrabenazine has been used to produce deficits in classic animal models of depression such as the forced swim test (Tadano et al., 2000; Wang et al., 2010). Considerable evidence indicates that tetrabenazine can induce a low-effort bias in rats tested on the FR5/chow-feeding choice (Nunes et al., 2013b; Yohn et al., 2016b, d,e), PROG/chow-feeding choice (Randall et al., 2015a), and T-maze barrier choice tests (Yohn et al., 2015a,b). A series of parallel control experiments has demonstrated that the effort-related effects of tetrabenazine were not due to actions such as loss of appetite, changes in preference for chow versus pellets or preference across different concentrations of sucrose, discrimination of reinforcement magnitude, or reference memory (Nunes et al., 2013; Randall et al., 2015a; Yohn et al., 2015b). In view of these findings, the tetrabenazine model appears to have potential utility for exploring possible drug strategies for human clinical use in treating effort-related motivational dysfunctions.

Because monoamine uptake inhibitors are commonly used to treat depression, albeit with limited success in terms of restoring motivational function, several recent studies have focused on the ability of monoamine uptake inhibitors with different mechanisms of action to reverse the effort-related effects of tetrabenazine. The SSRIs fluoxetine and citalopram were investigated, and neither drug was able to reverse the effects of tetrabenazine on FR5/chow-feeding choice performance (Yohn et al., 2016b,e). The NE uptake blocker desipramine also was assessed, and it also failed to reverse the effects of tetrabenazine (Yohn et al., 2016b). In contrast, several DA transport blockers have been tested, and these drugs are generally effective at attenuating the effort-related actions of tetrabenazine. The catecholamine uptake blocker bupropion (Welbutrin) is widely used as an antidepressant, and clinical studies have reported it to be more effective than SSRIs at treating fatigue symptoms (Papakostas et al., 2006; Cooper et al., 2014). Bupropion reversed the effort-based effects of tetrabenazine in rats tested on the FR5/chow-feeding choice (Nunes et al., 2013b; Yohn et al., 2016a), PROG/chow-feeding choice (Randall et al., 2015a), and T-maze barrier choice tasks (Yohn et al., 2015b). A number of additional drugs that inhibit DA uptake have been assessed [GBR12909, PRX-14040, lisdexamfetamine (Vyvanse), methylphenidate, modafinil], and all have been reported to reverse the effort-related effects of tetrabenazine (Salamone et al., 2016c; Yohn et al., 2016b,d,e). In this context, it is interesting to note that amphetamine, methylphenidate, and modafinil have been reported to have positive effects on motivational symptoms in patients with depression (Stotz et al., 1999; Lam et al., 2007; Ravindran et al., 2008). Furthermore, modafinil is an atypical DA transporter blocker, which has different binding characteristics than amphetamine and cocaine. The assessment of novel atypical DA transporter blockers may lead to the discovery of compounds that are useful for treating motivational dysfunction, but have relatively low abuse liability.

Another approach to studying the effort-related effects of monoamine uptake inhibitors has been to assess the ability of drugs to enhance selection of high-effort choices in otherwise untreated animals. The concurrent PROG/chow-feeding choice task has been very useful in this regard. Due to the fact that the PROG lever-pressing requirement increases as the animal continues to respond, this schedule challenges the animal with a progressively incrementing work requirement. Because the concurrently available chow essentially acts as a low-cost substitute, rats typically respond to this challenge by initially lever pressing, but then switching to the concurrently available chow during the session. This pattern of responding generates a relatively low baseline rate of PROG lever pressing that can be sensitive to drugs that increase behavioral activation. Several DA transport blockers can increase selection of high-effort PROG lever pressing when administered on their own, including MRZ-9547 (Sommer et al., 2014), bupropion (Randall et al., 2015b; Salamone et al., 2016a), lisdexamfetamine (Yohn et al., 2016e), PRX-14040 (Yohn et al., 2016d), and GBR12909 (Yohn et al., 2016c). These results are consistent with studies reporting that there is enhanced selection of high-effort instrumental actions in mice with knockdown of DA transporters (Cagniard et al., 2006), and in mice that have increased expression of DA D2 receptors in nucleus accumbens induced during adulthood (Trifilieff et al., 2013). However, the SSRI fluoxetine and the NE transport inhibitors desipramine and atomoxetine did not increase PROG lever pressing across a broad range of doses (Yohn et al., 2016c), and atomoxetine also did not affect performance on a ratio-discounting task (Hosking et al., 2015). Taken together with the results from the DA antagonism and DA depletion studies discussed above, these findings on the effects of DA transport inhibitors, either alone or in combination with tetrabenazine, demonstrate that there is an important dopaminergic component to the neural regulation of effort-based choice. In summary, there is strong evidence of a bidirectional pharmacological modulation of effort-related choice induced by drugs acting on DA transmission.

As reviewed above, DA is not the only neural signaling molecule involved in the regulation of effort-based choice, and several papers have reported that adenosine A_2A antagonism and genetic deletion can reverse the effort-related effects of DA D2 receptor antagonists. Consistent with these observations, the adenosine A_2A receptor antagonist MSX-3 has been shown to reverse the effort-related effects of tetrabenazine in rodents tested on multiple task tests of effort-related choice (Nunes et al., 2013b; Randall et al., 2015a; Yohn et al., 2015b). Recent studies in our laboratory have focused on the effort-related effects of the adenosine A_2A receptor antagonist preladenant (Fig. 3). Preladenant is a highly selective antagonist of adenosine A_2A receptors (Alnouri et al., 2015), with a binding affinity of less than 1 nM, and greater than 1000-fold selectivity for A_2A versus A₁ receptors in the rat. Consistent with its high binding affinity for adenosine A_2A receptors, preladenant was very potent at reversing the alteration in effort-based choice induced by tetrabenazine (Fig. 3, A and B; 0.05–0.2 mg/kg intraperitoneal; compared with 1.0–2.0 mg/kg MSX-3). MSX-3 has previously been shown to increase selection of PROG lever pressing in rats tested on the PROG/chow-feeding choice task, and preladenant was much more potent at producing this effect (Fig. 3, C and D; 0.4 mg/kg preladenant; compared with 2.0 mg/kg MSX-3). The results of studies with MSX-3, MSX-4, istradefylline, and preladenant suggest that adenosine A_2A antagonists have some potential for clinical use in the treatment of motivational impairments. Consistent with this idea, recent studies indicate that istradefylline can reduce fatigue symptoms in Parkinson’s disease patients (Abe et al., 2016).

• Download figure
• Open in new tab
• Download powerpoint

Fig. 3.

Effects of the adenosine A_2A antagonist preladenant (PL) on effort-related choice behavior. (A and B) Ability of the adenosine A_2A antagonist preladenant to reverse the effects of tetrabenazine (TBZ) in rats responding on the concurrent FR5/chow choice task. All rats (n = 13; adult male, Sprague–Dawley rats; Harlan Sprague–Dawley, Indianapolis, IN) were trained on the FR5/chow-feeding choice procedure, as described in Yohn et al. (2016b,d,e), and tested in 30-minute sessions. Rats were tested 5 days per week, and, after several weeks of training, drug testing was conducted 1 day each week, with a randomized order of drug treatments. All animals received intraperitoneal (IP) injections of vehicle or 0.75 mg/kg tetrabenazine 120 minutes prior to testing, and also received IP injections of vehicle or preladenant IP (0.05, 0.1, 0.2 mg/kg) 25 minutes prior to testing. (A) Lever pressing. Mean (± S.E.M.) number of lever presses in the 30-minute session are shown. There was an overall significant effect of drug treatment on lever pressing [F(4,48) = 12.0, P < 0.001]. Planned comparisons showed that TBZ significantly decreased lever pressing compared with vehicle (^#P < 0.05), and that all doses of preladenant plus TBZ significantly increased lever pressing relative to TBZ plus vehicle (*P < 0.01). (B) Chow intake. Mean (± S.E.M.) gram quantities of chow intake are shown. There was an overall significant effect of drug treatment on chow intake [F(4,48) = 7.43, P < 0.001]. Planned comparisons showed that TBZ significantly increased chow consumption relative to vehicle (^#P < 0.05), and that all doses of preladenant plus TBZ significantly decreased chow intake relative to TBZ plus vehicle (*P < 0.01). (C and D). Effect of preladenant in rats responding on the PROG/chow-feeding choice task. The rats that were used for the FR5/chow-feeding experiment (n = 13) were then trained on the concurrent PROG/chow-feeding choice task, as described by Yohn et al. (2016a). Rats were tested 5 days per week, and, after several weeks of training, drug testing was conducted 1 day each week, with a randomized order of drug treatments. All animals received IP injections of vehicle or preladenant IP (0.1, 0.2, 0.4 mg/kg) 25 minutes prior to testing. (C) Lever pressing. Mean ± S.E.M. number of lever presses in the 30-minute session is shown. There was an overall significant effect of drug treatment on PROG lever pressing [F(3,36) = 3.7, P < 0.05]. Planned comparisons showed that the highest dose of preladenant (0.4 mg/kg) significantly increased lever pressing relative to vehicle (*P < 0.05). (D) Chow intake. Mean (± S.E.M.) gram quantities of chow intake are shown. The overall tendency of preladenant administration to decrease chow intake approached statistical significance [F(3,36) = 2.63, P = 0.065]. [All results from this figure were presented at the 2017 Society for Neuroscience meeting (Rotolo et al., 2017).]

In addition to stress or administration of tetrabenazine as ways of producing effort-related impairments in animal models, recent studies have focused upon the role of neuroinflammation in motivational symptoms. Proinflammatory cytokines and related pathways have been implicated in depression, Parkinson’s disease, schizophrenia, and other disorders, and there has been a particular focus on the association between cytokines and motivational symptoms such as anergia and fatigue (Capuron et al., 2007, 2009; Dantzer et al., 2008, 2012, 2014; Dantzer, 2009; Miller, 2009; Cattaneo et al., 2013; Felger and Treadway, 2016). Human patients are sometimes given interferon (IFN)-α as a treatment, and fatigue and loss of energy are reported to be the most common symptom induced by IFN-α. Although depressed mood was reported by some patients (30%–60%), fatigue and loss of energy occurred in 80% of patients receiving treatment with IFN-α (Miller, 2009). Moreover, patients who received IFN-α treatment, when compared with medically healthy people with major depression, showed less agitation and suicidal ideation, but significantly greater psychomotor slowing (Capuron et al., 2009). Some experimental studies have involved administration of the endotoxin lipopolysaccharide (LPS), which induces cytokine expression and produces the sickness behavior that is characteristic of overexpression of proinflammatory cytokines. One recent study with human participants used the effort expenditure for rewards task to assess the effects of LPS and observed that LPS did not reduce the tendency to select the high-effort option (Lasselin et al., 2017). In contrast, Draper et al. (2018) reported that LPS reduced selection of high-effort choices (squeezing a dynamometer) but did not affect reward sensitivity. In mice, Vichaya et al. (2014) demonstrated that the inflammatory response induced by LPS injection suppressed the overall exertion of effort in terms of total number of responses but left intact the relative selection of the high-effort/high-reward option. Other studies in animals have involved direct administration of the proinflammatory cytokines interleukin (IL)-1β and IL-6. In rats tested on the FR5/chow-feeding choice task, a low-effort bias was induced by administration of IL-1β (Nunes et al., 2014; Yohn et al., 2016e) and IL-6 (Yohn et al., 2016a) at doses that did not alter food preference or induce fever. These effort-related effects of cytokine administration were reversed by the adenosine A_2A antagonist MSX-3, and also by the DA transport inhibitors lisdexamfetamine and methylphenidate (Nunes et al., 2014; Yohn et al., 2016a,e). The behaviorally most effective dose of IL-6 also decreased extracellular DA in nucleus accumbens as measured by microdialysis (Yohn et al., 2016a). These studies suggest that there is important cross-talk between central neurotransmission and peripheral inflammatory responses and also illustrate possible strategies related to treatment based upon the anti-inflammatory actions of therapeutic agents.