Dissociation of the role of nucleus accumbens dopamine in responding to reward-predictive cues and waiting for reward

Abstract

The choice among behavioral options is influenced by the anticipated cost of working for the reward relative to the anticipated reward magnitude. Dopamine release in the nucleus accumbens (NAc) has been suggested to play an important role in cost/benefit computation. From a behavioral perspective, work involves two elements: the caloric expenditure of energy and the time required to complete the task. In many studies of the contribution of NAc dopamine to cost/benefit decisions, measures of work have conflated these separate elements. Here we describe a novel cued progressive delay task, an analog of the progressive ratio task that minimizes energy expenditure. In this task, rats obtain sucrose reward by entering a reward receptacle in response to a cue and remaining in the receptacle for a required wait that is increased after each successful trial. In an experiment in which the magnitude of reward was varied across sessions, the animals’ fail point (maximum wait achieved) was correlated with the amount of reward delivered. Microinjection of D1 or D2 dopamine receptor antagonists into the NAc did not affect the length of time animals were willing to wait for reward (fail point), but did reduce the proportion of cues to which the animal responded. These results suggest that waiting for reward without increased caloric energy expenditure does not require NAc dopamine.

Introduction

Many behavioral studies implicate NAc dopamine release in the appetitive phase of goal-directed behavior: NAc dopamine is often required for animals to exert effort to obtain reward, but not to consume it [4], [5], [22], [23]. An elegant series of experiments by Salamone and coworkers suggests that the release of dopamine in the NAc facilitates sustained goal-directed motor behavior (reviewed in [34], [35], [37]). For instance, depletion of NAc dopamine with 6-hydroxy-dopamine (6-OHDA) or injection of dopamine antagonists into the NAc causes animals to choose a smaller or less preferred reward that requires less effort to obtain than a larger (and, in control animals, normally chosen) reward that requires more effort [11], [12], [24], [29], [36], [38], [40]. Similarly, NAc 6-OHDA lesions reduce operant responding on high fixed-ratio (FR) schedules (e.g., FR64) but not schedules with lower ratios, such as FR1 [1]. Furthermore, NAc dopamine depletion strongly impairs performance of a progressive ratio task, in which the number of lever presses required to obtain reward increases with each reward earned, by causing a reduction in the rate of lever pressing and hence the number of rewards earned by the animal in a fixed period of time [2], [18].

Several different hypotheses could explain these results. One idea is that animals that have sustained NAc dopamine depletion or dopamine receptor antagonism suffer from deficits in motivation or motor ability. This can be ruled out by consistent findings that consumption of freely available food is not affected, nor is the ability to earn reward when every operant response is followed by a reward, as in FR1 [34]. Another possibility is that release of dopamine in the NAc increases the rate of work emitted per unit of reward. Delivering larger reward (e.g., 6 pellets after 300 presses instead of 1 pellet after 300 presses) increases normal animals’ ability to complete very high ratio requirements. However, increasing the reward in high fixed-ratio tasks, so that the work:reward ratio is constant (e.g., delivering 6 pellets after 300 lever presses instead of 1 pellet after 50 presses), does not rescue performance deficits caused by NAc 6-OHDA lesions [39]. Indeed, the effects of NAc 6-OHDA lesions were larger for the 300 presses:6 pellets task than for 50 presses:1 pellet. Thus, the contribution of NAc dopamine is apparently greater for larger work requirements than smaller requirements, even if the work:reward ratio is constant. This is consistent with the idea that the computation that is affected by the lesion is that of the amount of work required, not the amount of reward to be obtained or the ratio between the two.

An alternate interpretation is that NAc dopamine facilitates sustained work through time, even if the caloric energy expenditure is not large. This possibility arises from the fact that work in operant tasks has two components: caloric expenditure and time between work onset and delivery of reward. These two components covary in ratio tasks; for example, performing 300 lever presses requires more energy than performing 50 lever presses and also takes more time. The longer the interval between work onset and reward, the more the value of the predicted reward is discounted [19], [25], and the more likely the animal will interrupt its effort to obtain the reward. NAc dopamine could influence the discounting function by attenuating the fall-off in value as the delay increases, thereby facilitating both sustained effort and willingness to wait for larger reward. This idea is supported by studies showing that systemic dopamine antagonists reduce the choice of larger delayed rewards in favor of smaller immediate rewards [8], [43], and the indirect dopamine agonist amphetamine increases willingness to wait for larger reward [8], [14], [32]. Furthermore, lesions of the NAc core cause rats to choose smaller immediate reward over larger delayed reward [7]. Taken together, these studies provide indirect evidence that the release of NAc dopamine flattens the delay discounting function.

One interpretation of the studies suggesting a role for dopamine and the NAc in delay discounting [7], [8], [32], [43] is that NAc dopamine may be required to facilitate the response to cues that predict a large reward available after a long delay (or a higher energy cost). These studies employed discrete-trials tasks in which animals had to choose between levers that resulted in different outcomes. For instance, in the studies by Cardinal and coworkers [7], [8], the choice was between a lever (the “immediate lever”) that, if selected, resulted in small immediate reward, and a “delay lever” that resulted in larger reward after a delay. (Both levers were retracted after a response was made.) If indeed NAc dopamine facilitates the choice of larger delayed reward, it may do so either (1) directly, by promoting the response on the delay lever, perhaps because the dopamine is released in response to the reward-predictive stimulus of the delay lever but not the immediate lever; or (2) indirectly, by allowing the animal to maintain the association between the delay lever and the large reward throughout the delay to reward, thereby increasing the likelihood that the delay lever will be chosen on subsequent trials. The difference between these two hypotheses is critical for understanding the role of NAc dopamine in goal-directed behavior that involves a delay to reward: either the dopamine has its effects throughout the delay to enhance the ability of the animal to wait (or work) for reward, or its effects are much more transient, involving only the choice of behavior in response to discrete cues. A third hypothesis, suggested by the work of Salamone and coworkers [10], [33], is that NAc dopamine release is required during the expenditure of large (but not small) amounts of caloric energy in order to facilitate the expenditure over a given period of time.

To test among these possibilities, we adapted the progressive ratio task to eliminate the requirement for emitting increasing caloric energy, while maintaining the requirement that animals wait through an increasing delay for each successive reward. This novel progressive delay task measures the amount of time an animal is willing to wait for reward. In the task, a cue is presented to the animal on a VI 30-s schedule. The animal must respond to the cue by entering its head into a reward receptacle and remaining until reward is delivered. The delay between receptacle entry and reward delivery is increased with each reward earned; reward is not delivered if the animal withdraws early from the receptacle (“fail point”). Fail point is, therefore, analogous to break point on the progressive ratio schedule. We tested how NAc dopamine contributes to the performance of this task by injecting D1 or D2 antagonists into the NAc prior to daily behavioral sessions. The inclusion of an explicit cue to signal the animal to initiate a trial allowed us to dissociate the role of NAc dopamine in the initiation of behavioral responses to reward-predictive cues [13] from its potential subsequent role in waiting for reward.