Behavioral and neurobiological mechanisms of extinction in Pavlovian and instrumental learning

doi:10.1016/j.nlm.2013.08.012

Neurobiology of Learning and Memory

Volume 108, February 2014, Pages 52-64

https://doi.org/10.1016/j.nlm.2013.08.012 Get rights and content

Highlights

•
Reviews behavioral and neural mechanisms of extinction in Pavlovian and instrumental learning.
•
Discusses principles of extinction generated by research at the behavioral level.
•
Pavlovian and instrumental extinction are controlled by compatible behavioral and neural processes.

Abstract

This article reviews research on the behavioral and neural mechanisms of extinction as it is represented in both Pavlovian and instrumental learning. In Pavlovian extinction, repeated presentation of a signal without its reinforcer weakens behavior evoked by the signal; in instrumental extinction, repeated occurrence of a voluntary action without its reinforcer weakens the strength of the action. In either case, contemporary research at both the behavioral and neural levels of analysis has been guided by a set of extinction principles that were first generated by research conducted at the behavioral level. The review discusses these principles and illustrates how they have informed the study of both Pavlovian and instrumental extinction. It shows that behavioral and neurobiological research efforts have been tightly linked and that their results are readily integrated. Pavlovian and instrumental extinction are also controlled by compatible behavioral and neural processes. Since many behavioral effects observed in extinction can be multiply determined, we suggest that the current close connection between behavioral-level and neural-level analyses will need to continue.

Introduction

Behavioral research over the last few decades has made significant progress in uncovering the mechanisms that underlie extinction, the behavioral phenomenon in which learned behavior decreases in strength or frequency when the event that reinforced it is removed (e.g., Delamater & Westbrook, 2014). Extinction is important, in part because it is one of the most basic of all behavioral change effects, and in part because it is thought to be involved in many clinical treatments that are designed to get rid of unwanted learned behaviors, thoughts, and emotions (e.g., Craske et al., 2008, Craske et al., 2012). Since the 1970s, extinction has been extensively studied in Pavlovian conditioning, where responding to a conditioned stimulus (CS) that has been associated with an unconditioned stimulus (US) decreases when the CS is then presented repeatedly alone. The results of this literature, along with the larger behavioral research literature of which it is a part, support a set of principles of extinction that are summarized in Table 1. These principles have gone onto shape further research on extinction at both the behavioral and neurobiological levels of analysis. The purpose of the present article is to discuss these principles and explore how they have facilitated progress in behavioral and neurobiological research on Pavlovian extinction and a “newer” frontier concerning the extinction of instrumental learning, where the focus is on voluntary behaviors that are controlled by their consequences (reinforcers). We also suggest that continued success at elucidating the neurobiological mechanisms of Pavlovian and instrumental extinction will require continued research at the behavioral level.

Section snippets

Summary extinction principles

The first extinction principle supported by behavioral research is perhaps the most widely recognized today: Although behavior goes away in extinction, Extinction is not the same as erasure. Learning theorists have long recognized that there is a difference between behavior on the one hand and the organism’s knowledge on the other. Extinction is a good example of the so-called “learning-performance distinction:” Although performance is at a zero level at the end of extinction, the original

Neurobiological mechanisms of Pavlovian extinction

Advances in our understanding of extinction from a behavioral point of view contributed directly to a new interest in the underlying neurobiological mechanisms that began to crystallize and mature in the 2000s (e.g., Delamater, 2004, Myers and Davis, 2002). In general, the neuroscience literature can be divided into several areas that emphasize one of the major principles just described. Perhaps the largest area is one investigating the neural circuits and synaptic changes that support new

Behavioral mechanisms of instrumental extinction

Other advances are being made in understanding the behavioral and neurobiological mechanisms of instrumental extinction. In instrumental (or operant) learning, organisms learn to perform certain behaviors (e.g., pressing a lever or pulling a chain) when a reinforcing event like a food pellet is presented as a consequence of it. In extinction, the behavior declines when the reinforcer is no longer presented. Instrumental extinction is as important to understand as Pavlovian extinction if we want

Neurobiological mechanisms of instrumental extinction

Instrumental learning and extinction procedures have recently become important tools for investigating the neurobiology of drug-taking and relapse (e.g., Marchant, Li, & Shaham, 2013). The approach has been stimulated and enriched by behavioral work. For example, many studies have examined the contextual control of extinguished operant behavior reinforced by drugs of abuse (see Bouton, Winterbauer, & Vurbic, 2012; Millan, Marchant, & McNally, 2011 for reviews). The area has the potential to

Conclusion

Our review has illustrated a tight coupling between behavior theory and research on the neural mechanisms of extinction. Indeed, it can be said that the study of neurobiological mechanisms of extinction has stood on the shoulders of research that has investigated its behavioral underpinnings. However, we will close by noting that, in all likelihood, the linkage between these levels of analysis will need to be ongoing. As our review indicates, extinction and the many behavioral effects that

Acknowledgment

Preparation of the manuscript was supported by National of Institute on Drug Abuse Grant RO1 DA033123 to M.E.B.

References (202)

A. Amsel
Partial reinforcement effects on vigor and persistence
F. Botreau et al.
d-Cycloserine facilitates extinction of a cocaine-induced conditioned place preference
Behavioral Brain Research
(2006)
M.E. Bouton
Context and ambiguity in the extinction of emotional learning: Implications for exposure therapy
Behaviour Research and Therapy
(1988)
M.E. Bouton
Context, ambiguity, and unlearning: Sources of relapse after behavioral extinction
Biological Psychiatry
(2002)
M.E. Bouton et al.
Contextual control of the extinction of conditioned fear
Learning and Motivation
(1979)
M.E. Bouton et al.
Sources of relapse after extinction in Pavlovian and instrumental learning
Clinical Psychology Review
(1991)
M.E. Bouton et al.
d-Cycloserine facilitates context specific fear extinction learning
Neurobiology of Learning and Memory
(2008)
M.E. Bouton et al.
Occasional reinforced trials during extinction can slow the rate of rapid reacquisition
Learning and Motivation
(2004)
A. Burgos-Robles et al.
Consolidation of fear extinction requires NMDA receptor-dependent bursting in the ventromedial prefrontal cortex
Neuron
(2007)
V. Campese et al.
ABA and ABC renewal of conditioned magazine approach are not impaired by dorsal hippocampus inactivation or lesions
Behavioral Brain Research
(2013)

E.J. Capaldi

A sequential hypothesis of instrumental learning

Psychology of Learning and Motivation

(1967)

M.G. Craske et al.

Optimizing inhibitory learning during exposure therapy

Behaviour Research and Therapy

(2008)

A.R. Delamater

Issues in the extinction of specific stimulus–outcome associations in Pavlovian conditioning

Behavioural Processes

(2012)

S.B. Flagel et al.

Individual differences in the attribution of incentive salience to reward-related cues: Implications for addiction

Neuropsychopharmacology

(2009)

P.W. Frey et al.

Extinction after aversive conditioning: An associative or nonassociative process?

Learning and Motivation

(1977)

A.S. Hamlin et al.

Renewal of extinguished cocaine-seeking

Neuroscience

(2008)

A.S. Hamlin et al.

The neural correlates and role of D1 dopamine receptors in renewal of extinguished alcohol-seeking

Neuroscience

(2007)

P.C. Holland

Occasion setting in Pavlovian conditioning

P.C. Holland et al.

Transfer of inhibitory stimulus control in operant feature-negative discrimination

Learning and Motivation

(1993)

D. Ishii et al.

No erasure effect of retrieval–extinction trial on fear memory in the hippocampus-independent and dependent paradigms

Neuroscience Letters

(2012)

D.N. Kearns et al.

Deepened extinction of cocaine cues

Drug and Alcohol Dependence

(2012)

M. Kindt et al.

Reconsolidation in a human fear conditioning study: A test of extinction as updating mechanism

Biological Psychology

(2013)

J.C. Leslie et al.

Facilitation of extinction and re-extinction of operant behavior in mice by chlordiazepoxide and d-cycloserine

Neurobiology of Learning and Memory

(2013)

B.T. Litz et al.

A randomized placebo-controlled trial of d-cycloserine and exposure therapy for posttraumatic stress disorder

Journal of Psychiatric Research

(2012)

N.J. Marchant et al.

Recent developments in animal models of drug relapse

Current Opinion in Neurobiology

(2013)

A. Auber et al.

Post-retrieval extinction as reconsolidation interference: Methodological issues or boundary conditions?

Psychopharmacology

(2013)

A.G. Baker

Contextual conditioning during free-operant extinction: Unsignaled, signaled, and backward-signaled noncontingent food

Animal Learning & Behavior

(1990)

A.G. Baker et al.

Contextual conditioning and reinstatement of extinguished instrumental responding

The Quarterly Journal of Experimental Psychology

(1991)

B.W. Balleine et al.

Still at the choice point: Action selection and initiation in instrumental conditioning

Annals of the New York Academy of Sciences

(2007)

M.L. Banks et al.

Effects of ambient temperature on the relative reinforcing strength of MDMA using a choice procedure in monkeys

Psychopharmacology

(2008)

J.M. Bossert et al.

A role of ventral tegmental area glutamate in contextual cue-induced relapse to heroin seeking

The Journal of Neuroscience

(2004)

J.M. Bossert et al.

Ventral medial prefrontal cortex neuronal ensembles mediate context-induced relapse to heroin

Nature Neuroscience

(2011)

Bossert, J. M., Marchant, N. J., Calu, D. J., & Shaham, Y. (2013). The reinstatement model of drug relapse: recent...

L.C. Botly et al.

Characterization of methylphenidate self-administration and reinstatement in the rat

Psychopharmacology

(2008)

M.E. Bouton

Differential control by context in the inflation and reinstatement paradigms

Journal of Experimental Psychology: Animal Behavior Processes

(1984)

M.E. Bouton

Context, time, and memory retrieval in the interference paradigm of Pavlovian learning

Psychological Bulletin

(1993)

M.E. Bouton

Signals for whether versus when an event will occur

M.E. Bouton

A learning theory perspective on lapse and relapse and the maintenance of behavior change

Health Psychology

(2000)

M.E. Bouton

Context and behavioral processes in extinction

Learning & Memory

(2004)

M.E. Bouton et al.

Role of conditioned contextual stimuli in reinstatement of extinguished fear

Journal of Experimental Psychology: Animal Behavioral Processes

(1979)

M.E. Bouton et al.

Intertrial interval as a contextual stimulus: Further analysis of a novel asymmetry in temporal discrimination learning

Journal of Experimental Psychology: Animal Behavior Processes

(2011)

M.E. Bouton et al.

State-dependent fear extinction with two benzodiazepine tranquilizers

Behavioral Neuroscience

(1990)

M.E. Bouton et al.

Contextual control of the extinction of conditioned fear: Tests for the associative value of the context

Journal of Experimental Psychology: Animal Behavior Processes

(1983)

M.E. Bouton et al.

Context effects on conditioning, extinction, and reinstatement in an appetitive conditioning preparation

Animal Learning & Behavior

(1989)

M.E. Bouton et al.

Renewal of extinguished responding in a second context

Animal Learning & Behavior

(1994)

M.E. Bouton et al.

Effects of contextual conditioning and unconditional stimulus presentation on performance in appetitive conditioning

The Quarterly Journal of Experimental Psychology

(1993)

Bouton, M. E., Todd, T. P., & León, S. P. (2013). Contextual control of a discriminated operant behavior. Journal of...

M.E. Bouton et al.

Renewal after the extinction of free operant behavior

Learning & Behavior

(2011)

M.E. Bouton et al.

Relapse processes after the extinction of instrumental behavior: Renewal, resurgence, and reacquisition

Behavioral Processes

(2013)

M.E. Bouton et al.

Context and extinction: Mechanisms of relapse in drug self-administration

Cited by (104)

Role of nucleus accumbens D1-type medium spiny neurons in the expression and extinction of sign-tracking
2024, Behavioural Brain Research
While sign-tracking, also known as autoshaping, has been studied for many decades, only recently has the tendency to show sign-tracking behavior been linked to the development and persistence of addiction. Sign-tracking is dependent upon dopamine activity in the nucleus accumbens (NAc). The NAc is comprised predominantly of medium spiny projection neurons (MSN) that can be differentiated by their D1-like or D2-like dopamine receptor expression. Here we determined how reducing activity of D1-type MSNs in the NAc affects the expression and extinction of sign-tracking. To address this, we transfected the NAc of transgenic male and female rats that selectively express Cre recombinase in D1-type MSNs with a DIO viral vector expressing hM4Di. Cre- rats were given the same viral infusion but did not express the hM4Di receptor and therefore served as controls. Rats were then conditioned to associate lever presentations with pellet delivery. After sign-tracking was established, all rats were administered clozapine-n-oxide (CNO) prior to three additional conditioning sessions to assess the effects of NAc D1-MSNs inhibition on sign-tracking in the presence of reward. CNO treatment did not alter the expression of sign-tracking in Cre+ or Cre- rats. Next rats underwent extinction training where lever presentations occurred without pellet delivery and all rats received a CNO injection prior to each extinction session. In these extinction conditions, Cre+ rats exhibited robust extinction of sign-tracking across sessions, whereas Cre- rats did not. To determine if D1-MSN inhibition merely produced a temporary cessation of sign-tracking or instead had facilitated a persistent loss of sign-tracking, we evaluated the reemergence of sign-tracking in a test for reconditioning. During testing, reintroduction of the CS-US pairing did not promote the reemergence of sign-tracking in Cre+ rats, but restored sign-tracking in Cre- rats. Thus, chemogenetic inhibition of NAc D1-MSNs promoted extinction of sign-tracking. Collectively, these data suggest that D1-MSNs play an important role in resistance to extinction that typifies sign-tracking behavior.
Predictions transform memories: How expected versus unexpected events are integrated or separated in memory
2023, Neuroscience and Biobehavioral Reviews
Our brains constantly generate predictions about the environment based on prior knowledge. Many of the events we experience are consistent with these predictions, while others might be inconsistent with prior knowledge and thus violate our predictions. To guide future behavior, the memory system must be able to strengthen, transform, or add to existing knowledge based on the accuracy of our predictions. We synthesize recent evidence suggesting that when an event is consistent with our predictions, it leads to neural integration between related memories, which is associated with enhanced associative memory, as well as memory biases. Prediction errors, in turn, can promote both neural integration and separation, and lead to multiple mnemonic outcomes. We review these findings and how they interact with factors such as memory reactivation, prediction error strength, and task goals, to offer insight into what determines memory for events that violate our predictions. In doing so, this review brings together recent neural and behavioral research to advance our understanding of how predictions shape memory, and why.
Behavioral and Neural Mechanisms of Latent Extinction: A Historical Review
2022, Neuroscience
The present paper provides a comprehensive review of latent extinction. In maze learning situations, latent extinction involves confining an animal to a previously reinforced goal location without food. When returned to the starting position after latent extinction, the animal typically shows a response decrement. Such findings have suggested that latent extinction is sufficient to invoke extinction learning, despite the animal having been prevented from making the original response. The majority of research on latent extinction was conducted between 19491980 and focused on what is being learned during the latent placements. Stimulus–response (S–R) theorists attempted to explain latent extinction via novel S–R mechanisms, namely, the fractional anticipatory response (rG). However, research did not support the role of rG in latent extinction. Cognitive expectancy theorists provided a simpler, more adequate explanation for latent extinction, more consistent with the data. Specifically, latent extinction might instill a change in expectation (i.e., animals learn to expect absence of reinforcement). Evidence also suggests that latent extinction involves place learning mechanisms and is sensitive to modulation via certain experimental factors. More recent work has uncovered some of the neural mechanisms of latent extinction. The hippocampus is critically involved in latent extinction, whereas other brain regions typically implicated in regular “response extinction” in the maze, such as the dorsolateral striatum, are not required for latent extinction. Similar to other kinds of learning, latent extinction requires NMDA receptor activity, suggesting the involvement of synaptic plasticity. Consistent with a multiple memory systems perspective, research on latent extinction supports the hypothesis that extinction learning is not a unitary process but rather there are different kinds of extinction learning mediated by distinct neural systems.
Situating desire: Situational cues affect desire for food through eating simulations
2022, Appetite
How do situations influence food desire? Although eating typically occurs in rich background situations, research on food desire often focuses on the properties of foods and consumers, rather than on the situations in which eating takes place. Here, we take a grounded cognition perspective and suggest that a situation that is congruent with consuming a food increases simulations of eating it, which, in turn, affect desire, and the expected and actual liking of the food. We tested this idea in four pre-registered experiments (N = 524). Participants processed an image of a food presented in a congruent situation, an incongruent situation, or no background situation. Compared to the incongruent situation, the congruent situation increased expected liking of the food and desire, and this was partially or fully mediated by eating simulations. The congruent situation also increased salivation, a physiological indicator of preparing to eat. However, there was only weak and indirect evidence for congruence effects on actual liking of the food when tasted. These findings show that situational cues can affect desire for food through eating simulations. Thus, background situations play an important but understudied role in human food desires. We address implications for research using food images, and for applications to promote healthy and sustainable eating behaviour.
The DA-antagonist Tiapride affects context-related extinction learning in a predictive learning task, but not initial forming of associations, or renewal
2021, Neurobiology of Learning and Memory
Renewal describes the recovery of an extinguished response if the contexts of extinction and recall differ, highlighting the context dependency of extinction. Studies demonstrated dopaminergic (DA) signalling to be important for context-related extinction learning with and without a fear component. In a previous study in humans, administration of the dopamine D2/D3 antagonist tiapride prior to extinction impaired extinction learning in a novel, but not a familiar context, without affecting renewal. In a further study, context processing during initial acquisition of associations was shown to be related to renewal. In this human fMRI study we investigated the potential role of DA signalling during this initial conditioning for the learning process and for renewal. While tiapride, administered prior to the start of learning, did not affect initial acquisition and renewal, extinction learning in a novel context was impaired, associated with reduced BOLD activation in vmPFC, left iFG and ACC – regions mediating response inhibition and selection from competing options using contextual information. Thus, different timepoints of administration of tiapride (before initial conditioning or extinction) had largely similar effects upon extinction and renewal. In addition, retrieval of previously acquired associations was impaired, pointing towards weaker association forming during acquisition.
Conceivably, effects of the DA blockade are associated with the challenge present in the respective task rather than the administration timepoint: the cognitive flexibility required for forming a new inhibitory association that includes a novel element clearly requires DA processing, while initial forming of associations, or of inhibitory associations without a new element, apparently rely less on the proper function of the DA system.
The Empirical Status of Cue Exposure and Response Prevention Treatment for Binge Eating: A Systematic Review
2021, Behavior Therapy
Approximately 50% of individuals fail to obtain treatment benefits when undergoing cognitive-behavioral therapy (CBT) for binge-eating behaviors, making it necessary to evaluate additional approaches. Cue exposure and response prevention (CERP) is one such approach, although its effectiveness across studies has been inconsistent. This may be due to inconsistent implementation of theoretically based CERP strategies. This possibility has not yet been systematically investigated. To address this gap, this review investigated which CERP strategies have been incorporated into treatment protocols for binge eating, and if the use of certain strategies improves treatment effectiveness. Relevant studies were identified through reference lists, grey literature, and searches of electronic databases using multiple search terms related to CERP and binge eating, which resulted in 18 eligible studies. Most studies were underpowered, many were of low methodological quality, and none of the included studies utilized all of the strategies that have been recommended to optimize CERP. Despite these weaknesses, CERP appeared to reduce the frequency of binge eating in the short and long term. This review underscores the need for higher quality research that utilizes larger samples and uniform outcome measures that are more strongly grounded in theory. Such research would help improve treatment outcomes for binge eating.

View all citing articles on Scopus

View full text

ReviewBehavioral and neurobiological mechanisms of extinction in Pavlovian and instrumental learning

Highlights

Abstract

Introduction

Section snippets

Summary extinction principles

Neurobiological mechanisms of Pavlovian extinction

Behavioral mechanisms of instrumental extinction

Neurobiological mechanisms of instrumental extinction

Conclusion

Acknowledgment

Behavioral Brain Research

Behaviour Research and Therapy

Biological Psychiatry

Learning and Motivation

Clinical Psychology Review

Neurobiology of Learning and Memory

Learning and Motivation

Neuron

Behavioral Brain Research

Psychology of Learning and Motivation

Behaviour Research and Therapy

Behavioural Processes

Neuropsychopharmacology

Learning and Motivation

Neuroscience

Neuroscience

Learning and Motivation

Neuroscience Letters

Drug and Alcohol Dependence

Biological Psychology

Neurobiology of Learning and Memory

Journal of Psychiatric Research

Current Opinion in Neurobiology

Post-retrieval extinction as reconsolidation interference: Methodological issues or boundary conditions?

Psychopharmacology

Contextual conditioning during free-operant extinction: Unsignaled, signaled, and backward-signaled noncontingent food

Animal Learning & Behavior

Contextual conditioning and reinstatement of extinguished instrumental responding

The Quarterly Journal of Experimental Psychology

Still at the choice point: Action selection and initiation in instrumental conditioning

Annals of the New York Academy of Sciences

Effects of ambient temperature on the relative reinforcing strength of MDMA using a choice procedure in monkeys

Psychopharmacology

A role of ventral tegmental area glutamate in contextual cue-induced relapse to heroin seeking

The Journal of Neuroscience

Ventral medial prefrontal cortex neuronal ensembles mediate context-induced relapse to heroin

Nature Neuroscience

Characterization of methylphenidate self-administration and reinstatement in the rat

Psychopharmacology

Differential control by context in the inflation and reinstatement paradigms

Journal of Experimental Psychology: Animal Behavior Processes

Context, time, and memory retrieval in the interference paradigm of Pavlovian learning

Psychological Bulletin

Signals for whether versus when an event will occur

A learning theory perspective on lapse and relapse and the maintenance of behavior change

Health Psychology

Context and behavioral processes in extinction

Learning & Memory

Role of conditioned contextual stimuli in reinstatement of extinguished fear

Journal of Experimental Psychology: Animal Behavioral Processes

Intertrial interval as a contextual stimulus: Further analysis of a novel asymmetry in temporal discrimination learning

Journal of Experimental Psychology: Animal Behavior Processes

State-dependent fear extinction with two benzodiazepine tranquilizers

Behavioral Neuroscience

Contextual control of the extinction of conditioned fear: Tests for the associative value of the context

Journal of Experimental Psychology: Animal Behavior Processes

Context effects on conditioning, extinction, and reinstatement in an appetitive conditioning preparation

Animal Learning & Behavior

Renewal of extinguished responding in a second context

Animal Learning & Behavior

Effects of contextual conditioning and unconditional stimulus presentation on performance in appetitive conditioning

The Quarterly Journal of Experimental Psychology

Renewal after the extinction of free operant behavior

Learning & Behavior

Relapse processes after the extinction of instrumental behavior: Renewal, resurgence, and reacquisition

Behavioral Processes

Context and extinction: Mechanisms of relapse in drug self-administration

Review
Behavioral and neurobiological mechanisms of extinction in Pavlovian and instrumental learning