Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Neural reactivations during sleep determine network credit assignment

Nature Neuroscience volume 20, pages 1277–1284 (2017)Cite this article

Subjects

This article has been updated

Abstract

A fundamental goal of motor learning is to establish the neural patterns that produce a desired behavioral outcome. It remains unclear how and when the nervous system solves this 'credit assignment' problem. Using neuroprosthetic learning, in which we could control the causal relationship between neurons and behavior, we found that sleep-dependent processing was required for credit assignment and the establishment of task-related functional connectivity reflecting the casual neuron–behavior relationship. Notably, we observed a strong link between the microstructure of sleep reactivations and credit assignment, with downscaling of non-causal activity. Decoupling of spiking to slow oscillations using optogenetic methods eliminated rescaling. Thus, our results suggest that coordinated firing during sleep is essential for establishing sparse activation patterns that reflect the causal neuron-behavior relationship.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Rescaling of task activations after sleep.
Figure 2: Changes in functional connectivity of direct neuronal pairs and reactivation microstructure.
Figure 3: Consistency of reward and frames of reference.
Figure 4: Pairwise correlation of neural firing during task performance and reactivations during sleep.
Figure 5: Optogenetic inhibition of neural activity during sleep.
Figure 6: Optogenetic inhibition during UP states prevents consolidation.
Figure 7: Optogenetic inhibition during UP states prevents rescaling of task activations.

Similar content being viewed by others

Change history

  • 18 July 2017

    In the version of this article initially published online, the abstract read "casual neuron–behavior relationship" instead of "causal neuron–behavior relationship." The error has been corrected in the print, PDF and HTML versions of this article.

  • 31 July 2017

    In the version of this article initially published online, the x-axis label for the righthand column in each graph in Figure 6b read BMI1Early; it should have read BMI2Early. The error has been corrected in the print, PDF and HTML versions of this article.

References

  1. Yin, H.H. et al. Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat. Neurosci. 12, 333–341 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  2. Dayan, E. & Cohen, L.G. Neuroplasticity subserving motor skill learning. Neuron 72, 443–454 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  3. Tumer, E.C. & Brainard, M.S. Performance variability enables adaptive plasticity of 'crystallized' adult birdsong. Nature 450, 1240–1244 (2007).

    CAS  PubMed  Google Scholar 

  4. Shmuelof, L. & Krakauer, J.W. Are we ready for a natural history of motor learning? Neuron 72, 469–476 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  5. Peters, A.J., Chen, S.X. & Komiyama, T. Emergence of reproducible spatiotemporal activity during motor learning. Nature 510, 263–267 (2014).

    CAS  PubMed  Google Scholar 

  6. Ganguly, K. & Carmena, J.M. Emergence of a stable cortical map for neuroprosthetic control. PLoS Biol. 7, e1000153 (2009).

    PubMed  PubMed Central  Google Scholar 

  7. Huber, D. et al. Multiple dynamic representations in the motor cortex during sensorimotor learning. Nature 484, 473–478 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  8. Ganguly, K., Dimitrov, D.F., Wallis, J.D. & Carmena, J.M. Reversible large-scale modification of cortical networks during neuroprosthetic control. Nat. Neurosci. 14, 662–667 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  9. Abbott, L.F., DePasquale, B. & Memmesheimer, R.M. Building functional networks of spiking model neurons. Nat. Neurosci. 19, 350–355 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Lee, D., Seo, H. & Jung, M.W. Neural basis of reinforcement learning and decision making. Annu. Rev. Neurosci. 35, 287–308 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  11. Clancy, K.B., Koralek, A.C., Costa, R.M., Feldman, D.E. & Carmena, J.M. Volitional modulation of optically recorded calcium signals during neuroprosthetic learning. Nat. Neurosci. 17, 807–809 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  12. Tononi, G. & Cirelli, C. Sleep and the price of plasticity: from synaptic and cellular homeostasis to memory consolidation and integration. Neuron 81, 12–34 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  13. Diekelmann, S. & Born, J. The memory function of sleep. Nat. Rev. Neurosci. 11, 114–126 (2010).

    CAS  PubMed  Google Scholar 

  14. Genzel, L., Kroes, M.C., Dresler, M. & Battaglia, F.P. Light sleep versus slow wave sleep in memory consolidation: a question of global versus local processes? Trends Neurosci. 37, 10–19 (2014).

    CAS  PubMed  Google Scholar 

  15. Cramer, S.C. et al. Motor cortex activation is preserved in patients with chronic hemiplegic stroke. Ann. Neurol. 52, 607–616 (2002).

    PubMed  Google Scholar 

  16. Marshall, L. & Born, J. The contribution of sleep to hippocampus-dependent memory consolidation. Trends Cogn. Sci. 11, 442–450 (2007).

    PubMed  Google Scholar 

  17. Wilson, M.A. & McNaughton, B.L. Reactivation of hippocampal ensemble memories during sleep. Science 265, 676–679 (1994).

    CAS  PubMed  Google Scholar 

  18. Nere, A., Hashmi, A., Cirelli, C. & Tononi, G. Sleep-dependent synaptic down-selection (I): modeling the benefits of sleep on memory consolidation and integration. Front. Neurol. 4, 143 (2013).

    PubMed  PubMed Central  Google Scholar 

  19. Jarosiewicz, B. et al. Functional network reorganization during learning in a brain-computer interface paradigm. Proc. Natl. Acad. Sci. USA 105, 19486–19491 (2008).

    CAS  PubMed  PubMed Central  Google Scholar 

  20. Koralek, A.C., Jin, X., Long, J.D. II, Costa, R.M. & Carmena, J.M. Corticostriatal plasticity is necessary for learning intentional neuroprosthetic skills. Nature 483, 331–335 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  21. Taylor, D.M., Tillery, S.I. & Schwartz, A.B. Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832 (2002).

    CAS  PubMed  Google Scholar 

  22. Moritz, C.T., Perlmutter, S.I. & Fetz, E.E. Direct control of paralyzed muscles by cortical neurons. Nature 456, 639–642 (2008).

    CAS  PubMed  PubMed Central  Google Scholar 

  23. Gulati, T., Ramanathan, D.S., Wong, C.C. & Ganguly, K. Reactivation of emergent task-related ensembles during slow-wave sleep after neuroprosthetic learning. Nat. Neurosci. 17, 1107–1113 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  24. Gulati, T. et al. Robust neuroprosthetic control from the stroke perilesional cortex. J. Neurosci. 35, 8653–8661 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  25. Fetz, E.E. Volitional control of neural activity: implications for brain-computer interfaces. J. Physiol. (Lond.) 579, 571–579 (2007).

    CAS  Google Scholar 

  26. Koralek, A.C., Costa, R.M. & Carmena, J.M. Temporally precise cell-specific coherence develops in corticostriatal networks during learning. Neuron 79, 865–872 (2013).

    CAS  PubMed  Google Scholar 

  27. Orsborn, A.L. et al. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron 82, 1380–1393 (2014).

    CAS  PubMed  Google Scholar 

  28. Mitchell, J.F., Sundberg, K.A. & Reynolds, J.H. Spatial attention decorrelates intrinsic activity fluctuations in macaque area V4. Neuron 63, 879–888 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  29. Watson, B.O., Levenstein, D., Greene, J.P., Gelinas, J.N. & Buzsáki, G. Network homeostasis and state dynamics of neocortical sleep. Neuron 90, 839–852 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  30. Peyrache, A., Khamassi, M., Benchenane, K., Wiener, S.I. & Battaglia, F.P. Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat. Neurosci. 12, 919–926 (2009).

    CAS  PubMed  Google Scholar 

  31. Ramanathan, D.S., Gulati, T. & Ganguly, K. Sleep-dependent reactivation of ensembles in motor cortex promotes skill consolidation. PLoS Biol. 13, e1002263 (2015).

    PubMed  PubMed Central  Google Scholar 

  32. Lansink, C.S., Goltstein, P.M., Lankelma, J.V., McNaughton, B.L. & Pennartz, C.M. Hippocampus leads ventral striatum in replay of place-reward information. PLoS Biol. 7, e1000173 (2009).

    PubMed  PubMed Central  Google Scholar 

  33. de Lavilléon, G., Lacroix, M.M., Rondi-Reig, L. & Benchenane, K. Explicit memory creation during sleep demonstrates a causal role of place cells in navigation. Nat. Neurosci. 18, 493–495 (2015).

    PubMed  Google Scholar 

  34. Singer, A.C. & Frank, L.M. Rewarded outcomes enhance reactivation of experience in the hippocampus. Neuron 64, 910–921 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  35. Churchland, M.M. et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat. Neurosci. 13, 369–378 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  36. Song, W. & Giszter, S.F. Adaptation to a cortex-controlled robot attached at the pelvis and engaged during locomotion in rats. J. Neurosci. 31, 3110–3128 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  37. Miyamoto, D. et al. Top-down cortical input during NREM sleep consolidates perceptual memory. Science 352, 1315–1318 (2016).

    CAS  PubMed  Google Scholar 

  38. Chuong, A.S. et al. Noninvasive optical inhibition with a red-shifted microbial rhodopsin. Nat. Neurosci. 17, 1123–1129 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  39. Steriade, M., Nuñez, A. & Amzica, F. A novel slow (<1 Hz) oscillation of neocortical neurons in vivo: depolarizing and hyperpolarizing components. J. Neurosci. 13, 3252–3265 (1993).

    CAS  PubMed  PubMed Central  Google Scholar 

  40. Yang, G. et al. Sleep promotes branch-specific formation of dendritic spines after learning. Science 344, 1173–1178 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  41. de Vivo, L. et al. Ultrastructural evidence for synaptic scaling across the wake/sleep cycle. Science 355, 507–510 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  42. Maret, S., Faraguna, U., Nelson, A.B., Cirelli, C. & Tononi, G. Sleep and waking modulate spine turnover in the adolescent mouse cortex. Nat. Neurosci. 14, 1418–1420 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  43. Gupta, A.S., van der Meer, M.A., Touretzky, D.S. & Redish, A.D. Hippocampal replay is not a simple function of experience. Neuron 65, 695–705 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  44. O'Doherty, J.P., Cockburn, J. & Pauli, W.M. Learning, reward, and decision making. Annu. Rev. Psychol. 68, 73–100 (2017).

    PubMed  Google Scholar 

  45. Schultz, W. Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115 (2006).

    PubMed  Google Scholar 

  46. Ishii, S., Yoshida, W. & Yoshimoto, J. Control of exploitation-exploration meta-parameter in reinforcement learning. Neural Netw. 15, 665–687 (2002).

    PubMed  Google Scholar 

  47. Wallstrom, G., Liebner, J. & Kass, R.E. An implementation of Bayesian adaptive regression splines (BARS) in C with S and R Wrappers. J. Stat. Softw. 26, 1–21 (2008).

    PubMed  PubMed Central  Google Scholar 

  48. Mitra, P. & Bokil, H. Observed Brain Dynamics (Oxford University Press, 2008).

Download references

Acknowledgements

This work was supported by awards from the Department of Veterans Affairs, Veterans Health Administration (VA Merit: 1I01RX001640 to K.G., VA CDA 1IK2BX003308 to D.S.R.); the National Institute of Neurological Disorders and Stroke (1K99NS097620 to T.G. and 5K02NS093014 to K.G.); the American Heart/Stroke Association (15POST25510020 to T.G.); the Burroughs Wellcome Fund (1009855 to K.G.); and start-up funds from the SFVAMC, NCIRE and UCSF Department of Neurology (to K.G.).

Author information

Authors and Affiliations

  1. Neurology and Rehabilitation Service, San Francisco Veterans Affairs Medical Center, San Francisco, California, USA

    Tanuj Gulati, Ling Guo, Dhakshin S Ramanathan, Anitha Bodepudi & Karunesh Ganguly

  2. Department of Neurology, University of California-San Francisco, San Francisco, California, USA

    Tanuj Gulati, Ling Guo, Anitha Bodepudi & Karunesh Ganguly

  3. Center for Neural Engineering and Prostheses, University of California-Berkeley and University of California-San Francisco, California, USA

    Tanuj Gulati, Ling Guo, Dhakshin S Ramanathan & Karunesh Ganguly

  4. Department of Psychiatry, San Francisco Veterans Affairs Medical Center, San Francisco, California, USA

    Dhakshin S Ramanathan

  5. Department of Psychiatry, University of California-San Francisco, San Francisco, California, USA

    Dhakshin S Ramanathan

Authors
  1. Tanuj Gulati
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ling Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dhakshin S Ramanathan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anitha Bodepudi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Karunesh Ganguly
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

T.G. and K.G. conceived of the experiments. L.G. and T.G. performed surgical procedures and collected the data. A.B., D.S.R. and T.G. analyzed the data. T.G. and K.G. wrote the manuscript. L.G. and D.S.R. edited the manuscript.

Corresponding author

Correspondence to Karunesh Ganguly.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Modulations of TRD neurons are not significantly high during random time points in sleep.

a, Mean modulation of all TRD and TRI neurons at randomly picked times during Sleeppost. For this analysis, snippets of activity (50 ms window of spiking activity that was then binned at 5 ms resolution) were randomly sampled from all the reactivation times. b, Scatter plots showing the modulation depth for all the individual TRD and TRI (mean in solid line ± s.e.m. in box; unpaired t test t121 = 0.69, P = 0.49).

Supplementary Figure 2 Depth modulation during sleep reactivations predicts rescaling of task–related firing rate during BMI2.

MDreactivation from Sleeppost (in Fig 2c) are compared to changes in modulation from BMI1 to BMI2 for both TRD and TRI neurons (linear regression R2 = 0.17, P < 10-5).

Supplementary Figure 3 Characteristics of OPTOUP and OPTODOWN experiments.

a, Comparison of the total number of 100 ms optogenetic stimulation periods for OPTOUP and OPTODOWN experiments (mean in solid line ± s.e.m. in box; color convention same as Fig 5; unpaired t test t17 = 0.99, P = 0.33). b, Comparison of the proportion of total time that the LED was turned on for the respective Sleeppost during the OPTOUP and OPTODOWN experiments (mean in solid line ± s.e.m. in box; color convention same as Fig 5; unpaired t test t17 = 2.07, P = 0.054).

Supplementary Figure 4 Sleep durations in optogenetic experiments.

Comparison of the total sleep durations for Sleeppre and Sleeppost during the OPTOUP, OPTODOWN and OPTOOFF experiments (mean in solid line ± s.e.m. in box, one-way ANOVA, F5,48 = 0.92, P = 0.47; post hoc t test shows no significant difference between any pairwise comparison).

Supplementary Figure 5 PC weights for PCA based reactivation analysis.

Examples of the first principal components from two separate experiments. Notably, TRD and TRI neurons both had non-zero weights (TRD enclosed in red box).

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gulati, T., Guo, L., Ramanathan, D. et al. Neural reactivations during sleep determine network credit assignment. Nat Neurosci 20, 1277–1284 (2017). https://doi.org/10.1038/nn.4601

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.4601

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing