Area 2: Neural Replay
Project 2.1: The Role of Replay in Statistical Learning of Successor Representations
Project 2.2: The Role of Replay in Generalization
A second main focus of our group is the study of neural replay in the human brain. Replay refers to the sequential reactivation of experience-related brain activity patterns during sleep, wakeful rest, and brief pauses from active behavior. Our motivation for studying replay stems from the fact that several studies have linked it to memory consolidation and behavior. In addition, machine learning research has shown that replay can lead to substantial performance improvements in artificial agents, where it can result in faster learning, less forgetting, better planning, and generalization (Wittkuhn et al., 2021). Our activities in this area are twofold: We have been actively developing new methods that address the challenge of studying replay non-invasively using fMRI (Schuck & Niv, 2019, Science; Wittkuhn & Schuck, 2021, Nature Communications), and we have used the newly developed analysis methods to gain novel insights about replay in the (human) brain.
Key References
Project 2.1: The Role of Replay in Statistical Learning of Successor Representations
Research Scientists
Lennart Wittkuhn
Christoph Koch
Lena M. Krippner
Nicolas W. Schuck
This project aims to investigate how on-task replay reflects and shapes statistical learning, which can also occur without conscious awareness and is sometimes thought to be hippocampus-independent. Our particular focus was to understand if statistical sequence learning can be described by a successor representation (SR) model and whether replay reflects the resulting predictive SR-representations during brief pauses from ongoing task performance. With its focus on SR mechanisms, this project links memory-related functions of replay to its putative role in planning, known from findings that show that place cell activations can sweep ahead of the animal along multiple potential future trajectories.
Human participants (n = 39) performed a statistical learning paradigm that exposed them to a fast-paced stream of six different images. Unbeknownst to the participants, the sequential ordering of images was governed by their arrangement in two ring-like graph structures that resulted in distinct transition probabilities between the images. Crucially, the image-to-image transition probabilities were structured such that distinct two-step, three-step etc. transitions could be inferred by participants. We hypothesized that while two or more step transitions were not necessary for responding quickly and accurately in our task, incidental statistical learning would still lead participants to extract higher-order predictive relationships among the task stimuli, and that this knowledge would then drive replay during on-task pauses.
Response time analysis showed that participants indeed developed multi-step transition expectations, as expected from a SR model. Interestingly, the predictive depth of this representation varied depending on which graph structure participants had learned and in which order. Applying the sequential fMRI pattern analysis methods developed in our previous work (Wittkuhn & Schuck, 2021), we examined the data for evidence of online neural replay during short on-task intervals that were interspersed with ongoing task performance. We used the SR model fitted to behavioral data to derive a probability distribution over possible sequences and examined our sequentiality metric for the most likely sequences given this predictive representation. This showed that particularly those sequences that were most likely given the SR model were replayed. Results from a post-task verbal questionnaire indicated that sequence knowledge remained implicit in half of the participants. Together, these results provide novel insights into how neural replay interacts with internal task representations in the brain and open avenues to further understand how the reactivation of experience supports adaptive behavior in humans.
Project 2.2: The Role of Replay in Generalization
Research Scientists
Sam Hall-McMaster
Lennart Wittkuhn
Luianta Verra
Noa Hedrich
Peter Dayan (MPI for Biological Cybernetics, Tübingen, Germany)
Nicolas W. Schuck
The ability to generalize our previous experience to new situations is central to human intelligence. When we visit New York for the first time, for example, we can navigate around Manhattan using the subway, even though the stations have completely different names and layouts from those we are used to. Doing this requires us to leverage similarities between our current and previous experiences.
In Project 2.2, we ask what role replay plays in this kind of generalization. Our core idea was that replay could support generalization by consolidating relationships between events and reactivating those previous events to guide choices in new scenarios. To investigate this notion, we conducted a three-session fMRI experiment (n = 52). During a behavioral pre-learning session, participants first learned associations between videos and static images. Participants then viewed the same videos inside the MRI scanner and needed to recall the associated images from memory. Data from this session then trained a pattern classification algorithm that was later used to identify sequential reactivation events in the brain. Participants next learned that the images from the first task phase occurred in a specific sequential order. Following sequential learning, participants were moved to the MRI scanner, where they were shown a series of cues that indicated their position within the sequence and asked to plan the next four upcoming images. This planning task was also done in the last fMRI session. Importantly, during this session participants had to learn a new order of images that was partly the same as the previous one. This overlap allowed participants to generalize their knowledge from the previous session to the final session. The goal for this session was to test the behavioral relevance of sequential reactivation for generalizing previous knowledge to a new decision problem. Resting state scans were acquired between active task phases.
Analysis of participants’ behavior revealed that participants learned to successfully plan sequences in session two and to generalize their knowledge to session three, as witnessed by the very high level of performance during session two and the more accurate responses in session three in trials that allowed the generalizing of learned sequence transitions from session two. The main neural results have centered on the visual cortex, where we have identified evidence for partial sequence recovery during active planning. We are now scrutinizing this result and confirming its validity before characterizing reactivation properties, including order and speed, as well as testing for sequence recovering during periods of rest. Neural responses in the medial temporal lobe, which includes the hippocampus, are much weaker in this dataset and we are continuing to assess the evidence for sequential reactivation in this area.