New research from an international team of neuroscientists has provided some of the first reliable evidence that a connection between replay, or mental simulation, and indirect learning exists in humans, illuminating a process that may help us consider the relationship between actions and outcomes and make effective decisions. 

The researchers designed a reward-based behavioral task that allowed them to study the neural mechanisms behind nonlocal learning in humans, or indirect learning through inferences, that was detailed in a paper published Thursday in Science. Nonlocal learning had not been extensively studied in people. The team found that the human participants used mental simulation processes that were similar to those in rodents in order to indirectly learn information about the task and earn rewards. 

"Effective decision-making incorporates new experience into our existing knowledge of the world. This allows us to infer the likely future consequences of different actions without having to experience them," the authors explained in the paper. "When you encounter a traffic jam at a crossroads, for example, you learn that the route just taken should be avoided, but you might also infer the value in avoiding the alternate paths leading to this same location."

Nathaniel D. Daw, a co-author of the paper and neuroscience professor at Princeton University, told The Academic Times that his previous work involved theoretical predictions about which possible courses of action the brain thinks about, when it thinks about them, and why. But his previous work was mostly based on data from rodents. He came together with the research team behind the current paper, including first authors Yunzhe Liu and Marcelo G. Mattar, to test these predictions in humans.

Prior studies of rodents have succeeded in measuring their neural replay, in which they think about spatial paths. For example, if the rodent sees a path change in a landscape with which it is familiar, such as a maze, it may think about taking that path and consider where it could lead. This is called replay; the term refers to a mental simulation process that creates representations of actions or situations that are not happening in the present moment.

Replay has mostly been observed in the brain's hippocampus region, which is associated with memory and spatial navigation. Daw said it is challenging to measure replay in humans because it requires invasive technology, but that scientists have assumed that humans use the same processes to make choices.

"It's easy to see that if I do X and get Y, I can connect X and Y in my brain; that's not hard," Daw said. "The hard part is that, what if I had done Z? Could I also have gotten to Y? That's the inference part, and that's what we're really trying to figure out how they do."

The task proposed in the paper measured the behavioral effect of nonlocal learning in humans and the corresponding neural processes. It did so by capturing whole-brain activity using magnetoencephalography, a neuroimaging technique that can pick up on electrical currents within the brain. The researchers manipulated factors of the task to test how the brain prioritized the processes.

"What we had people do is play a video game where they do trial-and-error decision-making to earn money, while decoding how they were thinking about the different steps in this video game, or paths," Daw said. "And what we found was actually surprising, and sort of backward from the idea of deliberation."

"It's not that when people were making a choice, they thought about where the path they were [following] would lead," he continued. "When they got to the end, they thought about different ways they could have gotten there." That sort of backward-looking cognition corresponds to the concept of backward replay. 

"Rodents think backwards, and that's what we also saw in the humans in our task," Daw explained. "And there was a connection between backward thinking and the choices they made."

The task that Daw and his colleagues created allowed them to successfully measure replay and connect it to the behavioral choices participants made during the task, which Daw said is among the first and best evidence to show that a connection between the two exists in humans. The team also studied how and why people chose specific paths out of the many that were available to them.

"Our proposal had been, theoretically, that the brain would be smart about selecting the most useful or the most relevant possible courses of action to contemplate, as it cannot do everything," Daw said. "And, indeed, that's what we saw. We saw that the most useful ones were the ones that were being reactivated in different situations."

"The data were consistent with the idea that the brain is judicious about managing its deliberation in this way — thinking about the paths that matter and not the paths that don't," he continued, noting that the brain assigns priority to the most useful paths that are most likely to help it make better choices in the future.

This discovery may also have implications in the mental health field, Daw said, because there is potential to measure and quantify thought processes in people with mental illnesses: "The promise here is if we could understand things that are going on in the brain that are actual biological mechanisms, and that are going wrong that are producing these [mental illness] symptoms, then in principle, we could study the disease better and understand it." He added that this could potentially lead to the development of better tests for mental illnesses in place of today's checklists of self-reported symptoms.

"I think being able to measure this, understand something about the mechanism, and quantify it is a big advance. A lot of our real-world, regular decisions depend on this kind of thing," Daw said.

The study, "Experience replay is associated with efficient nonlocal learning," published May 20 in Science, was authored by Yunzhe Liu, Beijing Normal University, Chinese Institute for Brain Research and University College London; Marcelo G. Mattar, University of California, San Diego; Timothy E. J. Behrens, University College London and University of Oxford; Nathaniel D. Daw, Princeton University; and Raymond J. Dolan, Beijing Normal University, University College London and Universitätsmedizin Berlin.