Scientists Recreate Movies From Mouse Brain Activity

Breakthrough in Neural Decoding

Imagine watching a movie through someone else's eyes, literally. Scientists have successfully reconstructed videos purely from the brain activity of mice, showing what the mice were seeing, in a new study led by UCL researchers. The findings, published in eLife, could help shed light on the intricate workings of how the brain processes visual information and open new avenues for exploring how different species perceive the world.

This remarkable achievement represents a significant leap beyond previous research that used fMRI scans on humans. The new study builds on this approach but instead used single-cell recordings in mice, which offer the potential to provide a more precise measure of the brain's representations. The team recorded activity from approximately 8,000 individual neurons in the visual cortex while mice watched 10-second video clips.

How Brain Movies Work

Using the same dataset, the UCL team further refined this model by calculating the difference between the predicted activity of neurons if a mouse had seen a blank screen, and the actual activity of the neurons (measured using a microscopic imaging technique that detects which individual brain cells are firing based on localised boosts in calcium levels). This allowed the scientists to gradually update the pixels of the blank movie through an algorithm and refine the model so that the output video closely resembled the video presented to the mouse.

Once the model was sufficiently trained, the researchers were then able to construct a 10-second movie based on a mouse's neural activity alone, measured as the mouse watched a video that had not been used previously in training the model. In this case, the researchers achieved a correlation of up to 0.569 between the original and reconstructed videos.

Beyond Perfect Vision

The research reveals something profound about how brains process visual information. Dr Bauer concluded: "We don't have a perfect representation of the world in our heads. The visual processing pipeline skews and warps our representation in a way that modifies information. This deviation between reality and representations in the brain is not necessarily an error but a feature, reflecting how our minds interpret and augment sensory information.

Lead author Dr. Joel Bauer (Sainsbury Wellcome Centre at UCL) said, "We wanted to have a better way of investigating how the brain interprets what we see. The current methods of understanding what specific groups of neurons are representing are not very generalizable to situations which haven't been specifically tested for. And so, we wanted to develop a method that can capture what is being represented in the brain and compare that to reality."

Future Implications

While the current reconstructions focus on timing accuracy rather than perfect resolution, the implications stretch far beyond laboratory curiosities. Next, the team plan to use the technique to uncover new insights into the brain's visual processing capabilities. Specifically, they are interested in understanding how visual representations in the brain can deviate from what's actually in front of our eyes.

This technology could eventually help researchers understand how different species perceive their environments, potentially revolutionizing fields from neuroscience to artificial intelligence. The ability to decode visual experiences directly from brain activity opens doors to understanding consciousness itself, though researchers emphasize they're still in the early stages of this scientific frontier.