Researchers See Through a Mouse's Eyes by Decoding Brain Signals

May 3, 2023, Gizmodo article link here

You may have heard of getting a "bird's-eye view" on things, meaning looking at them from above. (Nowadays, we might have to change that to "drone's-eye view".) A perspective from the ground has sometimes been called a "worm's-eye view". The lenses on door peepholes produce a "fisheye effect" of extreme wide angle. All these animal names now tie into the Gizmodo article on what a mouse sees and what we can use with that information.

Scientists from the Swiss research facility École Polytechnique Fédérale de Lausanne (EPFL) have recently written some software that can decodes the brain signals in a mouse and allows us to see what it’s seeing. The algorithm detects a short black and white movie that the researchers show to the mouse, and plays it back with remarkable clarity. It's a little jerky and shaky, but it's still high quality. Watch a video of Professor Mackenzie Mathis explaining this and demonstrating a side-by-side comparison of the mouse's view and the original movie. I'll get into the details.

Photo credit: EPFL

Science fiction stories have had people tap into another's mind to see what they see using telepathy or some other means. How does the EPFL research actually work, though?

In 2011, investigators at the University of California Berkeley put 3 people into an MRI machine for several hours to monitor what their brains registered as they watched movie trailers. To translate that data into an image, they took 5,000 hours of random videos and told the computer to match the MRI data with the videos as best as it could. They compared that technique with the video information being an artist's paint pallet and the MRI data as what an artist imagines. The result was very blurry but still recognizable simulated moving images. But that's not real-time imaging.

From the UC-Berkeley video

Imagine, if you can, just how much information your brain has to process from your eyes from moment to moment. It's a lot! Here's a video to show you what you can see as your eyes track around a picture (the small white circle on the right picture). Putting all of that together by your brain to make the perceived smooth view that we experience is an amazing feat.

Compare the white circled view on the right with the close up on the left

The EPFL researchers described 2 techniques to gather the data from mice, like the UC-Berkeley people gathered data from humans. Their techniques let the researchers actually see nerve cells (neurons) firing signals to each other while the mouse was watching the short video. They can see the neurons in action because they flicker and glow like Christmas lights, but all in one color! How can they do that?

Both techniques attach a device to a mouse's skull. In one case, the mouse is restrained (left below), and in the other, it is free to move around (right). The device is something called a two-photon microscope which sends a laser into specific areas of the brain and records on computer video pictures of the neurons flickering in real time as they fire. Which neurons fire can be tracked as the mouse does whatever the researchers are investigating.

Screenshot from video by by Raul Ramos and Emmanuel J. Rivera-Rodríguez

Early experiments by other scientists could only look at nerve cells near the surface of the brain. Those were called patch-clamp experiments, and what they did was literally attach a glass tube (pipette) with an electrode directly to one neuron and measure the electrical activity when the nerve cell fired.

Image from a 2014 research paper by Valentina Castagnola

Now, people like Mackenzie Mathis can see all cells firing in a large area without the need to attach this delicate tube to a cell. So, what's this "firing" all about, and how do they see flashes of light? In the 1966 movie Fantastic Voyage, people were shrunk to go inside a man's brain to dissolve a blood clot, and they showed electrical impulses flickering around them. Here's a video clip showing how the movie demonstrated electrical pulses flickering. But those were just movie special effects. What's reality?

The outer skin of the neuron is called a cell membrane. It has 2 layers shown in orange below. When it is resting, the sodium and chloride ions outside the cell (top) and the potassium and organic ions inside the cell (bottom) have a difference in total electrical charge. The inside of the cell is more negative by about 70 millivolts. To "trigger" a neuron into firing, chemicals called neurotransmitters enter from another neuron, and they open up special pores (actually, sodium pumps, green below) in the membrane to force more sodium enter the cell. This changes the millivolt difference (electrical potential) between outside and inside and is the beginning of a signal.

Image from 2-minute neuroscience video

It moves down the nerve cell until it reaches the end where another nerve cell is waiting.

Image modified from textbook Molecular Cell Biology, 4th edition

The signal then turns on another pore down there (blue) to allow calcium ions in, and those cause a network of bags (vesicles) to float free of their protein network strands. It's like removing Christmas ornaments from the tree.

Image modified from 2-minute neuroscience video

They contain the neurotransmitters (the ones mentioned earlier), and when they spill their contents out the end of the nerve cell, the neurotransmitters float the 40 nm distance to the next nerve cell, and the signal process continues. 

Signal from the left neuron causes calcium at the end to enter the cell, and that helps neurotransmitter vesicles move to the end, where they float to the neuron on the right, and the signal continues. (from video 2-minute neuroscience)

With all that in mind, how did the EPFL researchers get the firing cells to glow? It's not movie magic. Calcium is the key. The two techniques Mathis mentioned put a special gene on the cell's DNA to make a molecule called GCamp. When GCamp has calcium around and sticks to it, part of the molecule glows, and since the molecule is inside the cell, the cell lights up wherever GCamp is stuck to calcium.

• One way to put GCamp in cells is to inject a virus that infects cells and carries the GCamp gene to the mouse cell DNA. This works only for the cells that the virus solution touches, a pretty limited area.

Image from article in Frontiers in Neural Circuits

• The other way is to genetically engineer the mice themselves before they are born so they already have GCamp in every nerve cell DNA. It is a much more expensive method, but every neuron will have the potential (no pun intended) to flash when they send signals. To make such a mouse, you inject a mouse egg with the gene for GCamp, then the baby mouse has it.

Image modified from 2022 article in Cells journal

So, after the GCamp is in the nerve cells, all you have to do is watch their responses with a two-photon (laser) microscope to do the calcium imaging. Mathis said ten years ago, this was done to see what still images mice detected from watching pictures or symbols, but now the technology is so much better. They can records millions of neurons firing and put the data together to make video reproductions of the short movie clip that the mice saw. The data are measurements of the voltage of each signal for individual neurons, and that information is collected like micro-encephalograms which the computer translates into the video.

Image taken from YouTube video by Animated biology With arpan

This type of imaging maps the exact locations of neurons that fire in response to whatever you use to stimulate the mouse, whether they watch a video and you "see" what they see, or you simply tickle their whiskers and see what part of the brain is used to detect that feeling. So, the whole point is not just to see from a mouse's point of view, but to learn what parts of the brain control the animal, and maybe which parts aren't working properly.