"
NEWS / 0445

Technology

Brain-IT Rebuilds Viewed Images From Brain Scans but Can't Reach Memories

Published
Nov 13, 2026
Views
489

Brain-IT rebuilds images people viewed from fMRI scans, matching methods trained on 40 hours of data using just one hour. Memories remain out of reach.

Brain-IT Rebuilds Viewed Images From Brain Scans but Can't Reach Memories

Brain-IT is an AI system that can reconstruct the pictures people were looking at, using only their brain activity. The results are eerily close to the originals. But the system's real significance is narrower than the headline images suggest, and that's worth sorting out before anyone gets carried away.

The tool relies on functional magnetic resonance imaging (fMRI) to measure activity in the visual cortex, the region that processes what we see. A computer model first predicts the broad layout of an image and high-level information about its content. It then hands those predictions to an AI image generator. The researchers laid out the method in a study posted to the preprint database arXiv, and the work was presented at the ICLR 2026 conference, held April 23-25. The authors publicized it in September.

Brain-IT grows out of an earlier approach called the Universal Brain Encoder. That model runs in the opposite direction. It starts with an image and predicts the fMRI response expected in individual voxels, which are small three-dimensional units of a brain scan.

"The key idea is that every voxel, a tiny cube of brain tissue, gets its own learned vector that captures what it cares about in images," said study first author Roman Beliy, a doctoral candidate in the Department of Computer Science and Applied Mathematics at the Weizmann Institute of Science in Israel.

Some voxels appear tuned to simple features such as edges, textures or locations. Others seem to respond to higher-level categories like faces, bodies or scenes. That variety is what makes the approach workable.

Peering inside your mind

Earlier image-to-fMRI systems were typically trained separately for each participant and dataset. The Universal Brain Encoder takes a more flexible route. It learns from data gathered across many people and can carry information between participants, datasets and MRI systems with different magnetic-field strengths, according to the study.

Here's the thing: a voxel can work out which parts of an image matter to it, based on its learned profile. Beliy's shorthand for this is a voxel asking itself, "I recognize edges. Which parts of this image are edges?"

Brain-IT runs the logic backward. Where the encoder goes from image to predicted activity, Brain-IT starts from measured activity and asks what the person was looking at. It sorts thousands of small brain-scan measurements from the visual cortex into 128 groups, based on the kinds of visual information they tend to carry. The groups then share clues with one another. That lets the system combine cues about a scene's shape, color and likely objects into a fuller picture, Beliy explained.

The decoder produces two kinds of prediction. The first captures semantic information, meaning the features tied to what's probably in a scene. For a dog standing in grass, that means the features of a dog and an outdoor, grassy setting. The second captures structure: the broad visual layout and how shapes are arranged. In the same example, that's the outline of the dog and where it sits in the frame.

The structural prediction gives the image generator a starting point, so it doesn't begin from pure noise. The semantic prediction then guides the generator as it strips away the remaining static until the picture resolves.

The original image of a baseball game (left) and the reconstruction (right) generated by the Brain-IT AI model based on the brain activity of a person viewing the image. (Image credit: Brain-IT)

Even so, the system can't read thoughts in any general sense. It can recreate what people see. It can't reproduce memories or imagined scenes.

"We're not reading people's thoughts," said study co-author Michal Irani, a computer scientist at the Weizmann Institute of Science, in an interview with Live Science. "The images we do manage to recover from fMRI require people's cooperation and full intended collaboration."

How scientists tested the model

The team compared Brain-IT's reconstructions with the images participants actually saw. According to the authors, the system beat previous leading methods on standard objective reconstruction metrics and produced more faithful results.

The most striking number concerns data efficiency. When only limited fMRI data was available from a new participant, Brain-IT performed well. One hour of data from a new person produced results as accurate as earlier approaches trained on 40 hours of recordings. For clinical or research settings where scan time is scarce, that gap matters.

A high score or a realistic-looking output doesn't mean the reconstruction is a perfect copy of what someone saw, though. That's the caveat that deserves the most attention.

The final stage of processing uses an off-the-shelf diffusion generator trained on massive numbers of images. Models like this produce plausible pictures. They smooth textures, sharpen facial features and fill in missing details. Beliy said this "beautification" can make reconstructions more polished and convincing, and it can add details that don't necessarily come from the brain data. The scan constrains the output, but part of the realism comes from patterns the model learned from photographs.

Why this is not memory decoding

The gap between seen images and remembered ones is where the technology's limits become clear. The system works well on images people are actively viewing, and it can't reconstruct an imagined or remembered picture.

"Brain-IT relies on how the brain processes a seen image," Irani said. "Information flows from the eyes to the primary visual cortex and onward through a series of visual areas that recognize objects and scenes."

Training depends on a clean reference. In a controlled experiment, researchers know exactly which image was shown, which gives the model a target during both training and evaluation. Memories and imagined images don't come with that reference. They can be vague, and they're shaped by brain networks involved in memory and thought. Visual regions reactivate only partially and more weakly, Irani said.

Hardware is the other constraint. Reliable results currently require fMRI, which is cumbersome and expensive. The team is experimenting with electroencephalogram (EEG) data, which is gathered through electrodes placed on the head and is far easier to collect. Irani said that work is in the very early stages, so don't expect a wearable version of this any time soon.

Where this could lead

The nearer-term payoff is for neuroscience. The system could help researchers map which visual information is represented in different brain regions, how those regions interact and how individuals' patterns differ. Those differences could eventually help clinicians spot neurological, neurodevelopmental and psychiatric conditions by revealing shifts in visual processing.

Irani pointed to that broader horizon. "There are numerous useful medical applications which we are currently striving toward," she said. "These include using our brain encoder/decoder to detect and analyze brain disorders such as MS, PTSD, autism and psychiatric disorders."

Those applications are a long way off, and the current work doesn't prove them. What it does show is that a model trained across many people can carry what it learns to new participants with very little data. That's the part I'd watch. If the efficiency holds up beyond the lab, the bottleneck shifts from collecting brain data to deciding which clinical questions are worth asking of it.

Source: Alan Bradley · www.livescience.com

Discussion

Sign in to join the discussion.