Generative AI creates video based on a person's thoughts

In recent years, we have seen steady progress in the capabilities of machines equipped with artificial intelligence (AI), including with regard to reading human minds.

Accordingly, researchers have used AI-based video generation technology to provide a “real” view of what is going on in our minds.

The interpretation of air signals is mainly driven by the hope that, one day, we will be able to offer new ways of communication for people in a coma or with different forms of paralysis.

In addition, technology can also create more intuitive interfaces between humans and machines, with possible applications for healthy people.

To date, most research has focused on recreating patients' internal monologues by identifying the words they are thinking about using AI systems.

Although the most promising results have been achieved using invasive aerial implants, this approach is unlikely to be the practice most people use.

AI used to create “mind videos”

Researchers at the National University of Singapore and the Chinese University of Hong Kong achieved breakthroughs by combining non-invasive embedded scans with image generation technology. AI images.

They could create short snippets of video that are strikingly similar to the clips participants were watching when their radial data was collected.

To achieve this result, the researchers first trained a model using large datasets collected using fMRI aerial scanners.

They then combined that model with imaging technology AI Stable Diffusion, open source, to create the corresponding images.

One article recently published on the arXiv prepress server takes a similar approach to the authors' previous research.

However, this time they adapted the system to interpret radial data streams and convert them into videos instead of still images.

At first, the researcher followed the training of the model using extensive data sets fMRI so that he could gain knowledge of the general characteristics of these scans electrical.

Then they expanded the training so that the model could process a sequence of fMRI scans instead of treating them individually.

Subsequently, the model was maintained for a new training, this time using the combination of fMRI scans, video clips that evoked this brain activity, and text sequence corresponding.

In a separate approach, the researcher adapted the pre-trained model Stable Diffusion to generate videos instead of still images.

This model was then subjected to a new training using the same videos and text sequence that were used in training the first model.

Subsequently, the two models were combined and fitted using the fMRI scans and their respective associated videos.

Search result

After combining and tuning the models, the resulting system was able to perform new fMRI scans that had not been previously available. previously found and generate videos that had revealed similarities to the clips that the human participants had watched.

While there is still room for improvement, the AI ​​output is generally very close to the original videos, accurately reproducing scenes of crops or horse herds, and maintaining visualization with the color palette used.

The researchers behind the study say this area of ​​research has potential applications both in fundamental neuroscience and in future brain-machine interfaces.

However, they also recognize the need for government regulations and efforts by the scientific community to protect the privacy of biological data and prevent potential malicious uses of this technology as authorized in their work.

This line of research paves the way for advances capable of reaching the understanding of the human mind and the development of technologies that can establish more sophisticated interfaces between the brain and machines.

While there are important considerations to be addressed, such as protecting personal data and preventing abuse, the potential scientific and technological benefit is promising.

Lover of movies and series and everything that involves cinema. An active curious on the networks, always connected to information about the web.