AI turns thoughts into text

Researchers at the University of Texas at Austin have developed a new artificial intelligence system called a “semantic decoder” that has the ability to translate brain activity of a person in a continuous stream of text.

This innovative technology has the potential to help people who are mentally aware but cannot develop speech, such as those who have been debilitated by strokes.

When listening to a story or silently imagining the narrative, the system decodes brain signals and converts them into text, enabling the expression of these people's ideas and thoughts.

This promising achievement could open new perspectives for communication and quality of life for individuals facing speech challenges due to medical conditions or injuries.

The study, which resulted in the development of the "semantic decoder", was conducted by Jerry Tang, a student at in Computer Science, and Alex Huth, Assistant Professor of Neuroscience and Computer Science at university.

The results of this research were published in the journal Nature Neuroscience, one of the most respected scientific publications in the field.

Tang and Huth's joint leadership of the research highlights the collaboration between computer science and of neuroscience, seeking significant advances in the interface between the human brain and intelligence artificial.

The work carried out by the researchers makes use of a transformer model, similar to those used in systems such as Bard – from Google – and ChatGPT – from OpenAI.

However, the system developed by the researchers is different because it does not require surgical implants in the subjects, making it a non-invasive method. Furthermore, unlike other language decoding systems under development, participants are not limited to a prescribed list of words to communicate.

How does the 'mind reader' method work?

After extensive decoder training, in which the patient listens to hours of podcasts on the scanner, brain activity is measured using an fMRI scanner.

Later, if the participant is willing to have his thoughts decoded, the machine is able to generate the corresponding text from brain activity alone, whether listening to a new story or imagining telling a new one. history.

The researchers designed the decoding system to capture the essence of what is being said or thought, rather than providing an exact word-for-word transcript.

Although imperfect, the system has demonstrated the ability to produce text that approximates, and sometimes accurately, the intended meanings of the original words.

The decoder developed by the researchers allows the continuous decoding of language for long periods of time, encompassing complex ideas.

During approximately half of the time that the decoder was trained to monitor a participant's brain activity, the machine generated text that reflected the desired meanings of words, contributing to more effective communication and understandable.

According to Huth, this approach represents a significant advance compared to previous methods, which were often limited to single words or short sentences.

The system does not seek a literal word-for-word transcription, but the capture of the essence of what is being said or thought, even if imperfectly.

Although the current system relies on the use of a functional magnetic resonance imaging (fMRI) scanner, which limits its viability outside the laboratory environment, the researchers believe their work can be adapted for more portable brain imaging systems such as functional near-infrared spectroscopy (fNIRS).

According to Huth, fNIRS measures blood flow in the brain at different points in time, which is essentially the same type of signal that fMRI detects.

Therefore, the approach used in the study could be applied to fNIRS. Despite this limitation, it is believed that the essence of method developed by the researchers can be adapted for fNIRS, paving the way for a more portable and accessible system for decoding brain activity.

Lover of movies and series and everything that involves cinema. An active curious on the networks, always connected to information about the web.