University of Texas at Austin researchers have devised a
non-intrusive approach
that uses AI and brain scans to transform a person's thoughts into sentences.
The "
semantic decoder
" can transcribe the gist of what an individual has heard, viewed, or imagined and translate it into a stream of text, according to their research. Although the technology is still in its early stages, it has the
potential to serve as a foundation for non-invasive brain-computer interfaces that could help people with disabilities such as paralysis communicate intelligibly.
- For the first time, scientists were able to decode an individual's "continuous language" from their brain recordings without the use of surgical implants.
-
The study involved three human participants who listened to narrative podcasts while inside an fMRI machine for 16 hours.
- The researchers mapped how the words translated to responses in language-processing parts of the brain.
- A large language model was used to match patterns in the participants' brain activity to phrases and words they heard.
- Later, the participants listened to a new story or imagined telling a story, and the decoder generated corresponding text from their brain activity.
- While not a word-for-word transcript, the decoder could "recover the gist of what the user was hearing," said lead author Jerry Tang.
- The study, which was published in Nature Neuroscience, emphasizes that "brain-computer interfaces should respect mental privacy."
