Mind-reading AI can translate brainwaves into written text

Using only a sensor-filled helmet combined with artificial intelligence, a team of scientists has announced they can turn a person’s thoughts into written words.

In the study, participants read passages of text while wearing a cap that recorded electrical brain activity through their scalp using an electroencephalogram (EEG). These recordings were then converted into text using an AI model called DeWave.

Chin-Teng Lin from the University of Technology Sydney (UTS), Australia, says the technology is non-invasive, relatively inexpensive and easily transportable.

While the system is far from perfect, with an accuracy of approximately 40 per cent, Lin says more recent data currently being peer-reviewed shows an improved accuracy exceeding 60 per cent.

In the study presented at the NeurIPS conference in New Orleans, the participants read the sentences aloud, even though the DeWave program does not use spoken words. However, in the team’s latest research, participants read the sentences silently.

Last year, a team led by Jerry Tang at the University of Texas at Austin reported a similar accuracy converting thoughts to text, but they used MRI scans to interpret brain activity. Using EEG is much more practical, as subjects don’t have to lie still inside a scanner.

The DeWave system was trained by looking at lots of examples where these brain signals match up with specific sentences, says team member Charles Zhou.

“For instance, when you think about saying ‘hello’, your brain sends out certain signals,” says Zhou. “DeWave learns how these signals relate to the word ‘hello’ by seeing many examples of these signals for different words or sentences.

Once DeWave understood the brain signals well, the team connected it to an open-source large language model (LLM), akin to the AI that powers ChatGPT.

“This LLM is like a brainy writer that can make sentences. We tell this writer to pay attention to the signals from DeWave and use them as a guide to create sentences,” says Zhou.

Finally, the team trained both DeWave and the language model together to get even better at writing sentences based on the EEG data.

With further refinement, the researchers predict that the system could revolutionise communication for people who have lost speech, such as those affected by a stroke, and could also have applications in robotics.

Craig Jin from the University of Sydney says he is impressed with the work by Lin’s team. “It’s excellent progress,” he says.

“People have been wanting to turn EEG into text for a long time and the team’s model is showing a remarkable amount of correctness. Several years ago the conversions from EEG to text were complete and utter nonsense.”