For starters, scientists have created a system that translates thoughts into understandable, recognizable speech, an advance that could help people who cannot speak regain their ability to communicate with the outside world.
By monitoring someone's brain activity, technology developed by researchers at Columbia University in the US can reconstruct the words a person hears with unprecedented clarity.
The discovery, which harnesses the power of speech synthesizers and artificial intelligence, could lead to new ways for computers to communicate directly with the brain.
It also lays the groundwork to help people who cannot speak, such as people living with amyotrophic lateral sclerosis (ALS) or recovering from a stroke, regain their ability to communicate with the outside world, the researchers said.
"Our voices help us communicate with our friends, family and the world around us, which is why losing our voice through injury or illness is so devastating," said Nima Mesgarani, from Columbia University in the US.
"With today's study, we may have a way to fix all of that. We've shown that with the right technology, the thoughts of these people could be decoded and understood by any listener," said Mesgarani, the lead researcher on the published study.
Decades of research have shown that when people speak – or imagine – warning patterns of activity appear in their brains. Distinct patterns of signals also appear when we hear someone speak or imagine them listening.
Experts, trying to capture and decode these patterns, see a future in which thoughts need not remain hidden within the brain – but instead could be translated into spoken speech at will.
However, perfecting this feature has proven challenging. Early attempts at decoding brain signals by researchers focused on simple computer models that analyzed spectrograms, which are visual representations of sound frequencies.
However, because this approach failed to produce anything resembling intelligible speech, the team turned instead to a phonocoder, a computer algorithm that can synthesize speech after being trained on recordings of people speaking.
"This is the same technology used by Amazon Echo and Apple Siri to provide verbal answers to our questions," Mesgarani said.
The researchers plan to test more complex words and sentences next, and want to run the same tests on brain signals emitted when a person speaks or imagines speaking.
Ultimately, they hope their system could be part of an implant, similar to those worn by some epilepsy patients, that translates the user's thoughts into words.
