The neuroengineers of Columbia have discovered a system that translates thoughts into intelligible and recognizable speech. This technology can reconstruct the words a person hears with exceptional clarity.
The reports were published in Scientific Reports on 29 January 2019.
Researchers have claimed that this system will be helpful in regaining the ability to communicate for those who can not speak or suffering from Amyotrophic Lateral Sclerosis (ALS).
According to the research when people speak or even think about speaking, telltale patterns of activities appear in their brain. Similarly, when we listen to someone or at least imagine listening, a distinct pattern is formed. Experts are trying to recode and decode this pattern.
Dr. Mesgarani and his team with the help of first-author Hassan Akbari’s paper, developed vocoder, a computer algorithm that will synthesize the speech after being trained on recordings of people talking.
The vocoder will not only make sure that our thoughts are not left hidden inside the brain but also be verbally translated at person’s will.
The experiment was done on epilepsy patients, and those were asked to recite the digits between 0 to 9. The vocoder produces the sound in response to those signals which are analyzed and cleaned up by neural networks.
According to the Dr. Nima Mesgarani, they have used the same technology used by Amazon Echo and Apple Siri to give verbal responses to the questions.
Dr. Mesgarani said, “We found that people could understand and repeat the sounds about 75% of the time, which is well above and beyond any previous attempts. The sensitive vocoder and powerful neural networks represented the sounds the patients had originally listened to with surprising accuracy”.
Dr. Mesgarani claims that if a person who has lost his ability to speak whether, by injury or disease, the vocoder is the renewed chance for them to communicate.
In order to train vocoder to interpret brain activity, Dr. Mesgarani teamed up with Ashesh Dinesh Mehta, a neurosurgeon at Northwell Health Physician Partners Neurosurgeon Institute. The team is planning to work on more complicated words and sentence structures.
According to scientists and researchers, this device would be a game changer. As it is based on the computer algorithm that transmits the brain signals into audio recording, this can help millions who are incapable of speaking by birth or by disease.