Neuroscientists Decode Brain Speech Signals into Written Text

Neuroscientists decode brain speech signals into written text

“Doctors have turned the brain signals for speech into written sentences in a research project that aims to transform how patients with severe disabilities communicate in the future. The breakthrough is the first to demonstrate how the intention to say specific words can be extracted from brain activity and converted into text rapidly enough to keep pace with natural conversation. In its current form, the brain-reading software works only for certain sentences it has been trained on, but scientists believe it is a stepping stone towards a more powerful system that can decode in real time the words a person intends to say. Doctors at the University of California in San Francisco took on the challenge in the hope of creating a product that allows paralyzed people to communicate more fluidly than using existing devices that pick up eye movements and muscle twitches to control a virtual keyboard. The work, funded by Facebook, was possible thanks to three epilepsy patients who were about to have neurosurgery for their condition. Before their operations went ahead, all three had a small patch of tiny electrodes placed directly on the brain for at least a week to map the origins of their seizures. During their stay in hospital, the patients, all of whom could speak normally, agreed to take part in Chang’s research. He used the electrodes to record brain activity while each patient was asked nine set questions and asked to read a list of 24 potential responses. With the recordings in hand, Chang and his team built computer models that learned to match particular patterns of brain activity to the questions the patients heard and the answers they spoke. Once trained, the software could identify almost instantly, and from brain signals alone, what question a patient heard and what response they gave, with an accuracy of 76% and 61% respectively. While the work is still in its infancy, Winston Chiong, a neuroethicist at UCSF who was not involved in the latest study, said it was important to debate the ethical issues such systems might raise in the future. For example, could a ‘speech neuroprosthesis’ unintentionally reveal people’s most private thoughts? Chang said that decoding what someone was openly trying to say was hard enough, and that extracting their inner thoughts was virtually impossible. ‘I have no interest in developing a technology to find out what people are thinking, even if it were possible,’ he said. ‘But if someone wants to communicate and can’t, I think we have a responsibility as scientists and clinicians to restore that most fundamental human ability.’”

“Neuroscientists,” The Guardian, Jul. 30, 2019

---