A scientific team has unveiled a new brain-computer interface that allows people with paralysis of the vocal apparatus and body to transmit speech and upper body signals simultaneously, in a first of its kind development.
Although brain-computer interfaces have previously enabled one of these two types of communication, this new system is the first to combine them, more closely mimicking natural expression.
Scientists at the University of California, San Francisco used machine learning to decode the unique brain activity supporting simultaneous speech and body language in three patients. Their brain interface successfully translated the thoughts of two participants into commands controlling the expressions of a full-body virtual avatar.
Dr. Edward Chang, a professor of neurosurgery at the University of California, said: "Conversation is about more than just spoken words. It's a multi-layered, dynamic process that involves the entire motor cortex. This practical demonstration shows us that it is possible for a brain-computer interface to restore some of that freedom and flexibility."
Those with amyotrophic lateral sclerosis (ALS) or who have suffered brainstem strokes often become paralyzed, which can impair or eliminate their verbal and nonverbal abilities. Eye-tracking techniques, the current standard of care, allow text-to-speech conversion for patients, but they are slow, offer limited forms of expression, and can be physically demanding.
Zhang and his colleagues were working on a more natural solution, having previously implanted a thin strip of sensors called an "electrocortical array" (ECoG) on the motor cortex of several patients. Computer models called "decoders" then translated their brain signals into computer commands that controlled a digital head and face.
In the new study, researchers implanted ECoG arrays into patients with varying degrees of vocal and bodily paralysis, but this time they aimed to enable upper body expression by symbolically linking participants to a whole body. The researchers collected data as participants attempted to pronounce specific phrases or perform common gestures such as waving or giving a thumbs-up sign, either separately or simultaneously.
Although verbal and nonverbal communication were facilitated separately in the past, attempting to combine them simultaneously seems to reduce speech abilities.
Scientists used to think that brain signals produced during speech and body signals together were simply an aggregation of the two signals, but Zhang and his colleagues discovered that multimodal communication is more complex than just combining them.
Despite some overlap, the picture drawn by the data from concurrent expressions was very different from that created by speech or signals separately.
They found that decoding devices were more successful at decoding signals from mixed expressions if they had been previously trained on data collected while participants performed speech and gestures simultaneously and without discrete separation.
With this strategy, the researchers and participants demonstrated that the brain-computer interface device can enable multifaceted and realistic expressions.
Dr. Deebara Tucci, director of the National Institute on Deafness and Other Communication Disorders at the National Institutes of Health, said: "These promising results give me hope that in the future, patients with severe paralysis will be able to regain the holistic nature of human communication."
Although the brain-computer interface used in this study was a wired system that connected implanted sensors to external processing units, Zhang explained that his team will soon test a fully implantable wireless version with better prospects for long-term application.
The study was published inthe journal Nature Neuroscience.
what is your opinion on this news ?do let us know in the comments on tha comment box
if you liked this news article please share on friends & family & don't forget to follow on website & social media
