speech Archives - Amazing Health Advances

New Brain-Computer Interface Allows Man with ALS to ‘Speak’ Again

The AHA! Team — Fri, 11 Oct 2024 08:20:41 +0000

UC Davis Health via Newswise – Technology developed by UC Davis Health restores interpersonal communication A new brain-computer interface (BCI) developed at UC Davis Health translates brain signals into speech with up to 97% accuracy — the most accurate system of its kind. The researchers implanted sensors in the brain of a man with severely impaired speech due to amyotrophic lateral sclerosis (ALS). The man was able to communicate his intended speech within minutes of activating the system. A study about this work was published today in the New England Journal of Medicine. ALS, also known as Lou Gehrig’s disease, affects the nerve cells that control movement throughout the body. The disease leads to a gradual loss of the ability to stand, walk and use one’s hands. It can also cause a person to lose control of the muscles used to speak, leading to a loss of understandable speech. The new technology is being developed to restore communication for people who can’t speak due to paralysis or neurological conditions like ALS. It can interpret brain signals when the user tries to speak and turns them into text that is ‘spoken’ aloud by the computer. “Our BCI technology helped a man with paralysis to communicate with friends, families and caregivers,” said UC Davis neurosurgeon David Brandman. “Our paper demonstrates the most accurate speech neuroprosthesis (device) ever reported.” Brandman is the co-principal investigator and co-senior author of this study. He is an assistant professor in the UC Davis Department of Neurological Surgery and co-director of the UC Davis Neuroprosthetics Lab. The new BCI breaks the communication barrier When someone tries to speak, the new BCI device transforms their brain activity into text on a computer screen. The computer can then read the text out loud. To develop the system, the team enrolled Casey Harrell, a 45-year-old man with ALS, in the BrainGate clinical trial. At the time of his enrollment, Harrell had weakness in his arms and legs (tetraparesis). His speech was very hard to understand (dysarthria) and required others to help interpret for him. In July 2023, Brandman implanted the investigational BCI device. He placed four microelectrode arrays into the left precentral gyrus, a brain region responsible for coordinating speech. The arrays are designed to record the brain activity from 256 cortical electrodes. “We’re really detecting their attempt to move their muscles and talk,” explained neuroscientist Sergey Stavisky. Stavisky is an assistant professor in the Department of Neurological Surgery. He is the co-director of the UC Davis Neuroprosthetics Lab and co-principal investigator of the study. “We are recording from the part of the brain that’s trying to send these commands to the muscles. And we are basically listening into that, and we’re translating those patterns of brain activity into a phoneme — like a syllable or the unit of speech — and then the words they’re trying to say.” Faster training, better results Despite recent advances in BCI technology, efforts to enable communication have been slow and prone to errors. This is because the machine-learning programs that interpreted brain signals required a large amount of time and data to perform. “Previous speech BCI systems had frequent word errors. This made it difficult for the user to be understood consistently and was a barrier to communication,” Brandman explained. “Our objective was to develop a system that empowered someone to be understood whenever they wanted to speak.” Harrell used the system in both prompted and spontaneous conversational settings. In both cases, speech decoding happened in real time, with continuous system updates to keep it working accurately. The decoded words were shown on a screen. Amazingly, they were read aloud in a voice that sounded like Harrell’s before he had ALS. The voice was composed using software trained with existing audio samples of his pre-ALS voice. At the first speech data training session, the system took 30 minutes to achieve 99.6% word accuracy with a 50-word vocabulary. “The first time we tried the system, he cried with joy as the words he was trying to say correctly appeared on-screen. We all did,” Stavisky said. In the second session, the size of the potential vocabulary increased to 125,000 words. With just an additional 1.4 hours of training data, the BCI achieved a 90.2% word accuracy with this greatly expanded vocabulary. After continued data collection, the BCI has maintained 97.5% accuracy. “At this point, we can decode what Casey is trying to say correctly about 97% of the time, which is better than many commercially available smartphone applications that try to interpret a person’s voice,” Brandman said. “This technology is transformative because it provides hope for people who want to speak but can’t. I hope that technology like this speech BCI will help future patients speak with their family and friends.” The study reports on 84 data collection sessions over 32 weeks. In total, Harrell used the speech BCI in self-paced conversations for over 248 hours to communicate in person and over video chat. “Not being able to communicate is so frustrating and demoralizing. It is like you are trapped,” Harrell said. “Something like this technology will help people back into life and society.” “It has been immensely rewarding to see Casey regain his ability to speak with his family and friends through this technology,” said the study’s lead author, Nicholas Card. Card is a postdoctoral scholar in the UC Davis Department of Neurological Surgery. “Casey and our other BrainGate participants are truly extraordinary. They deserve tremendous credit for joining these early clinical trials. They do this not because they’re hoping to gain any personal benefit, but to help us develop a system that will restore communication and mobility for other people with paralysis,” said co-author and BrainGate trial sponsor-investigator Leigh Hochberg. Hochberg is a neurologist and neuroscientist at Massachusetts General Hospital, Brown University and the VA Providence Healthcare System. Brandman is the site-responsible principal investigator of the BrainGate2 clinical trial. The trial is enrolling participants. To learn more about the study, visit braingate.org or contact braingate@ucdavis.edu. A complete list of coauthors and funders is available in the article. Caution: Investigational device. Limited by Federal law to investigational use. To read the original article click here.

The post New Brain-Computer Interface Allows Man with ALS to ‘Speak’ Again appeared first on Amazing Health Advances.

How the Brain Detects the Rhythms of Speech

AHA Publisher — Sun, 24 Nov 2019 08:00:05 +0000

University of California – San Francisco via EurekAlert – “What I find most exciting about this work is that it shows a simple neural coding principle for the sense of rhythm that is absolutely fundamental to how our brains process speech…” Neuroscientists at UC San Francisco have discovered how the listening brain scans speech to break it down into syllables. The findings provide for the first time a neural basis for the fundamental atoms of language and insights into our perception of the rhythmic poetry of speech. For decades, speech neuroscientists have looked for evidence that neurons in auditory brain areas use fluctuations in speech volume to identify the beginnings and ends of syllables — like a lin-guis-tics pro-fes-sor di-a-gram-ming a sen-tence. So far, these efforts have met with little luck. In the new study, published November 20, 2019 in Science Advances, UCSF scientists discovered that the brain instead responds to a marker of vocal stress in the middle of each syllable — more like a poet scanning the sonnets of Shakespeare (Shàll Í còmpáre thèe tó à súmmèrs dáy?). The researchers showed that this signal — in an area of speech cortex called the middle superior temporal gyrus (mSTG) — is specifically based on the rising volume at the start of each vowel sound, which is a universal feature of human languages. Notably, the authors say, this simple syllabic marker could also provide the brain with direct information about patterns of stress, timing, and rhythm that are so central to conveying meaning and emotional context in English and many other languages. “What I find most exciting about this work is that it shows a simple neural coding principle for the sense of rhythm that is absolutely fundamental to how our brains process speech,” said neuroscientist Yulia Oganian, PhD, who led the new research. “Could this explain why humans are so sensitive to the sequence of stressed and unstressed syllables that make up spoken poetry, or even oral storytelling?” Oganian is a postdoctoral researcher in the lab of UCSF Health neurosurgeon Eddie Chang, MD, PhD, Bowes Biomedical Investigator at UCSF, member of the UCSF Weill Institute for Neurosciences, and a Howard Hughes Medical Institute (HHMI) Faculty Scholar, whose research laboratory studies the neural basis of human speech, movement, and emotion. “What really excites me is that we now understand how a simple sound cue, the rapid increase in loudness that happens at the onset of vowels, serves as a critical landmark for speech because it tells a listener when a syllable occurs and whether it is stressed. This is a rather central discovery about how the brain extracts syllable units from speech,” said Chang. The study involved volunteers from the UCSF Epilepsy Center who temporarily had post-it-note-sized arrays of electrodes placed on the surface of their brains for one to two weeks as part of standard preparation for neurosurgery. These brain recordings allow neurosurgeons like Chang to map out how to remove the brain tissue that causes patients’ seizures without damaging important nearby brain regions, but also allow scientists in Chang’s neuroscience research lab to ask questions about human brain function that are impossible to address any other way. Oganian recruited 11 volunteers whose seizure-mapping electrodes happened to overlap with areas of the brain involved in speech processing and who were happy to participate in a research study during their down-time in the hospital. She played each participant a selection of speech recordings from a variety of different speakers while recording patterns of brain activity in their auditory speech centers, then analyzed the data to identify neural patterns reflecting the syllabic structure of what they had heard. The data quickly revealed that mSTG activity contained a discrete marker of individual syllables — contradicting the dominant model in the field that had proposed that the brain sets up a continuous metronome-like oscillator to extract syllable boundaries from fluctuations in speech volume. But exactly what aspects of speech were these discrete syllable markers in the neural data responding to? To make it possible to identify what features of the audio recordings were driving the new-found syllable markers, Oganian asked four of her research volunteers to listen to recorded speech that was slowed down four-fold. These ultra-slow speech recordings let Oganian see that the syllable signals were occurring consistently at the moment of rising stress at the start of each vowel sound (e.g. as ‘b’ turns to ‘a’ in the syllable ‘ba’), and not at the peak of each syllable as other scientists had theorized. The syllabic marker Oganian discovered in the mSTG also varied with the emphasis the speaker placed on a particular syllable. This suggested that this first stage of speech processing simultaneously allows the brain to split speech into syllabic units and also to track the patterns of stress that are critical for meaning in English and many other languages (e.g. “computer console” vs. “console a friend”; “Did I do that?” vs. “Did I do that?”). The syllabic signal also provides a simple metronome for the brain to track the rhythm and speed of speech. “Some people speak fast; others speak slow. People change how quickly they speak when they are excited or sad. The brain needs to be able to adjust to that,” Oganian said. “By marking whenever a new syllable is occurring, this signal acts as an internal pacemaker within the speech signal itself.” The researchers are continuing to study how brain signals in the mSTG are interpreted to enable the brain to process speech rhythmicity and meaning. They also hope to explore how the brain’s interpretation of these signals varies in languages other than English that put more or less emphasis on the stress patterns of speech. To read the original article click here.

The post How the Brain Detects the Rhythms of Speech appeared first on Amazing Health Advances.