The artificial intelligence algorithms behind ChatGPT, a chatbot program that has gained attention for its ability to generate human written responses to some of the most creative queries, could one day help doctors detect Alzheimer’s disease early. Research from Drexel University’s School of Biomedical Engineering, Science and Health Systems recently demonstrated that the OpenAI GPT-3 program can identify spontaneous speech cues that are predictive of early dementia with an accuracy of 80%.
Reported in the journal PLOS Digital HealthDrexel’s study is the latest in a series of attempts to show the effectiveness of natural language processing programs for early prediction of Alzheimer’s disease, using current research suggesting that speech impairment may be an early indicator of neurodegenerative disorders.
Looking for an early sign
The current practice of diagnosing Alzheimer’s usually involves taking a medical history and a lengthy series of physical and neurological examinations and tests. While there is still no cure for the disease, early detection can give patients more options for therapy and support. Because speech impairment is a symptom in 60-80% of dementia patients, researchers have focused on programs that can pick up subtle cues, such as hesitancy, grammatical and pronunciation errors, and forgetting the meaning of words, as a quick test that can indicate whether the patient to undergo a complete examination.
“We know from current research that the cognitive effects of Alzheimer’s disease can manifest themselves in speech production,” said Hualou Liang, Ph.D., a professor at the Drexel School of Biomedical Engineering, Science and Health Systems and co-author of the study. “The most commonly used tests for early detection of Alzheimer’s disease look at acoustic features such as pauses, articulation and voice quality, in addition to cognition tests. But we believe that improvements in natural language processing programs offer another avenue for early detection of Alzheimer’s disease.”
A program that listens and learns
Officially the third generation of the General Pretrained Transformer (GPT) of OpenAI, GPT-3 uses a deep learning algorithm trained by processing vast amounts of information from the Internet, with a special focus on how words are used and how language is built. This training allows him to humanly respond to any task related to the language, from answering simple questions to writing poetry or essays.
GPT-3 is especially good at “zero data learning” – meaning it can answer questions that would normally require outside knowledge that wasn’t provided. For example, if you ask a program to write “Cliff’s Notes” over the text, it will usually require an explanation of what that means in summary. But GPT-3 has been trained enough to understand the reference value and adapt to the expected response.
“GPT3’s systematic approach to language analysis and production makes it a promising candidate for identifying subtle speech characteristics that can predict the onset of dementia,” said Felix Agbavor, a doctoral student at the School and lead author of the paper. “Training GPT-3 on a large set of interview data, some of which are about Alzheimer’s patients, will provide him with the information he needs to extract speech patterns that can then be used to identify markers in future patients.”
Searching for voice signals
The researchers tested their theory by training the program with a set of transcripts of part of a dataset of speech recordings compiled with support from the National Institutes of Health specifically to test the ability of natural language processing programs to predict dementia. The program captured significant characteristics of word usage, sentence structure, and text meaning to create what the researchers call “embedding,” a characteristic speech profile of Alzheimer’s disease.
They then used embedding to retrain the program into an Alzheimer’s screening machine. To test this, they asked the program to look at dozens of transcripts from the dataset and decide if each one was created by someone who developed Alzheimer’s.
By running the top two natural language processing programs at the same speed, the group found that GPT-3 performed better than both in terms of pinpointing Alzheimer’s examples, identifying non-Alzheimer’s examples, and with fewer missed cases than both programs.
The second test used GPT-3 text analysis to predict the scores of different patients from a dataset on a general test for predicting dementia severity called the Mini-Mental State Exam (MMSE).
The team then compared the accuracy of the GPT-3 prediction with that of an analysis that uses only acoustic features of the recordings, such as pauses, voice strength, and slurring, to predict the MMSE score. GPT-3 was found to be nearly 20% more accurate in predicting patients’ MMSE scores.
“Our results show that GPT-3-generated text embedding can be reliably used not only to identify people with Alzheimer’s disease among healthy individuals, but also to infer a subject’s cognitive test score based solely on speech data,” they wrote. . “We also show that text embedding is superior to the traditional acoustic feature approach and even competitive with finely tuned models. These results collectively suggest that GPT-3-based text embedding is a promising approach to assessing Alzheimer’s disease and may improve the early diagnosis of dementia.”
We continue the search
To build on these promising results, the researchers plan to develop a web-based application that can be used at home or in the doctor’s office as a pre-screening tool.
“Our proof-of-concept shows that it can be a simple, affordable, and sensitive enough tool for community-level testing,” Liang said. “This could be very useful for early screening and risk assessment before making a clinical diagnosis.”
PLOS Digital Health
Subject of study
Predicting dementia from spontaneous speech using large language models
Article publication date
December 22, 2022
Denial of responsibility: AAAS and EurekAlert! are not responsible for the accuracy of news reports posted on EurekAlert! facilitating institutions or to use any information through the EurekAlert system.