Artificial Intelligence Applied for Identifying Acute Stroke Lesions
According to a study published online in the journal Neurology on March 27th, large language models (LLMs), such as the pre-trained generative transformer GPTs, have shown potential in localizing acute stroke lesions. The study, conducted by Dr. Jung-Hyun Lee and his team at the State University of New York Downstate Health Sciences University, evaluated the accuracy of the GPT-4 model in lesion localization based on clinical presentations.
In the study, GPT-4 received the medical history and neurologic examination (H&P) information of published cases of acute stroke and used zero-shot chaining and text classification methods for clinical reasoning, ultimately making predictions about "single or multiple lesions," "sides," and "brain regions." The research team compared the output results of GPT-4 with imaging-based localization in 46 cases through three independent comparative experiments.
The results of the study showed that GPT-4 successfully generated accurate neuroanatomical localization and detailed clinical reasoning based on the processed H&P raw text. In terms of performance analysis, the specificity, sensitivity, precision, and F1 score for sides were 0.87, 0.74, 0.75, and 0.74, respectively, while the metrics for brain regions were 0.94, 0.85, 0.84, and 0.85, respectively. Within the brain regions, except for the cerebellum, the classification labels of other areas demonstrated high accuracy, which remained consistent across the three experiments.
The main causes of errors include external factors and internal factors. External factors refer to insufficient information in the published cases, while internal factors are related to logical failures or insufficient knowledge base of the model.
The authors pointed out that in the future, LLMs that are fine-tuned and trained for neurologic diseases and neuroanatomy in clinical practice are expected to provide strong support for healthcare providers with limited training in neuroanatomical localization and diagnosis of neurological diseases. This finding indicates the broad application prospects of large language models in the medical field.