Speech and Language Foundation Models for Accurate and Interpretable Alzheimer’s Dementia Recognition

Alzheimer's Disease (AD) is the leading cause of dementia worldwide, affecting over 57 million people. This number is projected to double every decade. Early diagnosis is critical for intervention, yet current diagnostic methods remain invasive, expensive, and inaccessible. Subtle linguistic and prosodic changes often emerge in the earliest stages of AD, offering a promising basis for non-invasive and cost-effective diagnostic tools. This work explores using speech and language foundation models to improve accuracy and interpretability of AD detection. Using the ADReSS 2020 dataset, we fine-tune transformer models on manual and Whisper ASR transcripts. Through prompt-based fine-tuning and ensemble strategies, we achieve state-of-the-art classification accuracy of 95.8% on manual transcripts and 89.6% on ASR transcripts. These results show that reliable models can be built without manual transcriptions, supporting broader deployment. This work further enhances model transparency, essential for clinical adoption. We introduce a Mixture of Experts (MoE) attention layer that routes input to specialised “experts,” each learning to focus on different linguistic and prosodic features, disentangling subtle speech patterns associated with AD. By examining token-level expert outputs, we can identify prediction-driving features, offering fine-grained interpretability. Our findings show that experts can be regularised to learn distinct speech cues separating AD from controls, and that the model’s decision-making process is traced back to specific tokens in participant transcripts. This work demonstrates that deep learning models can be designed with built-in interpretability, countering the "black-box" challenge limiting trust in AI-driven diagnostics. Challenges remain around data limitations, domain generalisation, and integration into clinical workflows.