The reporting of neuropsychiatric symptoms in electronic health records of individuals with Alzheimer’s disease: a natural language processing study

Willem S. Eikelboom*, Ellen H. Singleton, Esther van den Berg, Casper de Boer, Michiel Coesmans, Jeannette A. Goudzwaard, Everard G.B. Vijverberg, Michel Pan, Cornalijn Gouw, Merel O. Mol, Freek Gillissen, Jay L.P. Fieldhouse, Yolande A.L. Pijnenburg, Wiesje M. van der Flier, John C. van Swieten, Rik Ossenkoppele, Jan A. Kors, Janne M. Papma

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

1 Citation (Scopus)
19 Downloads (Pure)


Background: Neuropsychiatric symptoms (NPS) are prevalent in the early clinical stages of Alzheimer’s disease (AD) according to proxy-based instruments. Little is known about which NPS clinicians report and whether their judgment aligns with proxy-based instruments. We used natural language processing (NLP) to classify NPS in electronic health records (EHRs) to estimate the reporting of NPS in symptomatic AD at the memory clinic according to clinicians. Next, we compared NPS as reported in EHRs and NPS reported by caregivers on the Neuropsychiatric Inventory (NPI). Methods: Two academic memory clinic cohorts were used: the Amsterdam UMC (n = 3001) and the Erasmus MC (n = 646). Patients included in these cohorts had MCI, AD dementia, or mixed AD/VaD dementia. Ten trained clinicians annotated 13 types of NPS in a randomly selected training set of n = 500 EHRs from the Amsterdam UMC cohort and in a test set of n = 250 EHRs from the Erasmus MC cohort. For each NPS, a generalized linear classifier was trained and internally and externally validated. Prevalence estimates of NPS were adjusted for the imperfect sensitivity and specificity of each classifier. Intra-individual comparison of the NPS classified in EHRs and NPS reported on the NPI were conducted in a subsample (59%). Results: Internal validation performance of the classifiers was excellent (AUC range: 0.81–0.91), but external validation performance decreased (AUC range: 0.51–0.93). NPS were prevalent in EHRs from the Amsterdam UMC, especially apathy (adjusted prevalence = 69.4%), anxiety (adjusted prevalence = 53.7%), aberrant motor behavior (adjusted prevalence = 47.5%), irritability (adjusted prevalence = 42.6%), and depression (adjusted prevalence = 38.5%). The ranking of NPS was similar for EHRs from the Erasmus MC, although not all classifiers obtained valid prevalence estimates due to low specificity. In both cohorts, there was minimal agreement between NPS classified in the EHRs and NPS reported on the NPI (all kappa coefficients < 0.28), with substantially more reports of NPS in EHRs than on NPI assessments. Conclusions: NLP classifiers performed well in detecting a wide range of NPS in EHRs of patients with symptomatic AD visiting the memory clinic and showed that clinicians frequently reported NPS in these EHRs. Clinicians generally reported more NPS in EHRs than caregivers reported on the NPI.

Original languageEnglish
Article number94
JournalAlzheimer's Research and Therapy
Issue number1
Publication statusPublished - 12 May 2023

Bibliographical note

Funding Information:
JMP and RO were supported by an Alzheimer Nederland and Memorabel ZonMw Grant 733050823 (Deltaplan Dementie). Research of the Amsterdam UMC Alzheimer Centre is part of the neurodegeneration research program of the Neuroscience Campus Amsterdam. The Amsterdam UMC Alzheimer Center is supported by Alzheimer Nederland and Stichting VUmc Fonds. WMvdF holds the Pasman chair.

Publisher Copyright:
© 2023, The Author(s).


Dive into the research topics of 'The reporting of neuropsychiatric symptoms in electronic health records of individuals with Alzheimer’s disease: a natural language processing study'. Together they form a unique fingerprint.

Cite this