Cross-cohort generalizability of deep and conventional machine learning for MRI-based diagnosis and prediction of Alzheimer's disease

for the Alzheimer's Disease Neuroimaging Initiative

Research output: Contribution to journalArticleAcademicpeer-review

54 Citations (Scopus)
64 Downloads (Pure)

Abstract

This work validates the generalizability of MRI-based classification of Alzheimer's disease (AD) patients and controls (CN) to an external data set and to the task of prediction of conversion to AD in individuals with mild cognitive impairment (MCI). We used a conventional support vector machine (SVM) and a deep convolutional neural network (CNN) approach based on structural MRI scans that underwent either minimal pre-processing or more extensive pre-processing into modulated gray matter (GM) maps. Classifiers were optimized and evaluated using cross-validation in the Alzheimer's Disease Neuroimaging Initiative (ADNI; 334 AD, 520 CN). Trained classifiers were subsequently applied to predict conversion to AD in ADNI MCI patients (231 converters, 628 non-converters) and in the independent Health-RI Parelsnoer Neurodegenerative Diseases Biobank data set. From this multi-center study representing a tertiary memory clinic population, we included 199 AD patients, 139 participants with subjective cognitive decline, 48 MCI patients converting to dementia, and 91 MCI patients who did not convert to dementia. AD-CN classification based on modulated GM maps resulted in a similar area-under-the-curve (AUC) for SVM (0.940; 95%CI: 0.924–0.955) and CNN (0.933; 95%CI: 0.918–0.948). Application to conversion prediction in MCI yielded significantly higher performance for SVM (AUC = 0.756; 95%CI: 0.720-0.788) than for CNN (AUC = 0.742; 95%CI: 0.709-0.776) (p<0.01 for McNemar's test). In external validation, performance was slightly decreased. For AD-CN, it again gave similar AUCs for SVM (0.896; 95%CI: 0.855–0.932) and CNN (0.876; 95%CI: 0.836–0.913). For prediction in MCI, performances decreased for both SVM (AUC = 0.665; 95%CI: 0.576-0.760) and CNN (AUC = 0.702; 95%CI: 0.624-0.786). Both with SVM and CNN, classification based on modulated GM maps significantly outperformed classification based on minimally processed images (p=0.01). Deep and conventional classifiers performed equally well for AD classification and their performance decreased only slightly when applied to the external cohort. We expect that this work on external validation contributes towards translation of machine learning to clinical practice.

Original languageEnglish
Article number102712
JournalNeuroImage: Clinical
Volume31
Early online date4 Jun 2021
DOIs
Publication statusPublished - 2021

Bibliographical note

Funding Information:
Data collection and sharing was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12–2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health ( www.fnih.org ). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

Funding Information:
The authors are grateful to SURFsara for the processing time on the Dutch national supercomputer ( www.surfsara.nl/systems/cartesius ). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan V GPU used for this research.

Funding Information:
The authors would like to thank Judith Manni?n, Ilya de Groot, and Nienke Aaftink for their effort in data preparation. The authors are grateful to SURFsara for the processing time on the Dutch national supercomputer (www.surfsara.nl/systems/cartesius). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan V GPU used for this research. E.E. Bron acknowledges support from Dutch Heart Foundation (PPP Allowance, 2018B011) and the Netherlands CardioVascular Research Initiative (Heart-Brain Connection: CVON2012-06, CVON2018-28). E.E. Bron and W.J. Niessen are supported by Medical Delta Diagnostics 3.0: Dementia and Stroke. V. Venkatraghavan and W.J. Niessen acknowledge funding from the Health Holland LSH-TKI project Beyond (LSHM18049). This work is part of the EuroPOND initiative, which is funded by the European Union's Horizon 2020 research and innovation programme under grant agreement No. 666992. The work described in this study was carried out in the context of the Health-RI Parelsnoer Neurodegenerative Diseases Biobank. Parelsnoer biobanks are part of and funded by the Dutch Federation of University Medical Centers and has received initial funding from the Dutch Government (2007?2011). Data collection and sharing was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12?2-0012). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, and through generous contributions from the following: AbbVie, Alzheimer's Association; Alzheimer's Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc.; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; EuroImmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merck & Co. Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer's Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

Funding Information:
The work described in this study was carried out in the context of the Health-RI Parelsnoer Neurodegenerative Diseases Biobank. Parelsnoer biobanks are part of and funded by the Dutch Federation of University Medical Centers and has received initial funding from the Dutch Government (2007–2011).

Publisher Copyright: © 2021 The Author(s)

Fingerprint

Dive into the research topics of 'Cross-cohort generalizability of deep and conventional machine learning for MRI-based diagnosis and prediction of Alzheimer's disease'. Together they form a unique fingerprint.

Cite this