The KNee OsteoArthritis Prediction (KNOAP2020) challenge: An image analysis challenge to predict incident symptomatic radiographic knee osteoarthritis from MRI and X-ray images

J. Hirvasniemi*, J. Runhaar, R. A. van der Heijden, M. Zokaeinikoo, M. Yang, X. Li, J. Tan, H. R. Rajamohan, Y. Zhou, C. M. Deniz, F. Caliva, C. Iriondo, J. J. Lee, F. Liu, A. M. Martinez, N. Namiri, V. Pedoia, E. Panfilov, N. Bayramoglu, H. H. NguyenM. T. Nieminen, S. Saarakkala, A. Tiulpin, E. Lin, A. Li, V. Li, E. B. Dam, A. S. Chaudhari, R. Kijowski, S. Bierma-Zeinstra, E. H.G. Oei, S. Klein

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

1 Citation (Scopus)
2 Downloads (Pure)

Abstract

Objectives: The KNee OsteoArthritis Prediction (KNOAP2020) challenge was organized to objectively compare methods for the prediction of incident symptomatic radiographic knee osteoarthritis within 78 months on a test set with blinded ground truth. Design: The challenge participants were free to use any available data sources to train their models. A test set of 423 knees from the Prevention of Knee Osteoarthritis in Overweight Females (PROOF) study consisting of magnetic resonance imaging (MRI) and X-ray image data along with clinical risk factors at baseline was made available to all challenge participants. The ground truth outcomes, i.e., which knees developed incident symptomatic radiographic knee osteoarthritis (according to the combined ACR criteria) within 78 months, were not provided to the participants. To assess the performance of the submitted models, we used the area under the receiver operating characteristic curve (ROCAUC) and balanced accuracy (BACC). Results: Seven teams submitted 23 entries in total. A majority of the algorithms were trained on data from the Osteoarthritis Initiative. The model with the highest ROCAUC (0.64 (95% confidence interval (CI): 0.57–0.70)) used deep learning to extract information from X-ray images combined with clinical variables. The model with the highest BACC (0.59 (95% CI: 0.52–0.65)) ensembled three different models that used automatically extracted X-ray and MRI features along with clinical variables. Conclusion: The KNOAP2020 challenge established a benchmark for predicting incident symptomatic radiographic knee osteoarthritis. Accurate prediction of incident symptomatic radiographic knee osteoarthritis is a complex and still unsolved problem requiring additional investigation.

Original languageEnglish
Pages (from-to)115-125
Number of pages11
JournalOsteoarthritis and Cartilage
Volume31
Issue number1
Early online date12 Oct 2022
DOIs
Publication statusPublished - Jan 2023

Bibliographical note

Acknowledgements
ReumaNederland is acknowledged for sponsoring the prize for the challenge. Study supported in part by National Institutes of Health (R01AR074453).

Publisher Copyright: © 2022 The Author(s)

Fingerprint

Dive into the research topics of 'The KNee OsteoArthritis Prediction (KNOAP2020) challenge: An image analysis challenge to predict incident symptomatic radiographic knee osteoarthritis from MRI and X-ray images'. Together they form a unique fingerprint.

Cite this