Where is VALDO? : VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021

Carole H. Sudre; Kimberlin Van Wijnen; Florian Dubost; the ALFA study; Hieab Adams; David Atkinson; Frederik Barkhof; Mahlet A. Birhanu; Esther E. Bron; Robin Camarasa; Nish Chaturvedi; Yuan Chen; Zihao Chen; Shuai Chen; Qi Dou; Tavia Evans; Ivan Ezhov; Haojun Gao; Marta Girones Sanguesa; Juan Domingo Gispert; Beatriz Gomez Anson; Alun D. Hughes; M. Arfan Ikram; Silvia Ingala; H. Rolf Jaeger; Florian Kofler; Hugo J. Kuijf; Denis Kutnar; Minho Lee; Bo Li; Luigi Lorenzini; Bjoern Menze; Jose Luis Molinuevo; Yiwei Pan; Elodie Puybareau; Rafael Rehwald; Ruisheng Su; Pengcheng Shi; Lorna Smith; Therese Tillin; Guillaume Tochon; Helene Urien; Bas H. M. van der Velden; Isabelle F. van der Velpen; Benedikt Wiestler; Frank J. Wolters; Pinar Yilmaz; Marius de Groot; Meike W. Vernooij; Marleen de Bruijne

doi:10.1016/j.media.2023.103029

Where is VALDO? : VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021

Carole H. Sudre^*, Kimberlin Van Wijnen, Florian Dubost, the ALFA study, Hieab Adams, David Atkinson, Frederik Barkhof, Mahlet A. Birhanu, Esther E. Bron, Robin Camarasa, Nish Chaturvedi, Yuan Chen, Zihao Chen, Shuai Chen, Qi Dou, Tavia Evans, Ivan Ezhov, Haojun Gao, Marta Girones Sanguesa, Juan Domingo GispertBeatriz Gomez Anson, Alun D. Hughes, M. Arfan Ikram, Silvia Ingala, H. Rolf Jaeger, Florian Kofler, Hugo J. Kuijf, Denis Kutnar, Minho Lee, Bo Li, Luigi Lorenzini, Bjoern Menze, Jose Luis Molinuevo, Yiwei Pan, Elodie Puybareau, Rafael Rehwald, Ruisheng Su, Pengcheng Shi, Lorna Smith, Therese Tillin, Guillaume Tochon, Helene Urien, Bas H. M. van der Velden, Isabelle F. van der Velpen, Benedikt Wiestler, Frank J. Wolters, Pinar Yilmaz, Marius de Groot, Meike W. Vernooij, Marleen de Bruijne

^*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

8 Citations (Scopus)

9 Downloads (Pure)

Abstract

Imaging markers of cerebral small vessel disease provide valuable information on brain health, but their manual assessment is time-consuming and hampered by substantial intra-and interrater variability. Automated rating may benefit biomedical research, as well as clinical assessment, but diagnostic reliability of existing algorithms is unknown. Here, we present the results of the VAscular Lesions DetectiOn and Segmentation (Where is VALDO?) challenge that was run as a satellite event at the international conference on Medical Image Computing and Computer Aided Intervention (MICCAI) 2021. This challenge aimed to promote the development of methods for automated detection and segmentation of small and sparse imaging markers of cerebral small vessel disease, namely enlarged perivascular spaces (EPVS) (Task 1), cerebral microbleeds (Task 2) and lacunes of presumed vascular origin (Task 3) while leveraging weak and noisy labels. Overall, 12 teams participated in the challenge proposing solutions for one or more tasks (4 for Task 1-EPVS, 9 for Task 2-Microbleeds and 6 for Task 3Lacunes). Multi-cohort data was used in both training and evaluation. Results showed a large variability in performance both across teams and across tasks, with promising results notably for Task 1-EPVS and Task 2-Microbleeds and not practically useful results yet for Task 3-Lacunes. It also highlighted the performance inconsistency across cases that may deter use at an individual level, while still proving useful at a population level.

Original language	English
Article number	103029
Number of pages	23
Journal	Medical Image Analysis
Volume	91
DOIs	https://doi.org/10.1016/j.media.2023.103029
Publication status	Published - Jan 2024

Bibliographical note

Publisher Copyright:
© 2023 The Author(s)

Access to Document

10.1016/j.media.2023.103029Licence: CC BY

1-s2.0-S136184152300289X-mainFinal published version, 5.21 MBLicence: CC BY

Cite this

Sudre, C. H., Van Wijnen, K., Dubost, F., the ALFA study, Adams, H., Atkinson, D., Barkhof, F., Birhanu, M. A., Bron, E. E., Camarasa, R., Chaturvedi, N., Chen, Y., Chen, Z., Chen, S., Dou, Q., Evans, T., Ezhov, I., Gao, H., Sanguesa, M. G., ... de Bruijne, M. (2024). Where is VALDO? : VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021. Medical Image Analysis, 91, Article 103029. https://doi.org/10.1016/j.media.2023.103029

@article{412c6be494174abab2e47b155a9c0b18,

title = "Where is VALDO? : VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021",

abstract = "Imaging markers of cerebral small vessel disease provide valuable information on brain health, but their manual assessment is time-consuming and hampered by substantial intra-and interrater variability. Automated rating may benefit biomedical research, as well as clinical assessment, but diagnostic reliability of existing algorithms is unknown. Here, we present the results of the VAscular Lesions DetectiOn and Segmentation (Where is VALDO?) challenge that was run as a satellite event at the international conference on Medical Image Computing and Computer Aided Intervention (MICCAI) 2021. This challenge aimed to promote the development of methods for automated detection and segmentation of small and sparse imaging markers of cerebral small vessel disease, namely enlarged perivascular spaces (EPVS) (Task 1), cerebral microbleeds (Task 2) and lacunes of presumed vascular origin (Task 3) while leveraging weak and noisy labels. Overall, 12 teams participated in the challenge proposing solutions for one or more tasks (4 for Task 1-EPVS, 9 for Task 2-Microbleeds and 6 for Task 3Lacunes). Multi-cohort data was used in both training and evaluation. Results showed a large variability in performance both across teams and across tasks, with promising results notably for Task 1-EPVS and Task 2-Microbleeds and not practically useful results yet for Task 3-Lacunes. It also highlighted the performance inconsistency across cases that may deter use at an individual level, while still proving useful at a population level.",

author = "Sudre, {Carole H.} and {Van Wijnen}, Kimberlin and Florian Dubost and {the ALFA study} and Hieab Adams and David Atkinson and Frederik Barkhof and Birhanu, {Mahlet A.} and Bron, {Esther E.} and Robin Camarasa and Nish Chaturvedi and Yuan Chen and Zihao Chen and Shuai Chen and Qi Dou and Tavia Evans and Ivan Ezhov and Haojun Gao and Sanguesa, {Marta Girones} and Gispert, {Juan Domingo} and Anson, {Beatriz Gomez} and Hughes, {Alun D.} and Ikram, {M. Arfan} and Silvia Ingala and Jaeger, {H. Rolf} and Florian Kofler and Kuijf, {Hugo J.} and Denis Kutnar and Minho Lee and Bo Li and Luigi Lorenzini and Bjoern Menze and Molinuevo, {Jose Luis} and Yiwei Pan and Elodie Puybareau and Rafael Rehwald and Ruisheng Su and Pengcheng Shi and Lorna Smith and Therese Tillin and Guillaume Tochon and Helene Urien and Velden, {Bas H. M. van der} and Velpen, {Isabelle F. van der} and Benedikt Wiestler and Wolters, {Frank J.} and Pinar Yilmaz and {de Groot}, Marius and Vernooij, {Meike W.} and {de Bruijne}, Marleen",

note = "Publisher Copyright: {\textcopyright} 2023 The Author(s)",

year = "2024",

month = jan,

doi = "10.1016/j.media.2023.103029",

language = "English",

volume = "91",

journal = "Medical Image Analysis",

issn = "1361-8415",

publisher = "Elsevier",

}

Sudre, CH, Van Wijnen, K, Dubost, F, the ALFA study, Adams, H, Atkinson, D, Barkhof, F, Birhanu, MA , Bron, EE , Camarasa, R, Chaturvedi, N, Chen, Y, Chen, Z, Chen, S, Dou, Q, Evans, T, Ezhov, I, Gao, H, Sanguesa, MG, Gispert, JD, Anson, BG, Hughes, AD, Ikram, MA, Ingala, S, Jaeger, HR, Kofler, F, Kuijf, HJ, Kutnar, D, Lee, M, Li, B, Lorenzini, L, Menze, B, Molinuevo, JL, Pan, Y, Puybareau, E, Rehwald, R, Su, R, Shi, P, Smith, L, Tillin, T, Tochon, G, Urien, H, Velden, BHMVD, Velpen, IFVD, Wiestler, B, Wolters, FJ, Yilmaz, P, de Groot, M, Vernooij, MW & de Bruijne, M 2024, 'Where is VALDO? : VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021', Medical Image Analysis, vol. 91, 103029. https://doi.org/10.1016/j.media.2023.103029

TY - JOUR

T1 - Where is VALDO?

T2 - VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021

AU - Sudre, Carole H.

AU - Van Wijnen, Kimberlin

AU - Dubost, Florian

AU - the ALFA study

AU - Adams, Hieab

AU - Atkinson, David

AU - Barkhof, Frederik

AU - Birhanu, Mahlet A.

AU - Bron, Esther E.

AU - Camarasa, Robin

AU - Chaturvedi, Nish

AU - Chen, Yuan

AU - Chen, Zihao

AU - Chen, Shuai

AU - Dou, Qi

AU - Evans, Tavia

AU - Ezhov, Ivan

AU - Gao, Haojun

AU - Sanguesa, Marta Girones

AU - Gispert, Juan Domingo

AU - Anson, Beatriz Gomez

AU - Hughes, Alun D.

AU - Ikram, M. Arfan

AU - Ingala, Silvia

AU - Jaeger, H. Rolf

AU - Kofler, Florian

AU - Kuijf, Hugo J.

AU - Kutnar, Denis

AU - Lee, Minho

AU - Li, Bo

AU - Lorenzini, Luigi

AU - Menze, Bjoern

AU - Molinuevo, Jose Luis

AU - Pan, Yiwei

AU - Puybareau, Elodie

AU - Rehwald, Rafael

AU - Su, Ruisheng

AU - Shi, Pengcheng

AU - Smith, Lorna

AU - Tillin, Therese

AU - Tochon, Guillaume

AU - Urien, Helene

AU - Velden, Bas H. M. van der

AU - Velpen, Isabelle F. van der

AU - Wiestler, Benedikt

AU - Wolters, Frank J.

AU - Yilmaz, Pinar

AU - de Groot, Marius

AU - Vernooij, Meike W.

AU - de Bruijne, Marleen

PY - 2024/1

Y1 - 2024/1

N2 - Imaging markers of cerebral small vessel disease provide valuable information on brain health, but their manual assessment is time-consuming and hampered by substantial intra-and interrater variability. Automated rating may benefit biomedical research, as well as clinical assessment, but diagnostic reliability of existing algorithms is unknown. Here, we present the results of the VAscular Lesions DetectiOn and Segmentation (Where is VALDO?) challenge that was run as a satellite event at the international conference on Medical Image Computing and Computer Aided Intervention (MICCAI) 2021. This challenge aimed to promote the development of methods for automated detection and segmentation of small and sparse imaging markers of cerebral small vessel disease, namely enlarged perivascular spaces (EPVS) (Task 1), cerebral microbleeds (Task 2) and lacunes of presumed vascular origin (Task 3) while leveraging weak and noisy labels. Overall, 12 teams participated in the challenge proposing solutions for one or more tasks (4 for Task 1-EPVS, 9 for Task 2-Microbleeds and 6 for Task 3Lacunes). Multi-cohort data was used in both training and evaluation. Results showed a large variability in performance both across teams and across tasks, with promising results notably for Task 1-EPVS and Task 2-Microbleeds and not practically useful results yet for Task 3-Lacunes. It also highlighted the performance inconsistency across cases that may deter use at an individual level, while still proving useful at a population level.

AB - Imaging markers of cerebral small vessel disease provide valuable information on brain health, but their manual assessment is time-consuming and hampered by substantial intra-and interrater variability. Automated rating may benefit biomedical research, as well as clinical assessment, but diagnostic reliability of existing algorithms is unknown. Here, we present the results of the VAscular Lesions DetectiOn and Segmentation (Where is VALDO?) challenge that was run as a satellite event at the international conference on Medical Image Computing and Computer Aided Intervention (MICCAI) 2021. This challenge aimed to promote the development of methods for automated detection and segmentation of small and sparse imaging markers of cerebral small vessel disease, namely enlarged perivascular spaces (EPVS) (Task 1), cerebral microbleeds (Task 2) and lacunes of presumed vascular origin (Task 3) while leveraging weak and noisy labels. Overall, 12 teams participated in the challenge proposing solutions for one or more tasks (4 for Task 1-EPVS, 9 for Task 2-Microbleeds and 6 for Task 3Lacunes). Multi-cohort data was used in both training and evaluation. Results showed a large variability in performance both across teams and across tasks, with promising results notably for Task 1-EPVS and Task 2-Microbleeds and not practically useful results yet for Task 3-Lacunes. It also highlighted the performance inconsistency across cases that may deter use at an individual level, while still proving useful at a population level.

UR - http://www.scopus.com/inward/record.url?scp=85177735915&partnerID=8YFLogxK

U2 - 10.1016/j.media.2023.103029

DO - 10.1016/j.media.2023.103029

M3 - Article

C2 - 37988921

SN - 1361-8415

VL - 91

JO - Medical Image Analysis

JF - Medical Image Analysis

M1 - 103029

ER -

Where is VALDO? : VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this