Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in computed tomography angiography

Hortense Kirisli; Michiel Schaap; Coert Metz; Anoeshka Dharampal; Bob Meijboom; Elina Papadopoulou; Admir Dedic; Koen Nieman; MA (Michiel) van der Graaf; MFL Meijs; MJ Cramer; A Broersen; S Cetin; A Eslami; L Florez-Valencia; KL Lor; B Matuszewski; I Melki; B Mohr; I Oksuz; Rahil Shahzad; C Wang; PH Kitslaar; G Unal; A Katouzian; M Orkisz; CM Chen; F Precioso; L Najman; S Masood; D Unay; L van Vliet; R Moreno; R Goldenberg; E Vucini; Gabriel Krestin; Wiro Niessen; Theo van Walsum

doi:10.1016/j.media.2013.05.007

Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in computed tomography angiography

Hortense Kirisli, Michiel Schaap, Coert Metz, Anoeshka Dharampal, Bob Meijboom, Elina Papadopoulou, Admir Dedic, Koen Nieman, MA (Michiel) van der Graaf, MFL Meijs, MJ Cramer, A Broersen, S Cetin, A Eslami, L Florez-Valencia, KL Lor, B Matuszewski, I Melki, B Mohr, I OksuzRahil Shahzad, C Wang, PH Kitslaar, G Unal, A Katouzian, M Orkisz, CM Chen, F Precioso, L Najman, S Masood, D Unay, L van Vliet, R Moreno, R Goldenberg, E Vucini, Gabriel Krestin, Wiro Niessen, Theo van Walsum

Research output: Contribution to journal › Article › Academic

144 Citations (Scopus)

Abstract

Though conventional coronary angiography (CCA) has been the standard of reference for diagnosing coronary artery disease in the past decades, computed tomography angiography (CIA) has rapidly emerged, and is nowadays widely used in clinical practice. Here, we introduce a standardized evaluation framework to reliably evaluate and compare the performance of the algorithms devised to detect and quantify the coronary artery stenoses, and to segment the coronary artery lumen in CIA data. The objective of this evaluation framework is to demonstrate the feasibility of dedicated algorithms to: (I) (semi-)automatically detect and quantify stenosis on CIA, in comparison with quantitative coronary angiography (QCA) and CIA consensus reading, and (2) (semi-)automatically segment the coronary lumen on CIA, in comparison with expert's manual annotation. A database consisting of 48 multicenter multivendor cardiac CIA datasets with corresponding reference standards are described and made available. The algorithms from 11 research groups were quantitatively evaluated and compared. The results show that (1) some of the current stenosis detection/quantification algorithms may be used for triage or as a second-reader in clinical practice, and that (2) automatic lumen segmentation is possible with a precision similar to that obtained by experts. The framework is open for new submissions through the website, at http://coronary.bigr.nl/stenoses/. (C) 2013 Elsevier B.V. All rights reserved.

Original language	Undefined/Unknown
Pages (from-to)	859-876
Number of pages	18
Journal	Medical Image Analysis
Volume	17
Issue number	8
DOIs	https://doi.org/10.1016/j.media.2013.05.007
Publication status	Published - 2013

Access to Document

10.1016/j.media.2013.05.007

http://hdl.handle.net/1765/41673

Cite this

Kirisli, H., Schaap, M., Metz, C., Dharampal, A., Meijboom, B., Papadopoulou, E., Dedic, A., Nieman, K., van der Graaf, MA., Meijs, MFL., Cramer, MJ., Broersen, A., Cetin, S., Eslami, A., Florez-Valencia, L., Lor, KL., Matuszewski, B., Melki, I., Mohr, B., ... van Walsum, T. (2013). Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in computed tomography angiography. Medical Image Analysis, 17(8), 859-876. https://doi.org/10.1016/j.media.2013.05.007

@article{a79227340d36456693f4211482ff6cb3,

title = "Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in computed tomography angiography",

abstract = "Though conventional coronary angiography (CCA) has been the standard of reference for diagnosing coronary artery disease in the past decades, computed tomography angiography (CIA) has rapidly emerged, and is nowadays widely used in clinical practice. Here, we introduce a standardized evaluation framework to reliably evaluate and compare the performance of the algorithms devised to detect and quantify the coronary artery stenoses, and to segment the coronary artery lumen in CIA data. The objective of this evaluation framework is to demonstrate the feasibility of dedicated algorithms to: (I) (semi-)automatically detect and quantify stenosis on CIA, in comparison with quantitative coronary angiography (QCA) and CIA consensus reading, and (2) (semi-)automatically segment the coronary lumen on CIA, in comparison with expert's manual annotation. A database consisting of 48 multicenter multivendor cardiac CIA datasets with corresponding reference standards are described and made available. The algorithms from 11 research groups were quantitatively evaluated and compared. The results show that (1) some of the current stenosis detection/quantification algorithms may be used for triage or as a second-reader in clinical practice, and that (2) automatic lumen segmentation is possible with a precision similar to that obtained by experts. The framework is open for new submissions through the website, at http://coronary.bigr.nl/stenoses/. (C) 2013 Elsevier B.V. All rights reserved.",

author = "Hortense Kirisli and Michiel Schaap and Coert Metz and Anoeshka Dharampal and Bob Meijboom and Elina Papadopoulou and Admir Dedic and Koen Nieman and {van der Graaf}, {MA (Michiel)} and MFL Meijs and MJ Cramer and A Broersen and S Cetin and A Eslami and L Florez-Valencia and KL Lor and B Matuszewski and I Melki and B Mohr and I Oksuz and Rahil Shahzad and C Wang and PH Kitslaar and G Unal and A Katouzian and M Orkisz and CM Chen and F Precioso and L Najman and S Masood and D Unay and {van Vliet}, L and R Moreno and R Goldenberg and E Vucini and Gabriel Krestin and Wiro Niessen and {van Walsum}, Theo",

year = "2013",

doi = "10.1016/j.media.2013.05.007",

language = "Undefined/Unknown",

volume = "17",

pages = "859--876",

journal = "Medical Image Analysis",

issn = "1361-8415",

publisher = "Elsevier",

number = "8",

}

Kirisli, H, Schaap, M, Metz, C, Dharampal, A, Meijboom, B, Papadopoulou, E, Dedic, A, Nieman, K, van der Graaf, MA, Meijs, MFL, Cramer, MJ, Broersen, A, Cetin, S, Eslami, A, Florez-Valencia, L, Lor, KL, Matuszewski, B, Melki, I, Mohr, B, Oksuz, I, Shahzad, R, Wang, C, Kitslaar, PH, Unal, G, Katouzian, A, Orkisz, M, Chen, CM, Precioso, F, Najman, L, Masood, S, Unay, D, van Vliet, L, Moreno, R, Goldenberg, R, Vucini, E, Krestin, G , Niessen, W & van Walsum, T 2013, 'Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in computed tomography angiography', Medical Image Analysis, vol. 17, no. 8, pp. 859-876. https://doi.org/10.1016/j.media.2013.05.007

TY - JOUR

T1 - Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in computed tomography angiography

AU - Kirisli, Hortense

AU - Schaap, Michiel

AU - Metz, Coert

AU - Dharampal, Anoeshka

AU - Meijboom, Bob

AU - Papadopoulou, Elina

AU - Dedic, Admir

AU - Nieman, Koen

AU - van der Graaf, MA (Michiel)

AU - Meijs, MFL

AU - Cramer, MJ

AU - Broersen, A

AU - Cetin, S

AU - Eslami, A

AU - Florez-Valencia, L

AU - Lor, KL

AU - Matuszewski, B

AU - Melki, I

AU - Mohr, B

AU - Oksuz, I

AU - Shahzad, Rahil

AU - Wang, C

AU - Kitslaar, PH

AU - Unal, G

AU - Katouzian, A

AU - Orkisz, M

AU - Chen, CM

AU - Precioso, F

AU - Najman, L

AU - Masood, S

AU - Unay, D

AU - van Vliet, L

AU - Moreno, R

AU - Goldenberg, R

AU - Vucini, E

AU - Krestin, Gabriel

AU - Niessen, Wiro

AU - van Walsum, Theo

PY - 2013

Y1 - 2013

N2 - Though conventional coronary angiography (CCA) has been the standard of reference for diagnosing coronary artery disease in the past decades, computed tomography angiography (CIA) has rapidly emerged, and is nowadays widely used in clinical practice. Here, we introduce a standardized evaluation framework to reliably evaluate and compare the performance of the algorithms devised to detect and quantify the coronary artery stenoses, and to segment the coronary artery lumen in CIA data. The objective of this evaluation framework is to demonstrate the feasibility of dedicated algorithms to: (I) (semi-)automatically detect and quantify stenosis on CIA, in comparison with quantitative coronary angiography (QCA) and CIA consensus reading, and (2) (semi-)automatically segment the coronary lumen on CIA, in comparison with expert's manual annotation. A database consisting of 48 multicenter multivendor cardiac CIA datasets with corresponding reference standards are described and made available. The algorithms from 11 research groups were quantitatively evaluated and compared. The results show that (1) some of the current stenosis detection/quantification algorithms may be used for triage or as a second-reader in clinical practice, and that (2) automatic lumen segmentation is possible with a precision similar to that obtained by experts. The framework is open for new submissions through the website, at http://coronary.bigr.nl/stenoses/. (C) 2013 Elsevier B.V. All rights reserved.

AB - Though conventional coronary angiography (CCA) has been the standard of reference for diagnosing coronary artery disease in the past decades, computed tomography angiography (CIA) has rapidly emerged, and is nowadays widely used in clinical practice. Here, we introduce a standardized evaluation framework to reliably evaluate and compare the performance of the algorithms devised to detect and quantify the coronary artery stenoses, and to segment the coronary artery lumen in CIA data. The objective of this evaluation framework is to demonstrate the feasibility of dedicated algorithms to: (I) (semi-)automatically detect and quantify stenosis on CIA, in comparison with quantitative coronary angiography (QCA) and CIA consensus reading, and (2) (semi-)automatically segment the coronary lumen on CIA, in comparison with expert's manual annotation. A database consisting of 48 multicenter multivendor cardiac CIA datasets with corresponding reference standards are described and made available. The algorithms from 11 research groups were quantitatively evaluated and compared. The results show that (1) some of the current stenosis detection/quantification algorithms may be used for triage or as a second-reader in clinical practice, and that (2) automatic lumen segmentation is possible with a precision similar to that obtained by experts. The framework is open for new submissions through the website, at http://coronary.bigr.nl/stenoses/. (C) 2013 Elsevier B.V. All rights reserved.

U2 - 10.1016/j.media.2013.05.007

DO - 10.1016/j.media.2013.05.007

M3 - Article

SN - 1361-8415

VL - 17

SP - 859

EP - 876

JO - Medical Image Analysis

JF - Medical Image Analysis

IS - 8

ER -