Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in computed tomography angiography

Hortense Kirisli, Michiel Schaap, Coert Metz, Anoeshka Dharampal, Bob Meijboom, Elina Papadopoulou, Admir Dedic, Koen Nieman, MA (Michiel) van der Graaf, MFL Meijs, MJ Cramer, A Broersen, S Cetin, A Eslami, L Florez-Valencia, KL Lor, B Matuszewski, I Melki, B Mohr, I OksuzRahil Shahzad, C Wang, PH Kitslaar, G Unal, A Katouzian, M Orkisz, CM Chen, F Precioso, L Najman, S Masood, D Unay, L van Vliet, R Moreno, R Goldenberg, E Vucini, Gabriel Krestin, Wiro Niessen, Theo van Walsum

Research output: Contribution to journalArticleAcademic

166 Citations (Scopus)


Though conventional coronary angiography (CCA) has been the standard of reference for diagnosing coronary artery disease in the past decades, computed tomography angiography (CIA) has rapidly emerged, and is nowadays widely used in clinical practice. Here, we introduce a standardized evaluation framework to reliably evaluate and compare the performance of the algorithms devised to detect and quantify the coronary artery stenoses, and to segment the coronary artery lumen in CIA data. The objective of this evaluation framework is to demonstrate the feasibility of dedicated algorithms to: (I) (semi-)automatically detect and quantify stenosis on CIA, in comparison with quantitative coronary angiography (QCA) and CIA consensus reading, and (2) (semi-)automatically segment the coronary lumen on CIA, in comparison with expert's manual annotation. A database consisting of 48 multicenter multivendor cardiac CIA datasets with corresponding reference standards are described and made available. The algorithms from 11 research groups were quantitatively evaluated and compared. The results show that (1) some of the current stenosis detection/quantification algorithms may be used for triage or as a second-reader in clinical practice, and that (2) automatic lumen segmentation is possible with a precision similar to that obtained by experts. The framework is open for new submissions through the website, at (C) 2013 Elsevier B.V. All rights reserved.
Original languageUndefined/Unknown
Pages (from-to)859-876
Number of pages18
JournalMedical Image Analysis
Issue number8
Publication statusPublished - 2013

Research programs

  • EMC COEUR-09
  • EMC NIHES-03-30-01
  • EMC NIHES-03-30-03

Cite this