Multi-view Contour-constrained Transformer Network for Thin-cap Fibroatheroma Identification

Sijie Liu, Jingmin Xin*, Jiayi Wu, Yangyang Deng, Ruisheng Su, Wiro Niessen, Nanning Zheng, Theo van Walsum

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

36 Downloads (Pure)


Identification and detection of thin-cap fibroatheroma (TCFA) from intravascular optical coherence tomography (IVOCT) images is critical for treatment of coronary heart diseases. Recently, deep learning methods have shown promising successes in TCFA identification. However, most methods usually do not effectively utilize multi-view information or incorporate prior domain knowledge. In this paper, we propose a multi-view contour-constrained transformer network (MVCTN) for TCFA identification in IVOCT images. Inspired by the diagnosis process of cardiologists, we use contour constrained self-attention modules (CCSM) to emphasize features corresponding to salient regions (i.e., vessel walls) in an unsupervised manner and enhance the visual interpretability based on class activation mapping (CAM). Moreover, we exploit transformer modules (TM) to build global-range relations between two views (i.e., polar and Cartesian views) to effectively fuse features at multiple feature scales. Experimental results on a semi-public dataset and an in-house dataset demonstrate that the proposed MVCTN outperforms other single-view and multi-view methods. Lastly, the proposed MVCTN can also provide meaningful visualization for cardiologists via CAM.

Original languageEnglish
Pages (from-to)224-234
Number of pages11
Publication statusPublished - 28 Feb 2023

Bibliographical note

This work was supported in part by the National Key Research and Development Program of China under Grant 2017YFA0700800, and the National Natural Science Foundation of China under Grant 62088102 and 82000336. We thank Professor Bo Yu from Department of Cardiology, Second Affiliated Hospital of Harbin Medical University, Harbin, China for help in building the Optical coherence tomography Plaque Recognition Database (OPRD).

Publisher Copyright: © 2022 The Author(s)


Dive into the research topics of 'Multi-view Contour-constrained Transformer Network for Thin-cap Fibroatheroma Identification'. Together they form a unique fingerprint.

Cite this