A General Survey on Attention Mechanisms in Deep Learning

Gianni Brauwers, Flavius Frasincar*

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

236 Citations (Scopus)
637 Downloads (Pure)

Abstract

Attention is an important mechanism that can be employed for a variety of deep learning models across many different domains and tasks. This survey provides an overview of the most important attention mechanisms proposed in the literature. The various attention mechanisms are explained by means of a framework consisting of a general attention model, uniform notation, and a comprehensive taxonomy of attention mechanisms. Furthermore, the various measures for evaluating attention models are reviewed, and methods to characterize the structure of attention models based on the proposed framework are discussed. Last, future work in the field of attention models is considered.

Original languageEnglish
Pages (from-to)3279-3298
Number of pages20
JournalIEEE Transactions on Knowledge and Data Engineering
Volume35
Issue number4
DOIs
Publication statusPublished - 1 Apr 2023

Bibliographical note

Publisher Copyright:
© 1989-2012 IEEE.

Fingerprint

Dive into the research topics of 'A General Survey on Attention Mechanisms in Deep Learning'. Together they form a unique fingerprint.

Cite this