Gesture in the eye of the beholder: An eye-tracking study on factors determining the attention for gestures produced by people with aphasia

Karin van Nispen, Kazuki Sekine, Ineke van der Meulen, Basil C. Preisig*

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

3 Citations (Scopus)
85 Downloads (Pure)

Abstract

Co-speech hand gestures are an ubiquitous form of nonverbal communication, which can express additional information that is not present in speech. Hand gestures may become more relevant when verbal production is impaired, as in speakers with post-stroke aphasia. In fact, speakers with aphasia produce more gestures than non-brain damaged speakers. Furthermore, there is evidence to suggest that speakers with aphasia produce gestures that convey information essential to understand their communication. In the present study, we addressed the question whether these gestures catch the attention of their addressees. Healthy volunteers (observers) watched short video clips while their eye movements were recorded. These video clips featured speakers with aphasia and non-brain damaged speakers describing two different scenarios (buying a sweater or having witnessed an accident). Our results show that hand gestures produced by speakers with aphasia are on average attended to longer than gestures produced by non-brain damaged speakers. This effect was significant even when we controlled for the longer duration of the gestural movements in speakers with aphasia. Further, the amount of information in speech was also correlated with gesture attention. That is gestures produced by speakers with less informative speech were attended to more frequently. In conclusion, our findings suggest that listeners reallocate their attention and focus more strongly on non-verbal information from co-speech gestures if speech comprehension becomes challenging due to the speaker's verbal production deficits. These findings support a communicative function of co-speech gestures and advocate for instructing people with aphasia to communicate things in the form of gestures that cannot be expressed verbally because interlocutors take notice of these gestures.

Original languageEnglish
Article number108315
JournalNeuropsychologia
Volume174
DOIs
Publication statusPublished - 9 Sept 2022

Bibliographical note

Funding Information:
We thank all participants, that took part in our study, specifically the individuals with aphasia. Furthermore, we would like to thank Kim ten Felde, Jiska Koemans and Ellen van Drie for their role in the data collection and analysis of the data. This work was supported by grants awarded to B.C.P by the Swiss National Science Foundation [P2BEP3_ 168728 / 201864 ] and the Janggen-Pöhn Stiftung.

Publisher Copyright:
© 2022 The Authors

Fingerprint

Dive into the research topics of 'Gesture in the eye of the beholder: An eye-tracking study on factors determining the attention for gestures produced by people with aphasia'. Together they form a unique fingerprint.

Cite this