Semantic noise in the Winograd Schema Challenge of pronoun disambiguation

S. de Jager*

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

1 Citation (Scopus)


The Winograd Schema Challenge (WSC) of pronoun disambiguation is a Natural Language Processing (NLP) task designed to test to what extent the reading comprehension capabilities of language models (LMs) can be compared to those of human subjects. It is generally assumed across the NLP literature that human subjects are capable of resolving this task because of their acquired commonsense knowledge, thus setting a commonsense benchmark for LMs, one which has even been proposed as an alternative to the Turing test. In the context of complex natural language communications, Shannon and Weaver observed that the act of semantic interpretation is subject to semantic noise (Shannon and Weaver, 1964 (1949)). Semantic noise is a constraint that ensues from terms exhibiting variable interpretations across contexts, presenting a challenge to the resolution of tasks such as the WSC. However, the main argument of this paper is that rather than seeing semantic noise as a challenge to otherwise unambiguous communication, it can also be understood as a functional quality of natural language, given that it results in the conceptual negotiation of terms. Failing to theoretically attend to this linguistic matter of fact leads to unintended problems in instances where NLP applications are offered as unbiased or objectively applicable solutions. To address this, this article offers a renewed and original analysis of a series of Winograd Schemas, in order to demonstrate how they are not as straightforwardly solvable by human subjects as is commonly claimed across the NLP literature. The methodology employed is that of historical contextualisation in information theory, and qualitative cultural analysis drawing on examples from a wide variety of recent NLP literature.

Original languageEnglish
Article number161
Pages (from-to)1-10
Number of pages10
JournalHumanities and Social Sciences Communications
Issue number1
Publication statusPublished - 11 Apr 2023

Bibliographical note

Funding Information:
This article was partly made possible by the Noise Research Union (N.R.U.). The author would also like to acknowledge the dialogical, review-process exchanges that led to a legible version of the present article.

Publisher Copyright:
© 2023, The Author(s).


Dive into the research topics of 'Semantic noise in the Winograd Schema Challenge of pronoun disambiguation'. Together they form a unique fingerprint.

Cite this