Bias in expert product reviews

Ben Vollaard; Jan C. van Ours

doi:10.1016/j.jebo.2022.08.002

Bias in expert product reviews

Ben Vollaard, Jan C. van Ours^*

^*Corresponding author for this work

Applied Economics

Research output: Contribution to journal › Article › Academic › peer-review

8 Citations (Scopus)

Abstract

Many expert reviews of products such as cars, books, movies and restaurants are non-blind. Whether such reviews can be taken at face value is questionable, but hard evidence on the presence of reviewer bias is rare. This holds particularly true for conflicts of interest that are thought to be common in non-blind product reviews but are not readily observable: ad hoc relationships between reviewers and producers. We present a textbook case of a long-running expert product review in the food service industry for which we know the reviewer's conflict of interest: being affiliated to one particular producer. As is typical, only insiders were aware of the possible source of bias in the review. The review resembles other non-blind tests of product quality. We find evidence of a sizable bias in the reviewers’ ratings. Our findings suggest that reviewers’ ad hoc relationships with producers, often dismissed as ‘coming with the job’, can be very harmful.

Original language	English
Pages (from-to)	105-118
Number of pages	14
Journal	Journal of Economic Behavior and Organization
Volume	202
DOIs	https://doi.org/10.1016/j.jebo.2022.08.002
Publication status	Published - Oct 2022

Bibliographical note

Funding Information:
Joachim Winter was co-editor for this article. We thank him and two anonymous reviewers for helpful comments and suggestions on earlier drafts of the manuscript. We also thank four individuals for their independent ratings of the verbal judgments. Data and code are available on request.

Publisher Copyright:
© 2022 The Author(s)

Access to Document

10.1016/j.jebo.2022.08.002Licence: CC BY

Cite this

@article{36e0d5fba57343d6944e900b7164ec90,

title = "Bias in expert product reviews",

abstract = "Many expert reviews of products such as cars, books, movies and restaurants are non-blind. Whether such reviews can be taken at face value is questionable, but hard evidence on the presence of reviewer bias is rare. This holds particularly true for conflicts of interest that are thought to be common in non-blind product reviews but are not readily observable: ad hoc relationships between reviewers and producers. We present a textbook case of a long-running expert product review in the food service industry for which we know the reviewer's conflict of interest: being affiliated to one particular producer. As is typical, only insiders were aware of the possible source of bias in the review. The review resembles other non-blind tests of product quality. We find evidence of a sizable bias in the reviewers{\textquoteright} ratings. Our findings suggest that reviewers{\textquoteright} ad hoc relationships with producers, often dismissed as {\textquoteleft}coming with the job{\textquoteright}, can be very harmful.",

author = "Ben Vollaard and {van Ours}, {Jan C.}",

note = "Funding Information: Joachim Winter was co-editor for this article. We thank him and two anonymous reviewers for helpful comments and suggestions on earlier drafts of the manuscript. We also thank four individuals for their independent ratings of the verbal judgments. Data and code are available on request. Publisher Copyright: {\textcopyright} 2022 The Author(s)",

year = "2022",

month = oct,

doi = "10.1016/j.jebo.2022.08.002",

language = "English",

volume = "202",

pages = "105--118",

journal = "Journal of Economic Behavior and Organization",

issn = "0167-2681",

publisher = "Elsevier",

}

TY - JOUR

T1 - Bias in expert product reviews

AU - Vollaard, Ben

AU - van Ours, Jan C.

N1 - Funding Information: Joachim Winter was co-editor for this article. We thank him and two anonymous reviewers for helpful comments and suggestions on earlier drafts of the manuscript. We also thank four individuals for their independent ratings of the verbal judgments. Data and code are available on request. Publisher Copyright: © 2022 The Author(s)

PY - 2022/10

Y1 - 2022/10

N2 - Many expert reviews of products such as cars, books, movies and restaurants are non-blind. Whether such reviews can be taken at face value is questionable, but hard evidence on the presence of reviewer bias is rare. This holds particularly true for conflicts of interest that are thought to be common in non-blind product reviews but are not readily observable: ad hoc relationships between reviewers and producers. We present a textbook case of a long-running expert product review in the food service industry for which we know the reviewer's conflict of interest: being affiliated to one particular producer. As is typical, only insiders were aware of the possible source of bias in the review. The review resembles other non-blind tests of product quality. We find evidence of a sizable bias in the reviewers’ ratings. Our findings suggest that reviewers’ ad hoc relationships with producers, often dismissed as ‘coming with the job’, can be very harmful.

AB - Many expert reviews of products such as cars, books, movies and restaurants are non-blind. Whether such reviews can be taken at face value is questionable, but hard evidence on the presence of reviewer bias is rare. This holds particularly true for conflicts of interest that are thought to be common in non-blind product reviews but are not readily observable: ad hoc relationships between reviewers and producers. We present a textbook case of a long-running expert product review in the food service industry for which we know the reviewer's conflict of interest: being affiliated to one particular producer. As is typical, only insiders were aware of the possible source of bias in the review. The review resembles other non-blind tests of product quality. We find evidence of a sizable bias in the reviewers’ ratings. Our findings suggest that reviewers’ ad hoc relationships with producers, often dismissed as ‘coming with the job’, can be very harmful.

UR - http://www.scopus.com/inward/record.url?scp=85135928424&partnerID=8YFLogxK

U2 - 10.1016/j.jebo.2022.08.002

DO - 10.1016/j.jebo.2022.08.002

M3 - Article

AN - SCOPUS:85135928424

SN - 0167-2681

VL - 202

SP - 105

EP - 118

JO - Journal of Economic Behavior and Organization

JF - Journal of Economic Behavior and Organization

ER -

Bias in expert product reviews

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint