Tighter nets for smaller fishes? Mapping the development of statistical practices in consumer research between 2008 and 2020

Antonia Krefeld-Schwalb, Benjamin Scheibehenne

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

During the last decade, confidence in many social sciences, including consumer research, has been undermined by doubts about the replicability of empirical research findings. These doubts have led to increased calls to improve research practices and adopt new measures to increase the replicability of published work from various stakeholders such as funding agencies, journals, and scholars themselves. Despite these demands, it is unclear to which the research published in the leading consumer research journals has adhered to these calls for change. This article provides the first systematic empirical analysis of this question by surveying three crucial statistics of published consumer research over time: sample sizes, effect sizes, and the distribution of published p values. The authors compile a hand-coded sample of N = 258 articles published between 2008 and 2020 in the Journal of Consumer Psychology, the Journal of Consumer Research, and the Journal of Marketing Research. An automated text analysis across all publications in these three journals corroborates the representativeness of the hand-coded sample. Results reveal a substantial increase in sample sizes above and beyond the use of online samples along with a decrease in reported effect sizes. Effect and samples sizes are highly correlated which at least partially explains the reduction in reported effect sizes.

Original languageEnglish
JournalMarketing Letters
DOIs
Publication statusPublished - 20 Dec 2022

Bibliographical note

Funding Information:
The authors gratefully acknowledge funding from the Swiss National Science Foundation (Project No 172806).

Publisher Copyright:
© 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

Fingerprint

Dive into the research topics of 'Tighter nets for smaller fishes? Mapping the development of statistical practices in consumer research between 2008 and 2020'. Together they form a unique fingerprint.

Cite this