Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers

Jesse Chandler*, Pam Mueller, Gabriele Paolacci

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

704 Citations (Scopus)
131 Downloads (Pure)

Abstract

Crowdsourcing services-particularly Amazon Mechanical Turk-have made it easy for behavioral scientists to recruit research participants. However, researchers have overlooked crucial differences between crowdsourcing and traditional recruitment methods that provide unique opportunities and challenges. We show that crowdsourced workers are likely to participate across multiple related experiments and that researchers are overzealous in the exclusion of research participants. We describe how both of these problems can be avoided using advanced interface features that also allow prescreening and longitudinal data collection. Using these techniques can minimize the effects of previously ignored drawbacks and expand the scope of crowdsourcing as a tool for psychological research.

Original languageEnglish
Pages (from-to)112-130
Number of pages19
JournalBehavior Research Methods
Volume46
Issue number1
DOIs
Publication statusPublished - 9 Jul 2013

Research programs

  • RSM MKT

Fingerprint

Dive into the research topics of 'Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers'. Together they form a unique fingerprint.

Cite this