Running experiments on Amazon Mechanical Turk

Gabriele Paolacci, J Chandler, P Ipeirotis

Research output: Contribution to journalArticleAcademicpeer-review

3551 Citations (Scopus)
7 Downloads (Pure)


Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality of data provided by subjects recruited from online labor markets. We address these potential concerns by presenting new demographic data about the Mechanical Turk subject population, reviewing the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and comparing the magnitude of effects obtained using Mechanical Turk and traditional subject pools. We further discuss some additional benefits such as the possibility of longitudinal, cross cultural and prescreening designs, and offer some advice on how to best manage a common subject pool.
Original languageEnglish
Pages (from-to)411-419
Number of pages9
JournalJudgment and Decision Making
Issue number5
Publication statusPublished - 2010
Externally publishedYes

Research programs

  • ESE - MKT


Dive into the research topics of 'Running experiments on Amazon Mechanical Turk'. Together they form a unique fingerprint.

Cite this