Crowdsourcing Samples in Cognitive Science

N Stewart, J Chandler, Gabriele Paolacci

Research output: Contribution to journalArticleAcademicpeer-review

135 Citations (Scopus)
18 Downloads (Pure)


Crowdsourcing data collection from research participants recruited from online labor markets is now common in cognitive science. We review who is in the crowd and who can be reached by the average laboratory. We discuss reproducibility and review some recent methodological innovations for online experiments. We consider the design of research studies and arising ethical issues. We review how to code experiments for the web, what is known about video and audio presentation, and the measurement of reaction times. We close with comments about the high levels of experience of many participants and an emerging tragedy of the commons.
Original languageEnglish
Pages (from-to)736-748
Number of pages13
JournalTrends in Cognitive Sciences
Issue number10
Publication statusPublished - 21 Aug 2017


Dive into the research topics of 'Crowdsourcing Samples in Cognitive Science'. Together they form a unique fingerprint.

Cite this