Crowdsourcing Samples in Cognitive Science

N Stewart; J Chandler; Gabriele Paolacci

doi:10.1016/j.tics.2017.06.007

Crowdsourcing Samples in Cognitive Science

N Stewart, J Chandler, Gabriele Paolacci

Department of Marketing Management

Research output: Contribution to journal › Article › Academic › peer-review

157 Citations (Scopus)

42 Downloads (Pure)

Abstract

Crowdsourcing data collection from research participants recruited from online labor markets is now common in cognitive science. We review who is in the crowd and who can be reached by the average laboratory. We discuss reproducibility and review some recent methodological innovations for online experiments. We consider the design of research studies and arising ethical issues. We review how to code experiments for the web, what is known about video and audio presentation, and the measurement of reaction times. We close with comments about the high levels of experience of many participants and an emerging tragedy of the commons.

Original language	English
Pages (from-to)	736-748
Number of pages	13
Journal	Trends in Cognitive Sciences
Volume	21
Issue number	10
DOIs	https://doi.org/10.1016/j.tics.2017.06.007
Publication status	Published - 21 Aug 2017

Research programs

RSM MKT

Access to Document

10.1016/j.tics.2017.06.007

REPUB_101603-OA.pdfFinal published version, 1.51 MB

http://hdl.handle.net/1765/101603

Cite this

@article{18bc81bf32924c6788c8e614149a4703,

title = "Crowdsourcing Samples in Cognitive Science",

abstract = "Crowdsourcing data collection from research participants recruited from online labor markets is now common in cognitive science. We review who is in the crowd and who can be reached by the average laboratory. We discuss reproducibility and review some recent methodological innovations for online experiments. We consider the design of research studies and arising ethical issues. We review how to code experiments for the web, what is known about video and audio presentation, and the measurement of reaction times. We close with comments about the high levels of experience of many participants and an emerging tragedy of the commons.",

author = "N Stewart and J Chandler and Gabriele Paolacci",

year = "2017",

month = aug,

day = "21",

doi = "10.1016/j.tics.2017.06.007",

language = "English",

volume = "21",

pages = "736--748",

journal = "Trends in Cognitive Sciences",

issn = "1364-6613",

publisher = "Elsevier Ltd.",

number = "10",

}

TY - JOUR

T1 - Crowdsourcing Samples in Cognitive Science

AU - Stewart, N

AU - Chandler, J

AU - Paolacci, Gabriele

PY - 2017/8/21

Y1 - 2017/8/21

N2 - Crowdsourcing data collection from research participants recruited from online labor markets is now common in cognitive science. We review who is in the crowd and who can be reached by the average laboratory. We discuss reproducibility and review some recent methodological innovations for online experiments. We consider the design of research studies and arising ethical issues. We review how to code experiments for the web, what is known about video and audio presentation, and the measurement of reaction times. We close with comments about the high levels of experience of many participants and an emerging tragedy of the commons.

AB - Crowdsourcing data collection from research participants recruited from online labor markets is now common in cognitive science. We review who is in the crowd and who can be reached by the average laboratory. We discuss reproducibility and review some recent methodological innovations for online experiments. We consider the design of research studies and arising ethical issues. We review how to code experiments for the web, what is known about video and audio presentation, and the measurement of reaction times. We close with comments about the high levels of experience of many participants and an emerging tragedy of the commons.

U2 - 10.1016/j.tics.2017.06.007

DO - 10.1016/j.tics.2017.06.007

M3 - Article

C2 - 28803699

SN - 1364-6613

VL - 21

SP - 736

EP - 748

JO - Trends in Cognitive Sciences

JF - Trends in Cognitive Sciences

IS - 10

ER -

Crowdsourcing Samples in Cognitive Science

Abstract

Research programs

Access to Document

Fingerprint

Cite this