Making ERP research more transparent: Guidelines for preregistration

Mariella Paul; Gisela H. Govaart; Antonio Schettino

doi:10.1016/j.ijpsycho.2021.02.016

Making ERP research more transparent: Guidelines for preregistration

Mariella Paul, Gisela H. Govaart, Antonio Schettino^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

39 Citations (Scopus)

Abstract

A combination of confirmation bias, hindsight bias, and pressure to publish may prompt the (unconscious) exploration of various methodological options and reporting only the ones that lead to a (statistically) significant outcome. This undisclosed analytic flexibility is particularly relevant in EEG research, where a myriad of preprocessing and analysis pipelines can be used to extract information from complex multidimensional data. One solution to limit confirmation and hindsight bias by disclosing analytic choices is preregistration: researchers write a time-stamped, publicly accessible research plan with hypotheses, data collection plan, and the intended preprocessing and statistical analyses before the start of a research project. In this manuscript, we present an overview of the problems associated with undisclosed analytic flexibility, discuss why and how EEG researchers would benefit from adopting preregistration, provide guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies, and conclude by discussing possibilities and limitations of this open science practice.

Original language	English
Pages (from-to)	52-63
Number of pages	12
Journal	International Journal of Psychophysiology
Volume	164
DOIs	https://doi.org/10.1016/j.ijpsycho.2021.02.016
Publication status	Published - Jun 2021
Externally published	Yes

Bibliographical note

Funding Information:
We would like to thank Dr. Matt Craddock for his input on a preliminary draft, as well as the kind and thoughtful comments by Dr. Mante S. Nieuwland and another anonymous reviewer on an earlier version of the manuscript. MP is funded by the Max Planck Society and the Berlin School of Mind and Brain. GHG is funded by the Max Planck Society and the Einstein Center for Neurosciences Berlin. AS is employed at Erasmus University Rotterdam as Coordinator Open Science. Funders and employers had no role in project design, literature review, writing of the paper, and/or decision to submit for publication.

Publisher Copyright:
© 2021 The Authors

Access to Document

10.1016/j.ijpsycho.2021.02.016

Cite this

@article{efa672bf208142358b34c86b6a0382e7,

title = "Making ERP research more transparent: Guidelines for preregistration",

abstract = "A combination of confirmation bias, hindsight bias, and pressure to publish may prompt the (unconscious) exploration of various methodological options and reporting only the ones that lead to a (statistically) significant outcome. This undisclosed analytic flexibility is particularly relevant in EEG research, where a myriad of preprocessing and analysis pipelines can be used to extract information from complex multidimensional data. One solution to limit confirmation and hindsight bias by disclosing analytic choices is preregistration: researchers write a time-stamped, publicly accessible research plan with hypotheses, data collection plan, and the intended preprocessing and statistical analyses before the start of a research project. In this manuscript, we present an overview of the problems associated with undisclosed analytic flexibility, discuss why and how EEG researchers would benefit from adopting preregistration, provide guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies, and conclude by discussing possibilities and limitations of this open science practice.",

author = "Mariella Paul and Govaart, {Gisela H.} and Antonio Schettino",

note = "Funding Information: We would like to thank Dr. Matt Craddock for his input on a preliminary draft, as well as the kind and thoughtful comments by Dr. Mante S. Nieuwland and another anonymous reviewer on an earlier version of the manuscript. MP is funded by the Max Planck Society and the Berlin School of Mind and Brain. GHG is funded by the Max Planck Society and the Einstein Center for Neurosciences Berlin. AS is employed at Erasmus University Rotterdam as Coordinator Open Science. Funders and employers had no role in project design, literature review, writing of the paper, and/or decision to submit for publication. Publisher Copyright: {\textcopyright} 2021 The Authors",

year = "2021",

month = jun,

doi = "10.1016/j.ijpsycho.2021.02.016",

language = "English",

volume = "164",

pages = "52--63",

journal = "International Journal of Psychophysiology",

issn = "0167-8760",

publisher = "Elsevier",

}

TY - JOUR

T1 - Making ERP research more transparent

T2 - Guidelines for preregistration

AU - Paul, Mariella

AU - Govaart, Gisela H.

AU - Schettino, Antonio

N1 - Funding Information: We would like to thank Dr. Matt Craddock for his input on a preliminary draft, as well as the kind and thoughtful comments by Dr. Mante S. Nieuwland and another anonymous reviewer on an earlier version of the manuscript. MP is funded by the Max Planck Society and the Berlin School of Mind and Brain. GHG is funded by the Max Planck Society and the Einstein Center for Neurosciences Berlin. AS is employed at Erasmus University Rotterdam as Coordinator Open Science. Funders and employers had no role in project design, literature review, writing of the paper, and/or decision to submit for publication. Publisher Copyright: © 2021 The Authors

PY - 2021/6

Y1 - 2021/6

N2 - A combination of confirmation bias, hindsight bias, and pressure to publish may prompt the (unconscious) exploration of various methodological options and reporting only the ones that lead to a (statistically) significant outcome. This undisclosed analytic flexibility is particularly relevant in EEG research, where a myriad of preprocessing and analysis pipelines can be used to extract information from complex multidimensional data. One solution to limit confirmation and hindsight bias by disclosing analytic choices is preregistration: researchers write a time-stamped, publicly accessible research plan with hypotheses, data collection plan, and the intended preprocessing and statistical analyses before the start of a research project. In this manuscript, we present an overview of the problems associated with undisclosed analytic flexibility, discuss why and how EEG researchers would benefit from adopting preregistration, provide guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies, and conclude by discussing possibilities and limitations of this open science practice.

AB - A combination of confirmation bias, hindsight bias, and pressure to publish may prompt the (unconscious) exploration of various methodological options and reporting only the ones that lead to a (statistically) significant outcome. This undisclosed analytic flexibility is particularly relevant in EEG research, where a myriad of preprocessing and analysis pipelines can be used to extract information from complex multidimensional data. One solution to limit confirmation and hindsight bias by disclosing analytic choices is preregistration: researchers write a time-stamped, publicly accessible research plan with hypotheses, data collection plan, and the intended preprocessing and statistical analyses before the start of a research project. In this manuscript, we present an overview of the problems associated with undisclosed analytic flexibility, discuss why and how EEG researchers would benefit from adopting preregistration, provide guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies, and conclude by discussing possibilities and limitations of this open science practice.

UR - http://www.scopus.com/inward/record.url?scp=85102134610&partnerID=8YFLogxK

U2 - 10.1016/j.ijpsycho.2021.02.016

DO - 10.1016/j.ijpsycho.2021.02.016

M3 - Article

C2 - 33676957

AN - SCOPUS:85102134610

SN - 0167-8760

VL - 164

SP - 52

EP - 63

JO - International Journal of Psychophysiology

JF - International Journal of Psychophysiology

ER -

Making ERP research more transparent: Guidelines for preregistration

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this