Nonstandard Errors

Albert J. Menkveld; Anna  Dreber; Felix  Holzmeister; Juergen Huber; Magnus  Johannesson; Michael  Kirchler; Sebastian Neususs; Michael Razen; Utz Weitzel; David Abad-Diaz; Meni Abudy; Tobias Adrian; Yacine Ait-Sahalia; Olivier Akmansoy; Jamie T. Alcock; Vitali Alexeev; Arash Aloosh; Livia Amato; Diego Amaya; James J. Angel; Alejandro T. Avetikian; Amadeus Bach; Edwin Baidoo; Gaetan Bakalli; Li Bao; Andrea Barbon; Oksana Bashchenko; Parampreet C. Bindra; Geir H. Bjønnes; Jeffrey R. Black; Bernhard S. Black; Dimitar Bogoev; Santiago Bohorquez Correa; Oleg Bondarenko; Charles S. Bos; Ciril Bosch-Rosa; Elie Bouri; Christian Brownlees; Anna Calamia; Viet Nga Cao; Gunther Capelle-Blancard; Laura M. Capera Romero; Francesco Mazzola; Mathijs van Dijk; Patrick Verwijmeren; Sebastian Vogel; Wolf Wagner; Michel van der Wel; Antti Yang; Chen Zhou

doi:10.1111/jofi.13337

Nonstandard Errors

Albert J. Menkveld, Anna Dreber, Felix Holzmeister, Juergen Huber, Magnus Johannesson, Michael Kirchler, Sebastian Neususs, Michael Razen, Utz Weitzel, David Abad-Diaz, Meni Abudy, Tobias Adrian, Yacine Ait-Sahalia, Olivier Akmansoy, Jamie T. Alcock, Vitali Alexeev, Arash Aloosh, Livia Amato, Diego Amaya, James J. AngelAlejandro T. Avetikian, Amadeus Bach, Edwin Baidoo, Gaetan Bakalli, Li Bao, Andrea Barbon, Oksana Bashchenko, Parampreet C. Bindra, Geir H. Bjønnes, Jeffrey R. Black, Bernhard S. Black, Dimitar Bogoev, Santiago Bohorquez Correa, Oleg Bondarenko, Charles S. Bos, Ciril Bosch-Rosa, Elie Bouri, Christian Brownlees, Anna Calamia, Viet Nga Cao, Gunther Capelle-Blancard, Laura M. Capera Romero, Francesco Mazzola, Mathijs van Dijk, Patrick Verwijmeren, Sebastian Vogel, Wolf Wagner, Michel van der Wel, Antti Yang, Chen Zhou

Research output: Contribution to journal › Article › Academic › peer-review

25 Citations (Scopus)

Abstract

In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidencegenerating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

Original language	English
Pages (from-to)	2339-2390
Number of pages	52
Journal	Journal of Finance
Volume	79
Issue number	3
Early online date	17 Apr 2024
DOIs	https://doi.org/10.1111/jofi.13337
Publication status	Published - Jun 2024

Bibliographical note

JEL Classification: C12, C18, G1, G14

© 2024 The Authors. The Journal of Finance published by Wiley Periodicals LLC on behalf of American Finance Association.

Access to Document

10.1111/jofi.13337Licence: CC BY

3961574Final published version, 1.25 MBLicence: CC BY

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3961574Licence: CC BY

Cite this

@article{3d6779eda9f44f3f98d25df7353742ad,

title = "Nonstandard Errors",

abstract = "In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidencegenerating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.",

author = "Menkveld, {Albert J.} and Anna Dreber and Felix Holzmeister and Juergen Huber and Magnus Johannesson and Michael Kirchler and Sebastian Neususs and Michael Razen and Utz Weitzel and David Abad-Diaz and Meni Abudy and Tobias Adrian and Yacine Ait-Sahalia and Olivier Akmansoy and Alcock, {Jamie T.} and Vitali Alexeev and Arash Aloosh and Livia Amato and Diego Amaya and Angel, {James J.} and Avetikian, {Alejandro T.} and Amadeus Bach and Edwin Baidoo and Gaetan Bakalli and Li Bao and Andrea Barbon and Oksana Bashchenko and Bindra, {Parampreet C.} and Bj{\o}nnes, {Geir H.} and Black, {Jeffrey R.} and Black, {Bernhard S.} and Dimitar Bogoev and {Bohorquez Correa}, Santiago and Oleg Bondarenko and Bos, {Charles S.} and Ciril Bosch-Rosa and Elie Bouri and Christian Brownlees and Anna Calamia and Cao, {Viet Nga} and Gunther Capelle-Blancard and {Capera Romero}, {Laura M.} and Francesco Mazzola and {van Dijk}, Mathijs and Patrick Verwijmeren and Sebastian Vogel and Wolf Wagner and {van der Wel}, Michel and Antti Yang and Chen Zhou",

note = "JEL Classification: C12, C18, G1, G14 {\textcopyright} 2024 The Authors. The Journal of Finance published by Wiley Periodicals LLC on behalf of American Finance Association. ",

year = "2024",

month = jun,

doi = "10.1111/jofi.13337",

language = "English",

volume = "79",

pages = "2339--2390",

journal = "Journal of Finance",

issn = "0022-1082",

publisher = "Wiley-Blackwell Publishing Ltd",

number = "3",

}

Menkveld, AJ, Dreber, A, Holzmeister, F, Huber, J, Johannesson, M, Kirchler, M, Neususs, S, Razen, M, Weitzel, U, Abad-Diaz, D, Abudy, M, Adrian, T, Ait-Sahalia, Y, Akmansoy, O, Alcock, JT, Alexeev, V, Aloosh, A, Amato, L, Amaya, D, Angel, JJ, Avetikian, AT, Bach, A, Baidoo, E, Bakalli, G, Bao, L, Barbon, A, Bashchenko, O, Bindra, PC, Bjønnes, GH, Black, JR, Black, BS, Bogoev, D, Bohorquez Correa, S, Bondarenko, O, Bos, CS, Bosch-Rosa, C, Bouri, E, Brownlees, C, Calamia, A, Cao, VN, Capelle-Blancard, G, Capera Romero, LM, Mazzola, F , van Dijk, M , Verwijmeren, P , Vogel, S , Wagner, W , van der Wel, M, Yang, A & Zhou, C 2024, 'Nonstandard Errors', Journal of Finance, vol. 79, no. 3, pp. 2339-2390. https://doi.org/10.1111/jofi.13337

TY - JOUR

T1 - Nonstandard Errors

AU - Menkveld, Albert J.

AU - Dreber, Anna

AU - Holzmeister, Felix

AU - Huber, Juergen

AU - Johannesson, Magnus

AU - Kirchler, Michael

AU - Neususs, Sebastian

AU - Razen, Michael

AU - Weitzel, Utz

AU - Abad-Diaz, David

AU - Abudy, Meni

AU - Adrian, Tobias

AU - Ait-Sahalia, Yacine

AU - Akmansoy, Olivier

AU - Alcock, Jamie T.

AU - Alexeev, Vitali

AU - Aloosh, Arash

AU - Amato, Livia

AU - Amaya, Diego

AU - Angel, James J.

AU - Avetikian, Alejandro T.

AU - Bach, Amadeus

AU - Baidoo, Edwin

AU - Bakalli, Gaetan

AU - Bao, Li

AU - Barbon, Andrea

AU - Bashchenko, Oksana

AU - Bindra, Parampreet C.

AU - Bjønnes, Geir H.

AU - Black, Jeffrey R.

AU - Black, Bernhard S.

AU - Bogoev, Dimitar

AU - Bohorquez Correa, Santiago

AU - Bondarenko, Oleg

AU - Bos, Charles S.

AU - Bosch-Rosa, Ciril

AU - Bouri, Elie

AU - Brownlees, Christian

AU - Calamia, Anna

AU - Cao, Viet Nga

AU - Capelle-Blancard, Gunther

AU - Capera Romero, Laura M.

AU - Mazzola, Francesco

AU - van Dijk, Mathijs

AU - Verwijmeren, Patrick

AU - Vogel, Sebastian

AU - Wagner, Wolf

AU - van der Wel, Michel

AU - Yang, Antti

AU - Zhou, Chen

PY - 2024/6

Y1 - 2024/6

N2 - In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidencegenerating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

AB - In statistics, samples are drawn from a population in a data-generating process (DGP). Standard errors measure the uncertainty in estimates of population parameters. In science, evidence is generated to test hypotheses in an evidencegenerating process (EGP). We claim that EGP variation across researchers adds uncertainty: Non-standard errors (NSEs). We study NSEs by letting 164 teams test the same hypotheses on the same data. NSEs turn out to be sizable, but smaller for better reproducible or higher rated research. Adding peer-review stages reduces NSEs. We further find that this type of uncertainty is underestimated by participants.

UR - https://ssrn.com/abstract=3961574

UR - https://www.youtube.com/watch?v=M-8bQAIySGQ

UR - http://www.scopus.com/inward/record.url?scp=85190503164&partnerID=8YFLogxK

U2 - 10.1111/jofi.13337

DO - 10.1111/jofi.13337

M3 - Article

SN - 0022-1082

VL - 79

SP - 2339

EP - 2390

JO - Journal of Finance

JF - Journal of Finance

IS - 3

ER -

Nonstandard Errors

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this