Why we habitually engage in null-hypothesis significance testing: A qualitative study

Jonah Stunt*, Leonie van Grootel, Lex Bouter, David Trafimow, Trynke Hoekstra, Michiel de Boer

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

2 Citations (Scopus)
41 Downloads (Pure)


Background: Null Hypothesis Significance Testing (NHST) is the most familiar statistical procedure for making inferences about population effects. Important problems associated with this method have been addressed and various alternatives that overcome these problems have been developed. Despite its many well-documented drawbacks, NHST remains the prevailing method for drawing conclusions from data. Reasons for this have been insufficiently investigated. Therefore, the aim of our study was to explore the perceived barriers and facilitators related to the use of NHST and alternative statistical procedures among relevant stakeholders in the scientific system.

Methods: Individual semi-structured interviews and focus groups were conducted with junior and senior researchers, lecturers in statistics, editors of scientific journals and program leaders of funding agencies. During the focus groups, important themes that emerged from the interviews were discussed. Data analysis was performed using the constant comparison method, allowing emerging (sub)themes to be fully explored. A theory substantiating the prevailing use of NHST was developed based on the main themes and subthemes we identified.

Results: Twenty-nine interviews and six focus groups were conducted. Several interrelated facilitators and barriers associated with the use of NHST and alternative statistical procedures were identified. These factors were subsumed under three main themes: the scientific climate, scientific duty, and reactivity. As a result of the factors, most participants feel dependent in their actions upon others, have become reactive, and await action and initiatives from others. This may explain why NHST is still the standard and ubiquitously used by almost everyone involved.

Conclusion: Our findings demonstrate how perceived barriers to shift away from NHST set a high threshold for actual behavioral change and create a circle of interdependency between stakeholders. By taking small steps it should be possible to decrease the scientific community's strong dependence on NHST and p-values.

Original languageEnglish
Article numbere0258330
JournalPLoS ONE
Issue number10 (October)
Publication statusPublished - 15 Oct 2021

Bibliographical note

Funding Information:
This research was funded by the NWO (Nederlandse Organisatie voor Wetenschappelijk Onderzoek; Dutch Organization for Scientific Research) (https://www.nwo.nl/) The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The authors are grateful to Anja de Kruif for her contribution to the design of the study and for moderating one of the focus groups.

Publisher Copyright:
© 2021 Stunt et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.


Dive into the research topics of 'Why we habitually engage in null-hypothesis significance testing: A qualitative study'. Together they form a unique fingerprint.

Cite this