Effect on diagnostic accuracy of cognitive reasoning tools for the workplace setting: systematic review and meta-analysis

Justine Staal*, Jacky Hooftman, Sabrina T.G. Gunput, Sílvia Mamede, Maarten A. Frens, Walter W. Van Den Broek, Jelmer Alsma, Laura Zwaan

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

5 Citations (Scopus)
78 Downloads (Pure)



Preventable diagnostic errors are a large burden on healthcare. Cognitive reasoning tools, that is, tools that aim to improve clinical reasoning, are commonly suggested interventions. However, quantitative estimates of tool effectiveness have been aggregated over both workplace-oriented and educational-oriented tools, leaving the impact of workplace-oriented cognitive reasoning tools alone unclear. This systematic review and meta-analysis aims to estimate the effect of cognitive reasoning tools on improving diagnostic performance among medical professionals and students, and to identify factors associated with larger improvements.


Controlled experimental studies that assessed whether cognitive reasoning tools improved the diagnostic accuracy of individual medical students or professionals in a workplace setting were included. Embase.com, Medline ALL via Ovid, Web of Science Core Collection, Cochrane Central Register of Controlled Trials and Google Scholar were searched from inception to 15 October 2021, supplemented with handsearching. Meta-analysis was performed using a random-effects model.


The literature search resulted in 4546 articles of which 29 studies with data from 2732 participants were included for meta-analysis. The pooled estimate showed considerable heterogeneity (I2=70%). This was reduced to I2=38% by removing three studies that offered training with the tool before the intervention effect was measured. After removing these studies, the pooled estimate indicated that cognitive reasoning tools led to a small improvement in diagnostic accuracy (Hedges' g=0.20, 95% CI 0.10 to 0.29, p<0.001). There were no significant subgroup differences.


Cognitive reasoning tools resulted in small but clinically important improvements in diagnostic accuracy in medical students and professionals, although no factors could be distinguished that resulted in larger improvements. Cognitive reasoning tools could be routinely implemented to improve diagnosis in practice, but going forward, more large-scale studies and evaluations of these tools in practice are needed to determine how these tools can be effectively implemented.



Original languageEnglish
Pages (from-to)899-910
Number of pages12
JournalBMJ Quality & Safety
Issue number12
Early online date17 Nov 2022
Publication statusPublished - Dec 2022

Bibliographical note

Publisher Copyright:
© 2022 BMJ Publishing Group. All rights reserved.


Dive into the research topics of 'Effect on diagnostic accuracy of cognitive reasoning tools for the workplace setting: systematic review and meta-analysis'. Together they form a unique fingerprint.

Cite this