Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review

Cynthia Yang; Jan A Kors; Solomon Ioannou; Luis H John; Aniek F Markus; Alexandros Rekkas; Maria A J de Ridder; Tom M Seinen; Ross D Williams; Peter R Rijnbeek

doi:10.1093/jamia/ocac002

Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review

Cynthia Yang^*, Jan A Kors, Solomon Ioannou, Luis H John, Aniek F Markus, Alexandros Rekkas, Maria A J de Ridder, Tom M Seinen, Ross D Williams, Peter R Rijnbeek

^*Corresponding author for this work

Medical Informatics

Research output: Contribution to journal › Article › Academic › peer-review

2 Citations (Scopus)

7 Downloads (Pure)

Abstract

OBJECTIVES: This systematic review aims to provide further insights into the conduct and reporting of clinical prediction model development and validation over time. We focus on assessing the reporting of information necessary to enable external validation by other investigators.

MATERIALS AND METHODS: We searched Embase, Medline, Web-of-Science, Cochrane Library, and Google Scholar to identify studies that developed 1 or more multivariable prognostic prediction models using electronic health record (EHR) data published in the period 2009-2019.

RESULTS: We identified 422 studies that developed a total of 579 clinical prediction models using EHR data. We observed a steep increase over the years in the number of developed models. The percentage of models externally validated in the same paper remained at around 10%. Throughout 2009-2019, for both the target population and the outcome definitions, code lists were provided for less than 20% of the models. For about half of the models that were developed using regression analysis, the final model was not completely presented.

DISCUSSION: Overall, we observed limited improvement over time in the conduct and reporting of clinical prediction model development and validation. In particular, the prediction problem definition was often not clearly reported, and the final model was often not completely presented.

CONCLUSION: Improvement in the reporting of information necessary to enable external validation by other investigators is still urgently needed to increase clinical adoption of developed models.

Original language	English
Pages (from-to)	983-989
Number of pages	7
Journal	Journal of the American Medical Informatics Association : JAMIA
Volume	29
Issue number	5
DOIs	https://doi.org/10.1093/jamia/ocac002
Publication status	Published - 1 May 2022

Bibliographical note

Funding Information:
This work has received support from the European Health Data & Evidence Network (EHDEN) project. EHDEN has received funding from the Innovative Medicines Initiative 2 Joint Undertaking (JU) under grant agreement no 806968. The JU receives support from the European Union's Horizon 2020 research and innovation program and EFPIA.

Publisher Copyright:
© 2022 The Author(s) 2022. Published by Oxford University Press on behalf of the American Medical Informatics Association.

Access to Document

10.1093/jamia/ocac002Licence: CC BY

ocac002Final published version, 624 KBLicence: CC BY

Cite this

Yang, C., Kors, J. A., Ioannou, S., John, L. H., Markus, A. F., Rekkas, A., de Ridder, M. A. J., Seinen, T. M., Williams, R. D., & Rijnbeek, P. R. (2022). Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review. Journal of the American Medical Informatics Association : JAMIA, 29(5), 983-989. https://doi.org/10.1093/jamia/ocac002

@article{5858cfdc8a854cd6abfcd96f09349c2d,

title = "Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review",

abstract = "OBJECTIVES: This systematic review aims to provide further insights into the conduct and reporting of clinical prediction model development and validation over time. We focus on assessing the reporting of information necessary to enable external validation by other investigators.MATERIALS AND METHODS: We searched Embase, Medline, Web-of-Science, Cochrane Library, and Google Scholar to identify studies that developed 1 or more multivariable prognostic prediction models using electronic health record (EHR) data published in the period 2009-2019.RESULTS: We identified 422 studies that developed a total of 579 clinical prediction models using EHR data. We observed a steep increase over the years in the number of developed models. The percentage of models externally validated in the same paper remained at around 10%. Throughout 2009-2019, for both the target population and the outcome definitions, code lists were provided for less than 20% of the models. For about half of the models that were developed using regression analysis, the final model was not completely presented.DISCUSSION: Overall, we observed limited improvement over time in the conduct and reporting of clinical prediction model development and validation. In particular, the prediction problem definition was often not clearly reported, and the final model was often not completely presented.CONCLUSION: Improvement in the reporting of information necessary to enable external validation by other investigators is still urgently needed to increase clinical adoption of developed models.",

author = "Cynthia Yang and Kors, {Jan A} and Solomon Ioannou and John, {Luis H} and Markus, {Aniek F} and Alexandros Rekkas and {de Ridder}, {Maria A J} and Seinen, {Tom M} and Williams, {Ross D} and Rijnbeek, {Peter R}",

note = "Funding Information: This work has received support from the European Health Data & Evidence Network (EHDEN) project. EHDEN has received funding from the Innovative Medicines Initiative 2 Joint Undertaking (JU) under grant agreement no 806968. The JU receives support from the European Union's Horizon 2020 research and innovation program and EFPIA. Publisher Copyright: {\textcopyright} 2022 The Author(s) 2022. Published by Oxford University Press on behalf of the American Medical Informatics Association.",

year = "2022",

month = may,

day = "1",

doi = "10.1093/jamia/ocac002",

language = "English",

volume = "29",

pages = "983--989",

journal = "Journal of the American Medical Informatics Association : JAMIA",

issn = "1067-5027",

publisher = "Oxford University Press",

number = "5",

}

Yang, C , Kors, JA , Ioannou, S , John, LH , Markus, AF , Rekkas, A , de Ridder, MAJ , Seinen, TM , Williams, RD & Rijnbeek, PR 2022, 'Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review', Journal of the American Medical Informatics Association : JAMIA, vol. 29, no. 5, pp. 983-989. https://doi.org/10.1093/jamia/ocac002

TY - JOUR

T1 - Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review

AU - Yang, Cynthia

AU - Kors, Jan A

AU - Ioannou, Solomon

AU - John, Luis H

AU - Markus, Aniek F

AU - Rekkas, Alexandros

AU - de Ridder, Maria A J

AU - Seinen, Tom M

AU - Williams, Ross D

AU - Rijnbeek, Peter R

N1 - Funding Information: This work has received support from the European Health Data & Evidence Network (EHDEN) project. EHDEN has received funding from the Innovative Medicines Initiative 2 Joint Undertaking (JU) under grant agreement no 806968. The JU receives support from the European Union's Horizon 2020 research and innovation program and EFPIA. Publisher Copyright: © 2022 The Author(s) 2022. Published by Oxford University Press on behalf of the American Medical Informatics Association.

PY - 2022/5/1

Y1 - 2022/5/1

N2 - OBJECTIVES: This systematic review aims to provide further insights into the conduct and reporting of clinical prediction model development and validation over time. We focus on assessing the reporting of information necessary to enable external validation by other investigators.MATERIALS AND METHODS: We searched Embase, Medline, Web-of-Science, Cochrane Library, and Google Scholar to identify studies that developed 1 or more multivariable prognostic prediction models using electronic health record (EHR) data published in the period 2009-2019.RESULTS: We identified 422 studies that developed a total of 579 clinical prediction models using EHR data. We observed a steep increase over the years in the number of developed models. The percentage of models externally validated in the same paper remained at around 10%. Throughout 2009-2019, for both the target population and the outcome definitions, code lists were provided for less than 20% of the models. For about half of the models that were developed using regression analysis, the final model was not completely presented.DISCUSSION: Overall, we observed limited improvement over time in the conduct and reporting of clinical prediction model development and validation. In particular, the prediction problem definition was often not clearly reported, and the final model was often not completely presented.CONCLUSION: Improvement in the reporting of information necessary to enable external validation by other investigators is still urgently needed to increase clinical adoption of developed models.

AB - OBJECTIVES: This systematic review aims to provide further insights into the conduct and reporting of clinical prediction model development and validation over time. We focus on assessing the reporting of information necessary to enable external validation by other investigators.MATERIALS AND METHODS: We searched Embase, Medline, Web-of-Science, Cochrane Library, and Google Scholar to identify studies that developed 1 or more multivariable prognostic prediction models using electronic health record (EHR) data published in the period 2009-2019.RESULTS: We identified 422 studies that developed a total of 579 clinical prediction models using EHR data. We observed a steep increase over the years in the number of developed models. The percentage of models externally validated in the same paper remained at around 10%. Throughout 2009-2019, for both the target population and the outcome definitions, code lists were provided for less than 20% of the models. For about half of the models that were developed using regression analysis, the final model was not completely presented.DISCUSSION: Overall, we observed limited improvement over time in the conduct and reporting of clinical prediction model development and validation. In particular, the prediction problem definition was often not clearly reported, and the final model was often not completely presented.CONCLUSION: Improvement in the reporting of information necessary to enable external validation by other investigators is still urgently needed to increase clinical adoption of developed models.

UR - http://www.scopus.com/inward/record.url?scp=85128489029&partnerID=8YFLogxK

U2 - 10.1093/jamia/ocac002

DO - 10.1093/jamia/ocac002

M3 - Article

C2 - 35045179

SN - 1067-5027

VL - 29

SP - 983

EP - 989

JO - Journal of the American Medical Informatics Association : JAMIA

JF - Journal of the American Medical Informatics Association : JAMIA

IS - 5

ER -

Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this