Prospective assessment of inter-rater reliability of a neonatal adverse event severity scale

Thomas Salaets*, Thierry Lacaze-Masmonteil, Isamu Hokuto, Cheri Gauldin, Amjad Taha, Anne Smits, Liesbeth Thewissen, Ilse Van Horebeek, Armuchou Shoraisham, Khorshid Mohammad, Manami Suzuki, Shiori Komachi, Kurt Michels, Mark A. Turner, Karel Allegaert, Tamorah Lewis

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

1 Citation (Scopus)
10 Downloads (Pure)



To ensure the quality of clinical trial safety data, universal data standards are required. In 2019 the International Neonatal Consortium (INC) published a neonatal adverse event severity scale (NAESS) to standardize the reporting of adverse event (AE) severity. In this study the reliability of AE severity grading with INC NAESS was prospectively assessed in a real-world setting. 


Severity of AEs was assessed by two independent observers at each of four centers across the world. In each center two series of 30 neonatal adverse events were assessed by both observers: in a first phase with a generic (Common Terminology Criteria for Adverse Events, CTCAE) severity scale not specific to neonates, and in a second phase with INC NAESS (after a structured training). Intraclass correlation coefficients (ICC) were calculated to express inter-rater agreement in both phases, and bootstrap sampling was used to compare them. 


120 AEs were included in each of both phases. The ICC with the use of INC NAESS in phase 2 was 0.69. This represents a significant but modest improvement in comparison to the initial ICC of 0.66 in phase 1 (confidence interval of ratio of ICC in phase 2 to phase 1 = 1.005–1.146; excludes 1). The ICC was higher for those AEs for which a diagnosis specific AE severity table was available in INC NAESS (ICC 0.80). 


Good inter-rater reliability of the INC NAESS was demonstrated in four neonatal intensive care units (NICUs) across the globe. The ICC is comparable to what is reported for scales with similar purposes in different populations. There is a modest, but significant, improvement in inter-rater agreement in comparison to the naïve phase without INC NAESS. The better performance when reviewers use AE-specific NAESS tables highlights the need to expand the number of AEs that are covered by specific criteria in the current version of INC NAESS.

Original languageEnglish
Article number1237982
JournalFrontiers in Pharmacology
Publication statusPublished - 7 Sept 2023

Bibliographical note

Funding Information:
TS is supported by a travel grant of Fonds voor Wetenschappelijk Onderzoek Vlaanderen (FWO, V401622N). The research activities of ASm are supported by the Clinical Research and Education Council of the University Hospitals Leuven. TLe is funded by a NICHD NIH HHS grant (K23 HD091362).

Publisher Copyright:
Copyright © 2023 Salaets, Lacaze-Masmonteil, Hokuto, Gauldin, Taha, Smits, Thewissen, Van Horebeek, Shoraisham, Mohammad, Suzuki, Komachi, Michels, Turner, Allegaert and Lewis.


Dive into the research topics of 'Prospective assessment of inter-rater reliability of a neonatal adverse event severity scale'. Together they form a unique fingerprint.

Cite this