Automatic detection of actionable findings and communication mentions in radiology reports using natural language processing

Jacob J. Visser*, Marianne de Vries, Jan A. Kors

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

2 Citations (Scopus)
28 Downloads (Pure)

Abstract

Objectives: To develop and validate classifiers for automatic detection of actionable findings and documentation of nonroutine communication in routinely delivered radiology reports. Methods: Two radiologists annotated all actionable findings and communication mentions in a training set of 1,306 radiology reports and a test set of 1,000 reports randomly selected from the electronic health record system of a large tertiary hospital. Various feature sets were constructed based on the impression section of the reports using different preprocessing steps (stemming, removal of stop words, negations, and previously known or stable findings) and n-grams. Random forest classifiers were trained to detect actionable findings, and a decision-rule classifier was trained to find communication mentions. Classifier performance was evaluated by the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. Results: On the training set, the actionable finding classifier with the highest cross-validated performance was obtained for a feature set of unigrams, after stemming and removal of negated, known, and stable findings. On the test set, this classifier achieved an AUC of 0.876 (95% CI 0.854–0.898). The classifier for communication detection was trained after negation removal, using unigrams as features. The resultant decision rule had a sensitivity of 0.841 (95% CI 0.706–0.921) and specificity of 0.990 (95% CI 0.981–0.994) on the test set. Conclusions: Automatic detection of actionable findings and subsequent communication in routinely delivered radiology reports is possible. This can serve quality control purposes and may alert radiologists to the presence of actionable findings during reporting. Key Points: • Classifiers were developed for automatic detection of the broad spectrum of actionable findings and subsequent communication mentions in routinely delivered radiology reports. • Straightforward report preprocessing and simple feature sets can produce well-performing classifiers. • The resultant classifiers show good performance for detection of actionable findings and excellent performance for detection of communication mentions.

Original languageEnglish
Pages (from-to)3996-4002
Number of pages7
JournalEuropean Radiology
Volume32
Issue number6
Early online date6 Jan 2022
DOIs
Publication statusPublished - Jun 2022

Bibliographical note

Funding Information:
This study has received funding from the Stichting Kwaliteitsgelden Medisch Specialisten (Foundation Quality Funds Medical Specialists) (grant number 45368564).

Publisher Copyright:
© 2021, The Author(s), under exclusive licence to European Society of Radiology.

Fingerprint

Dive into the research topics of 'Automatic detection of actionable findings and communication mentions in radiology reports using natural language processing'. Together they form a unique fingerprint.

Cite this