Clinical application of ‘Sophia Observation withdrawal Symptoms‐Paediatric Delirium’ screening tool in Danish version: A feasibility study

Abstract Aims and objectives The aims of the present study were investigating the feasibility of: (1) using the Danish version of Sophia Observation withdrawal Symptoms‐Paediatric Delirium (SOS‐PD) screening tool in clinical practice and (2) comparing SOS‐PD performance to a child psychiatrist's assessment using the diagnostic criteria as a reference standard. Background Critically ill children risk developing delirium potentially causing discomfort and suffering. Intensive care delirium has a fluctuating course complicating detection. Systematic screening during and after intensive care is central to manage paediatric delirium. Design and methods We used a descriptive and comparative design. First aim: Bedside nurses were asked to evaluate their experience of using the SOS‐PD. Second aim: We compared the SOS‐PD performance with the child psychiatrist assessment in 50 children aged 4 weeks to 18 years. Results Nurses found the Danish version of the SOS‐PD applicable and easy to use. Of the 50 children included, 13 were diagnosed with delirium by the child psychiatrist. Consistency was found between the SOS‐PD score and the child psychiatrist's assessment (88%). We found three false‐negative and three false‐positive SOS‐PD cases. The false‐negative cases could be explained by the differences in time periods for the assessments. SOS‐PD assessments covered the past 4 h, whereas the psychiatric assessments covered the past 24 h. We assume the false‐positive cases represent an acceptable inconsistency between the two assessment methods. Conclusions The Danish version of the SOS‐PD appeared suitable for identifying paediatric delirium. Our results emphasised the importance of assessment at least once during each nursing shift to ensure delirium detection around the clock due to the fluctuating course of delirium. Relevance to clinical practice Implementing the Danish SOS‐PD may increase awareness of this critical disorder by improving systematic identification of paediatric delirium in clinical practice paving the way for improved delirium prevention and management.


INTRODUCTION
Critically ill children are at risk of developing paediatric delirium (PD), which may cause discomfort and suffering with symptoms such as disturbed sleep-wakefulness, disorientation, inattention, hallucinations, anxiety, altered behaviour and mood swings [1]. PD is associated with increased ventilator days, longer hospital stays, increased mortality and costs [2,3]. It is reasonable to assume that experiencing delirium during intensive care may contribute to the post-intensive care syndrome and delusional memories in child survivors [4,5]. Systematic PD screening during or after intensive care is needed to identify the disorder as well as monitoring the effectiveness of actions to prevent PD. The present study evaluates the feasibility of introducing a screening tool to identify PD in clinical practice.

BACKGROUND
In the American psychiatry Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-V), delirium is described as a neurocognitive disorder [1]. Delirium is manifested as an acute cerebral dysfunction with the following diagnostic criteria: disturbance in attention, awareness and cognition (memory deficit, disorientation, perception), development over a short period of time and tendency to fluctuate in severity during the course of a day [1]. Three types of delirium are described: hyperactive, hypoactive and mixed delirium [1]. The child with hyperactive delirium is characterised by turmoil, increased psychomotor activity, restlessness and combative behaviour while being insecure, anxious and vigilant [2,6]. Conversely, the child with hypoactive delirium is remarkably quiet, withdrawn, sleepy and with a nondemanding behaviour [2,6]. The mixed type comprises alternating symptoms of hyper-and hypoactive type delirium [2,6]. Diagnosis of delirium in adults focuses on cognitive changes or disruption, whereas emphasis is placed on behavioural changes in children [7,8]. The incidence of PD in critically ill children reported in other studies varies from 5% to 57% depending on associated risk factors such as age <2 years, developmental delay, cardiac surgery and awareness of the condition [9][10][11][12]. Many of the symptoms of PD overlap with other conditions often related to critical care such as pain, distress and iatrogenic withdrawal syndrome (IWS) [13]. Identifying and distinguishing between these conditions is important as they are treated differently. However, it can be challenging to distinguish these conditions without the use of validated screening tools.
At present, four tools for detection of PD have been validated in clinical practice: the paediatric Confusion Assessment Method for the ICU (p-CAM-ICU) [14]; the preschool Confusion Assessment Method for the ICU (ps-CAM-ICU) [6]; the Cornell Assessment of Paediatric Delirium (CAPD) [15] and Sophia Observation withdrawal Symptoms-Paediatric Delirium screening tool (SOS-PD) [16]. Both p-CAM-ICU and ps-CAM-ICU are assessment tools requiring patient interaction to assess delirium, as opposed to CAPD and SOS-PD that are observational screening tools based on observations on behaviour during the past 4 h [6, 14-16,]. In the present study, we evaluated the feasibility of the Danish version of the SOS-PD using a child psychiatrist's assessment as a reference standard. We have chosen the SOS-PD screening tool to ease application since it holds the advantage of providing concurrent assessment of PD and IWS, eliminating the need for using different tools for these two conditions [16,17]. Furthermore, this screening tool is based on observations during the past 4 h taking the fluctuating nature of PD into account. To our knowledge, the SOS-PD has not been evaluated in other countries than the Netherlands. Hence, in this study, we investigate the feasibility of the SOS-PD application in clinical practice to identify barriers while preparing for a study validating the SOS-PD.
The aims of the present study were investigating the feasibility of: (1) using the Danish version of Sophia Observation withdrawal Symptoms-Paediatric Delirium (SOS-PD) screening tool in clinical practice and 2) Foundation (NNF 20OC0066074) and Rigshospitalet, Copenhagen University Hospital.
Relevance to clinical practice: Implementing the Danish SOS-PD may increase awareness of this critical disorder by improving systematic identification of paediatric delirium in clinical practice paving the way for improved delirium prevention and management.

K E Y W O R D S
children, critical care nursing, family-centred care, feasibility studies, intensive care, newborn, paediatric delirium, paediatric intensive care unit, translation comparing the SOS-PD performance with a child psychiatrist's assessment using DSM-V as a reference standard.

Design
In this feasibility study, we applied a descriptive and comparative design using qualitative and quantitative methods. We used the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guideline [18], and international principles for translation and cultural adaption of patient reported outcome tools [19], including evaluation of the clinical application of the SOS-PD.

The SOS-PD screening tool used by nurses
The SOS-PD screening tool is an extended version of the 2009 Sophia observation withdrawal symptoms [20] validated at the Sophia Children's Hospital in the Netherlands to identify either PD or IWS, or both [16,20]. The target population is children from 3 months to 18 years of age. SOS-PD is an observational tool that does not require the cooperation of the child. Assessments are based on observations of the child's behaviour during the past 4 h. The tool includes 22 behavioural items whereof 10 items overlap in both conditions. The PD component consists of 17 items addressed to the nurses caring for the child, and one item addressed to the parents of the child ( Figure  1). Items receive a score if the symptom has been present at any time during the observation period. The maximum score is 17 points. The cut-off score indicating delirium is ≥4 [21]. The original Dutch SOS-PD screening tool used by Paediatric Intensive Care Unit (PICU) nurses has shown a sensitivity of 92.3% and specificity of 96.5% when compared to a child psychiatrist's diagnosis of delirium [16].
The English version of SOS-PD was translated into Danish according to principles of good practice for translation to ensure that the translation linguistically, conceptually and culturally was corresponding with the original tool [19]. Prior to translation, we obtained the permission from the original authors to translate the tool. Two authors of the present paper were certified to use the SOS-PD by the leading member of the team that developed the tool. The training session consisted of three elements: (1) a theoretical introduction to PD, (2) an instruction on how to use the SOS-PD screening tool and (3) PD assessment based on video recordings of three children. The video assessments were compared amongst the trainees and the instructor, and assessments were discussed with special focus on disagreements. Certifying the Danish authors ensured comprehension, correct use and interpretation of the SOS-PD and approval of the process and the final translation [16].

Assessment by child psychiatrist
The child psychiatrist performed a diagnostic assessment of the child using the Vanderbilt Assessment of Delirium in Infants and Children (VADIC) based on patient history from the medical record, interview with the child's parents and objective assessment of the child [8]. The VADIC is a valid bedside tool that has preserved the DSM-V criteria with paediatric specified modifiers integrated for paediatric delirium screening [8]. VADIC compasses organising behavioural observations made at the moment and during the past 24 h within six domains: (1) level of consciousness, (2) mental status and perception, (3) attention and cognition, (4) sleep-wake cycle, (5) affect and (6) language and thought, according to DSM-V criteria [1]. The VADIC tool was used to ensure consistency and structure format for identifying PD [8].

Setting
The evaluation of SOS-PD was conducted at five different units at a University Hospital in Copenhagen treating children from 4 weeks to 18 years of age: the neonatal intensive care unit (NICU), paediatric semi-intensive care unit, paediatric cardiac unit, paediatric cardiac intensive care unit and the general intensive care unit. All five units are the most specialised units in Denmark providing treatment and care for newborns and children with congenital malformations, heart diseases, trauma, respiratory distress and neurology.

Sampling and data collection
Data were collected from May to December 2020 using 2-3 available patients on one assigned weekday each week during dayshift. Regarding the first aim of the study the participants evaluating the use of SOS-PD were nurses caring for the included PICU patients on the day of the tool evaluation at the five hospital units. We conducted informal interviews with the nurses to determine their experience of the tool. Throughout the study, the PI used a logbook to record nurses' comments and reflections on the usefulness of the tool, facilitators and barriers to the progress and recorded the immediate responses from the nurses when supervising the SOS-PD assessment. The second aim of the study was to compare the SOS-PD performance used once a shift by the bedside nurse with the child psychiatrist assessment using VADIC. The children that were assessed were hospitalised 48 h or more, admitted to or transferred within the past week from an intensive care unit and lightly or non-sedated (sedation level at COMFORT behaviour score >11 [22] or Richmond Agitation Sedation Scale > −3 [23]). We did not perform delirium assessment on deeply sedated children because the basic criteria for delirium will not be present. Training of nurses and key persons in the five units was promoted by close collaboration with the management teams ( Table 1). The primary investigator (PI) identified children who met the inclusion criteria together with the unit key persons or the nurse leaders. Hereafter, the PI recruited 2-3 children and their parents to inform them of the study. All participating families received verbal and written information and gave written consent to participate. In a convenience sample, 50 patients were assessed. All assessments were performed by different bedside nurses and the child psychiatrist independently. If the bedside nurse had not completed the web-based training program or received training from the key persons, the PI performed the assessment with the bedside nurse.

Data analysis
First study aim: Data on usability of SOS-PD were bedside nurses' statements that were recorded by the PI in the logbook. The qualitative data were analysed deductively focusing on elements for further face validation as well as facilitators and barriers to the application.
Second study aim: Data compared SOS-PD and VADIC assessments. In this feasibility study, the data are not statistically significant, and we did not perform power calculations. The feasibility of the SOS-PD assessment tool was calculated from standard definitions of true positive, true negative, false-positive and false-negative cases. In addition, the initial calculation of sensitivity and specificity, as well as positive and negative predicted values were included.

Ethical considerations
The study was approved by the five management teams and the Danish Data Protection Agency (j.no.: P-2019-548). According to Danish legislation approval from the National Committee on Heath Research Ethics for this type of study is not required (reference number: 19076015).

Feasibility of SOS-PD in clinical practice
Based on comments and reflections noted by the PI in the logbook, all nurses generally found that the Danish SOS-PD was quick and easy to use and had the advantage of concurrently assessing both PD and IWS. Furthermore, the nurses found it valuable to collaborate with the parents in the assessments, because the parents had a knowledge about the child's habitual mental condition before admission. Some of the items needed further clarification because the nurses were unsure of the items 'tremors' and 'hunched shoulders'. These terms were not changed but the items were clarified in the supplementary text explaining the scoring of the items. In infants and young children <2 years of age, the nurses found challenges especially with the assessment of 'speech', 'attentiveness', 'purposeful acting' and 'hallucinations'. The text explaining item scoring for 'speech' was clarified describing that language also included the non-verbal communication and sounds, which the nurse or parent could imitate as the communication. The other terms were clarified and discussed in training sessions in how the terms could be seen in these young children.

Feasibility of SOS-PD performance
The bedside nurses and the child psychiatrist assessed 50 children aged 4 weeks to 17 years. Figure 2 shows the flowchart of included children and Table 2 shows the children's characteristics. Nurses performed 69 SOS-PD assessments in 50 children on the assigned weekdays. Many of the included children were ≤5 years of age (68%), had a cardiac diagnosis (54%) and did not require respiratory support at the assessment time (80%). Many of the SOS-PD items were comparable to the areas observed by the child psychiatrist. However, the items 'sweating', 'tremors', 'motor disturbance', 'muscle tension' and 'grimacing' were not included in the psychiatric evaluation. The child psychiatrist focused on the child's baseline level of development before hospitalisation, patient history from the hospital chart, clinical behavioural observation made at the bedside and the parents' description of the observations of their child's behaviour during the past 24 h.
The key persons were familiar with the screening tool in clinical practice after completing the training program but implementing the use of the SOS-PD around the clock was challenging. The nurses felt comfortable using the SOS-PD, and they found the assessment essential in their clinical practice. Despite this, we found that the SOS-PD was not used systematically once in every nursing shift as intended. The PI assisted the bedside nurse in 25% of the assessments in cases where the bedside nurse had not received training. We calculated the consistency between the nurses' and the psychiatrist's assessment and found agreement in 44 of 50 assessments (88%). The feasibility of SOS-PD compared to DSM-V criteria is shown in Table 2.
In all, there were six cases displaying inconsistencies between the SOS-PD screening and the reference standard being the child psychiatrist's assessments. With a SOS-PD cut-off ≥4 there were three false-negative PD assessments and three false-positive ( Table 3). The child psychiatrist identified 13 of the 50 assessed children as having a condition consistent with a PD disorder responding to a prevalence of 26%. The SOS-PD identified 10 of these ( Table  3). The inconsistencies in assessments leading to falsenegative results appeared mainly to be caused by the time scope of observation period. SOS-PD scoring was based on observations during the past 4 h in the dayshift, whereas the psychiatric assessment included the past 24 h. In the three false-negative cases, signs of delirium had been described in the patient records the previous evening and were also reported by the parents with children presenting signs of visual hallucinations. The SOS-PD score, however, had not been performed in the evening, and the next day the score was <4. In all three false-positive cases the SOS-PD score was 4, just reaching the cut-off score. We assume the false-positive cases represent an acceptable inconsistency between the two different assessments performed by different people. Ten % of assessments were conducted in newborns and children up to 2 months of age (Table 4). In all these cases there was consistency between the SOS-PD and the child psychiatrist's assessment.
The importance of parent participation in SOS-PD screening emerged as an unexpected but important finding. Both nurses and the child psychiatrist became aware that parents' contribution was critical for SOS-PD and VADIC assessments. The parents were present most of the time with their child and were alert to any changes in attention, cognition and awareness.

DISCUSSION
In this feasibility study, we evaluated the applicability of SOS-PD in clinical practice and explored disagreements between the SOS-PD and child psychiatrist's assessment as a reference standard in order to successfully adapt the Danish version for use in a Danish setting as well as identifying elements of importance for implementation T A B L E 2 Feasibility of SOS-PD compared to DSM-V criteria (n = 50) of effective PD screening. Our results indicate that implementation of the Danish version of the SOS-PD is feasible. SOS-PD seemed easy to use in clinical practice with the adaptations made based on the nurses' comments and experiences from the piloting patient cases. In general, the results of the SOS-PD assessment were consistent with the child psychiatrist assessment with good sensitivity and specificity in spite of being a novel instrument to the nurses. More importantly, we did found inconsistency in 6 cases. It became clear from the false-negative cases, that one SOS-PD assessment during the dayshift is not enough to ensure detection of PD. Systematic assessment covering 24 h is needed due to the possible fluctuation of PD. This would also be in accordance with the timespan included by the child psychiatrist. SOS-PD is intended to be applied three times a day [16], and it is actually recommended to assess for PD at least once per shift or as indicated by the child's condition [13]. In all three false-positive cases, the SOS-PD score just reached the cut-off score at 4. A larger validation study is needed to judge if the cut-off of 4 is optimal for the Danish version. Focus on PD is a new area in Danish PICUs, but awareness of and interest in this condition is increasing. Nurses are familiar with assessment tools to identify pain and performing systematic assessments are part of nurses' daily work in intensive care units in many countries [17,24]. Recent studies, however, reveal that regular monitoring for PD was applied in less than one-third of the participating PICUs [25,  26]. To remind the nurses to use the SOS-PD during every shift, magnets and action-and reminder cards were placed in visible spaces, but even so the nurses did not use the SOS-PD systematically around the clock as intended. All 50 children had at least one SOS-PD assessment from dayshift probably because the PI prompted the bedside nurse on the assigned weekday to perform the SOS-PD assessment.

SOS-PD
The nurses relied on their professional judgement and only used the tool if they suspected PD during evening and night shifts. Scepticism towards delirium screening tools is a common barrier along with knowledge deficits also described in other studies [27][28][29]. Suggestions for overcoming implementation barriers are described in several studies and includes: continuing education, flexible timing of assessment and computerised access to the screening tool [17,28,30,31]. Nurse leaders play a key role to reduce barriers in practice with directing guiding, motivation and supporting staff during the implementation [32,33]. In preparation for our future validation study, these barriers will be addressed by increased efforts by ensuring active participation of nurse managers to stimulate implementation, increased visual reminders, implementing a local guideline for PD assessment and integrating the SOS-PD in the electronic patient record to ease access and documentation. Understanding and implementing delirium monitoring and management for children require interdisciplinary contribution with collaboration amongst nurses, intensivists, paediatricians and psychiatrists. Only few physicians completed the web-based program. In this study, the focus was mostly on the nurses because they had to become familiar with the tool to be able to identify PD. Critical care nurses work in close proximity to the child, observing and providing care throughout their shift, whereby they observe the child's suffering and discomfort. This may promote motivation to learn how to identify PD in order to subsequently alleviate this discomfort. PD assessment could advantageously be an integral part of interdisciplinary daily rounds, where the healthcare professionals systematically evaluate pain, anxiety, level of sedation and delirium in the context of the child's overall condition. To this end, The Paediatric Road Map is a systematic approach to promote the monitoring of pain, anxiety, level of consciousness and delirium [34].
In the present study, the items 'purposeful acting', 'hallucinations', 'disorientation' and 'speech' were difficult for the nurses to assess in infants less than 2 years of age. It has previously been established that assessment of infants and young children is challenging [6,8]. The SOS-PD screening tool is not validated for newborns and infants less than 3 months of age. Nevertheless, we found it reasonable to include the 4-week old infants present in the neonatal unit because to our experience, these infants may display symptoms consistent with PD in older children. PD has previously been reported in NICU infants presenting classic symptoms of PD such as agitation, inconsolability, poor attention, restlessness and altered sleep-wake cycle, where these infants were identified as having delirium by a psychiatrist as well as ICU staff using the CAPD assessment tool [35]. We included five infants below 2 months of age with complete agreement between the SOS-PD and the child psychiatrist's assessment.
An unexpected finding was the importance of the parent's knowledge of their child for the nurses' and child psychiatrist's assessments because they were alert to any changes in attention, cognition and awareness. This emphasises the necessity of shared responsibility and partnership between nurses and parents in accordance with key elements in the family centred care approach [36,37]. It is described that parents want to be present and participate in the care of their child and that their child is calmer when they are involved [38,39]. Parents also feel that their active participation may help them cope better with their own stress [38]. Studies show that delirium detection by the family in adult settings seems promising because family caregivers are better able to detect changes in patient cognition and behaviour than nurses, because they know their relatives from pre-level and functioning [40][41][42]. This is most likely also true for parents of children with delirium. In a recent study it has been shown that children without a bedside parent was more likely to be screened positive for delirium [43]. Parental presence and involvement in PD detection and treatment should be explored in the future research.

Limitations
One of the strengths in this feasibility study was the collaboration with the original developers of the SOS-PD screening tool to ensure the translation process and the close collaboration with a child psychiatrist to ensure the accuracy of the translated version of SOS-PD. A limitation was the relatively small number of children on mechanical ventilation. Most of these children were deeply sedated and were excluded as they did not fulfil inclusion criteria. Another limitation was that the nurses did not use SOS-PD systematically around the clock.

CONCLUSION AND RELEVANCE TO CLINICAL PRACTICE
This study substantiated the feasibility of using the Danish version of the SOS-PD in critical care and intermediate paediatric care paving the way for systematic implementation in clinical practice. We successfully translated the screening tool into Danish and performed a preliminary evaluation of the SOS-PD. The nurses found the tool applicable and easy to use and there was a high degree of agreement between the nurses' assessments using SOS-PD and the child psychiatrist's assessment. Our results indicate that the translated version of SOS-PD is suitable for identifying PD and emphasise the importance of using the screening tool at least once during each nursing shift to ensure PD assessment around the clock as PD has a fluctuating course. Further studies are needed to validate the SOS-PD and to improve prevention and treatment of PD through non-pharmacological interventions including parental participation.