Update of the CHIP (CT in Head Injury Patients) decision rule for patients with minor head injury based on a multicenter consecutive case series

Objective: To update the existing CHIP (CT in Head Injury Patients) decision rule for detection of (intra)cranial ﬁndings in adult patients following minor head injury (MHI). Methods: The study is a prospective multicenter cohort study in the Netherlands. Consecutive MHI patients of 16 years and older were included. Primary outcome was any (intra)cranial traumatic ﬁnding on computed tomography (CT). Secondary outcomes were any potential neurosurgical lesion and neuro-surgical intervention. The CHIP model was validated and subsequently updated and revised. Diagnostic performance was assessed by calculating the c-statistic. Results: Among 4557 included patients 3742 received a CT (82%). In 383 patients (8.4%) a traumatic ﬁnding was present on CT. A potential neurosurgical lesion was found in 73 patients (1.6%) with 26 (0.6%) patients that actually had neurosurgery or died as a result of traumatic brain


a b s t r a c t
Objective: To update the existing CHIP (CT in Head Injury Patients) decision rule for detection of (intra)cranial findings in adult patients following minor head injury (MHI). Methods: The study is a prospective multicenter cohort study in the Netherlands. Consecutive MHI patients of 16 years and older were included. Primary outcome was any (intra)cranial traumatic finding on computed tomography (CT). Secondary outcomes were any potential neurosurgical lesion and neurosurgical intervention. The CHIP model was validated and subsequently updated and revised. Diagnostic performance was assessed by calculating the c-statistic. Results: Among 4557 included patients 3742 received a CT (82%). In 383 patients (8.4%) a traumatic finding was present on CT. A potential neurosurgical lesion was found in 73 patients (1.6%) with 26 (0.6%) patients that actually had neurosurgery or died as a result of traumatic brain injury. The original CHIP underestimated the risk of traumatic (intra)cranial findings in low-predicted-risk groups, while in highpredicted-risk groups the risk was overestimated. The c-statistic of the original CHIP model was 0.72 (95% CI 0.69-0.74) and it would have missed two potential neurosurgical lesions and one patient that underwent neurosurgery. The updated model performed similar to the original model regarding traumatic (intra)cranial findings (c-statistic 0.77 95% CI 0.74-0.79, after crossvalidation c-statistic 0.73). The

Introduction
Minor head injury (MHI) is a common and increasing cause of emergency department (ED) visits worldwide. [1][2][3] With aging of the population it is expected that the burden caused by MHI will continue to rise in the next decades. The vast majority ( > 90%) of patients with MHI will have no (intra)cranial traumatic lesions. [4 , 5] Nonetheless, (intra)cranial traumatic lesions can result in severe disability or death and therefore require clinical observation and a small percentage needs neurosurgical intervention. This study aims to provide a method to improve selection of patients that require a head computed tomography (CT) to identify traumatic lesions.
Currently the most used technique to rule out traumatic findings after MHI is CT. CT is widely available and the fraction of patients receiving a CT for MHI has increased significantly in the last decades. [6 , 7] The use of CT has many advantages because it is fast and reliable. However, its increasing use in MHI also has several important disadvantages. First, a CT exposes the patient to radiation risks. [8] Second, a CT is costly and should, in the light of ever-expanding healthcare costs, only be used when necessary. Last but not least, performing more diagnostic procedures such as CT may lead to prolonged ED throughput times and thus result in ED-crowding. [9] With increasing ED visits for MHI it is more important than ever to identify those patients that will benefit from a CT.
To enhance selective use of head-CT several decision rules for MHI have been developed. Worldwide the most used decision rules are probably the Canadian CT Head Rule (CCHR) and the New Orleans Criteria (NOC). [10 , 11] Both CCHR as NOC are only applicable to patients with loss of consciousness, post traumatic amnesia or confusion. However, most patients with MHI do not experience any of these and (intra)cranial findings can be present even in the absence of these risk factors. [12 , 13] Therefore, another decision rule was developed in four level one trauma centers in the Netherlands in the beginning of this century. This rule is applicable to all ED patients with MHI, the CT in Head Injury Patients (CHIP) rule. [4] The ACEP (American College of Emergency Physicians) clinical policy for neuroimaging in MHI includes recommendations from the CHIP study for patients without loss of consciousness or posttraumatic amnesia. [14] We recently validated the CHIP-rule and compared it to the NOC and the CCHR. [15] In line with an aging population, the patient population in this validation-study differed substantially from the original CHIP, NOC and CCHR studies. [1 , 4 , 10 , 11 , 15] The population was older and trauma was more often caused by ground level falls. In this validation-study sensitivity and specificity for any traumatic finding were 94% and 22% for the CHIP rule; 99% and 4% for the NOC and 80% and 44% for the CCHR. Based on these results we concluded that the CHIP rule performed well compared to several other prediction rules in terms of a proper balance between specificity and sensitivity. Nonetheless, we also conclude that there is room for improvement of the CHIP because the sensitivity for detecting (potential) neurosurgical lesions was less than 100%. [15] Given the potential for improvement of the CHIP as demonstrated in the validation study, the changing demographic characteristics of MHI patients and the fact that the CHIP was developed in level one trauma centers only, there seems to be need for an update of the CHIP. Therefore, the aim of the current study is to update and improve the CHIP decision rule for detection of (intra)cranial findings following MHI. The primary and secondary outcomes for this study are any traumatic intracranial lesion and (potential) neurosurgical lesions respectively.

Study design and setting
This prospective, multicenter cohort study was conducted in the Netherlands, data were collected between March 1st 2015 and January 1st 2017. Three level 1, one level 2 and two level three EDs participated in the study. [16] The participating study sites have on average approximately 33,0 0 0 ED visits (range 15,512-54,216 in 2016). The current study is a secondary analysis of data collected for the original study published in 2018 in which 9 EDs participated, in the current study only the 6 EDs that included patients with and without CT participated ( Fig. 1 ). [15] Selection of participants Consecutive patients of 16 years and older with MHI who arrived at one of the participating EDs within 24 h after blunt trauma to the head were included. MHI was defined as: Any trauma to the head, other than superficial injuries to the face and: -Glasgow Coma Scale (GCS) score 13-15 at first examination -Loss of consciousness (not required): no more than 30 min -Posttraumatic amnesia (not required): no more than 24 h Patients who were transferred from another hospital were excluded. Clinical data concerning risk factors as used in the CHIPrule and additional risk factors were collected (supplementary Table 1). [7] Outcomes Similar to the original CHIP, the primary outcome was any (intra)cranial traumatic finding on CT, defined as: subdural hematoma, epidural hematoma, subarachnoid hemorrhage, hemorrhagic contusion, non-hemorrhagic contusion, diffuse axonal injury, intraventricular hemorrhage, and skull fracture. The secondary outcome was any potential neurosurgical lesion, which was defined as an (intra)cranial traumatic finding on CT which could lead to a neurosurgical intervention or death. [15] The following traumatic findings were labelled as potential neurosurgical lesions: epidural hematoma, large acute subdural hematoma (mass), large contusion(s) (mass), depressed skull fracture, and any lesion with midline shift or herniation. Another secondary outcome was neurosurgical intervention for traumatic skull or brain injury within 30 days patients that died as a result of their traumatic brain injury were included in this outcome regardless whether they actually underwent neurosurgery.
A prerequisite of the (updated) model was not to miss any potential neurosurgical findings.

Study procedures and analysis
We described study procedures and data management in detail elsewhere. [15] Consecutive eligible patients were included by trained researcher physicians, who did not personally interview the patients. Clinical data were collected before diagnostic tests as far as possible by using forms the clinicians could fill in for each patient. The head CT scans were performed according to a routine trauma protocol at each hospital.
Sample size was based on 20 eligible variables in multivariable logistic regression. Per variable at least 10 events of the primary outcome measure were required. Based on earlier research the estimated incidence of traumatic findings on CT was 7.4% [7] , hence at least 2703 scanned patients had to be included.
In accordance to the original CHIP-study, we imputed loss of consciousness and posttraumatic amnesia as present if data was missing or unknown. Other missing data were assumed to be missing at random. We imputed missing data based on all the variables using "Multivariate Imputation by Chained Equations" (MICE) in R. Outcomes could not be observed in patients without CT. Therefore, we imputed the expected outcomes based on their risk factors with multiple imputation, acknowledging the uncertainty of imputations by performing the imputation multiple times ( n = 5). [17] The imputed missing data are the result of a combination of these five imputed datasets. Baseline and outcome are first reported without imputation mentioning any missing data. We used data with imputed outcomes for the primary analysis, similar to our previous study. [15] We performed a sensitivity analysis by including scanned patients only (without outcome imputation). Anal-yses were performed using IBM Statistical Package for Social Sciences version 24 and R foundation for statistical computing software, version 3.3.2.
Institutional ethics and research board approval was obtained, and informed consent was waived.

Validation and updating
Model validation, updating and revision were based on the methodology as described by Steyerberg. [18] First, we validated the original CHIP-rule. The predicted risk of any (intra)cranial traumatic finding was calculated for each patient using the original risk factors, regression coefficients and intercept. We calculated the observed frequency of any (intra)cranial traumatic finding in our dataset and present this in a calibration plot. A locally weighted regression curve (LOESS) was used in the calibration plot. The default setting of the val.prob.ci.2 function in R was used to create the calibration plot. [19] Updating of the CHIP decision model was performed based on the difference in fit of the CHIP-model and a newly fitted model in the current data. [18] To update the CHIP we performed re-calibration as a first step. The intercept was updated to correct a potential deviation in 'calibration-in-the-large'. Calibration-in-the-large refers to whether the mean observed outcome is equal to the mean predicted outcome. The second step was to update both the intercept and the overall calibration slope. The third step was to re-estimate the intercept and the regression coefficients of the original CHIP predictors in the study data.

Model revision
In the next steps the model was extended with new predictors and existing predictors with limited predictive value were eliminated. We assessed performance by calculating the area under the receiver operating characteristic curve (c-statistic). Calibration was assessed by plotting the observed proportions versus predicted chances of the primary outcome (calibration plot). A locally weighted regression curve (LOESS) was used in the calibration plot.
To improve the performance of the model in future populations, we multiplied the regression coefficients by a shrinkage factor obtained using bootstrapping. The updated model (without shrinkage factor) was cross-validated six times by re-estimating the intercept and regression coefficients in five centers and testing it in the sixth center. We present the validated c-statistics in a forest plot.

Results
For this study we included 4557 consecutive eligible MHI patients during the study period ( Fig. 1 ). Patient characteristics are summarized in Table 1 and supplementary Table 2. Compared to the original CHIP-study the current study population was older (mean age 53 versus 41 years) and more often female (42% versus 28%). Regarding trauma mechanism more injuries were the result of ground level falls (37% versus 22%) and less injuries were the result of assaults (15% versus 24%). [20] Of the 4557 included patients 3742 received a CT (82%). Compared to patients with CT, those without CT were on average younger (36 versus 57 years) and almost all of them had a GCS of 15 (99%). According to the CHIP-rule 3412 (75%) patients should have received a CT because of a predicted risk of ≥3% for traumatic (intra)cranial findings ( Table 2 ). [4] In 383 of 4557 patients (8.4% of all patients; 10.2% of all scanned patients) a traumatic (intra)cranial finding was present on CT (supplementary Table 3). A potential neurosurgical lesion was found in 73 patients (1.6%) with 18 (0.4%) undergoing neurosurgery. In total 1511 patients (33%) were hospitalized for any cause. The vast majority of patients ( n = 340, 89%) with traumatic findings on head-CT were hospitalized. In total 32 patients (0.7%) died during their hospital admission, in 11 patients (0.2%) this was a result of their traumatic brain injury. In total 26 patients (0.6%) underwent neurosurgery or died as a result of traumatic brain injury.
Validation Fig. 2 shows observed frequency of traumatic (intra)cranial findings in our population compared with the predictions according to the CHIP-model. In the low-predicted-risk patients, the original CHIP slightly underestimated the risk, while in the high-predictedrisk patients the model overestimated the risk. By applying the original CHIP-rule 30 traumatic findings would have been missed, including two potential neurosurgical lesions and one neurosurgical intervention. None of the fatal traumatic lesions was missed by the original CHIP. In total 1145 patients (25%) had no indication for CT according to the original CHIP (at a cut-off value of 3% predicted-risk). The sensitivity of the original CHIP for any traumatic lesion was 93% (95% CI 90-95%) and the specificity was 27% (95% CI 26-28%). Sensitivity and specificity for potential neurosurgical lesions were 97% (95% CI 90-100%) and 25% (95% CI 24-27%) respectively. Sensitivity and specificity for neurosurgical intervention or death were 96% (95% CI 80-100%) and 25% (95% CI 24-27%).

Updating
The overall observed frequency of traumatic (intra)cranial findings was slightly lower in our population (8.9% 1 ) compared to the CHIP predicted frequency (9.4%) ( p < 0.001). To correct for this "calibration in the large" the intercept was adjusted.
After that, we refitted the regression slope, the new calibration slope ( β overall ) was significantly steeper in the updated model compared to the original model ( p < 0.001). This adjustment would increase sensitivity to 97%, but at the cost of a decline in specificity to 11% (at a cut-off value of 3% predicted-risk). Next, we re-estimated regression coefficients of original risk factors in the current dataset. Some regression coefficients were similar in the validation data and the CHIP-model, others differed and two (use of anticoagulants and ejection from vehicle) had a negative regression coefficient in our dataset. Because we consider a protective effect of risk factors clinically implausible we omit these predictors from the updated model. (supplementary Table 4)

Model extension
Several updated models have been considered of which the model in Table 3 showed the best performance in terms of cstatistic and calibration ( Table 3 and Fig. 3 ). All selected variables showed significant effects ( p < 0.05). The c-statistic for any traumatic finding was 0.77 (95% CI 0.74-0.79). For potential neurosurgical lesions and for neurosurgical intervention lesions the cstatistic was 0.87 (95% CI 0.84-0.91) and 0.92 (95% CI 0.86-0.98) respectively. Table 4 lists variables included in the original CHIP versus variables included in the updated CHIP.
At a cut-off value for CT of 3% predicted-risk of any traumatic finding , similar to original CHIP, the sensitivity of the updated CHIP was 92% (95% CI 89-94%) and the specificity was 27% (95% CI 26-28%) ( Table 5 ). Sensitivity and specificity over a range of cut-off values are shown in supplementary Table 5. Sensitivity and specificity for potential neurosurgical lesions at a cut-off value for CT of 3% predicted-risk of any traumatic finding were 100% (95% CI 95- Table 4 Risk factors included in the original versus the updated CHIP.
Internal validation of the updated model using bootstrapping indicated optimism for the c-statistic, which we expected to decrease from 0.77 to 0.76 for any traumatic (intra)cranial finding. Internal validation using crossvalidation per center would decrease the c-statistic from 0.77 to 0.73 (supplementary Figs. 1 and 2). To correct for optimism penalized beta-coefficients were calculated ( Table 3 ).
A sensitivity analysis only including the 3742 scanned patients showed similar results for the updated CHIP. The c-statistic for any traumatic finding was 0.76 (95% CI 0.73-0.78). The c-statistic for potential neurosurgical lesions and neurosurgical intervention was 0.85 (0.81-0.89) and 0.90 (0.84-0.97) respectively. At a cut-off of 3% predicted-risk 16 traumatic (intra)cranial findings were missed of which none was a potential neurosurgical lesion or needed neurosurgical intervention (sensitivity 96%; specificity 34%).

Discussion
The aim of this study was to update the CHIP decision rule, this was done in a large multicenter study in a contemporary Dutch cohort. The original CHIP-model underestimated the risk of traumatic (intra)cranial findings in low-predicted-risk patients, while in high-predicted-risk patients the risk was overestimated. The updated model performed better over a wide range of predicted risks.
The updated model uses three variables less than the original CHIP-model (12 versus 15) which makes it easier to use ( Table 4 ). The c-statistic for any traumatic finding would improve from 0.72 to 0.77. However, it should be noted that internal val-idation using crossvalidation per center would decrease the cstatistic from 0.77 to 0.73. From the calibration plot it can be concluded that especially in the low-predicted-risk groups the updated model performs better than the original. Performance in these low-predicted-risk groups is most important because the high-predicted-risk groups will be scanned regardless of the exact predicted risk. Probably even more important, in contrast to the original CHIP, the updated CHIP would not miss any potential neurosurgical lesions or patients that actually underwent neurosurgery. Compared to the original CHIP-study potential neurosurgical lesions have been added as secondary outcome measure besides actual neurosurgical intervention. Neurosurgical intervention or death is rare in MHI patients and the decision to operate a patient is surgeon and country dependent. [21] Nonetheless nobody wants to miss a traumatic epidural hematoma or a large acute subdural hematoma, therefore the term potential neurosurgical lesion was introduced to more objectively identify the traumatic findings that definitely should not be missed. Hence, the largest gain of the updated model compared to the original CHIP is better identification of patients with (potential) neurosurgical lesions.
In the original CHIP-study a cut-off value of 3% predicted-risk for any traumatic finding for performing a CT is used. This rather arbitrary threshold is used in this update study as well. Nevertheless, one could argue that a different cut-off value can be more suitable depending on setting and preferences. For cut-off levels up to 3.5% and 6.0% predicted risk for any traumatic finding sensitivity for respectively potential neurosurgical lesions and actual neurosurgical intervention remained 100% in our study sample.
A striking difference between the original CHIP and this update is that the use of anticoagulants is no longer found to be a predictor of traumatic (intra)cranial findings, neither in univariable nor in multivariable analysis. Although it is impossible to establish the exact cause of this surprising change there are some possible explanations. First anticoagulants may be a smaller risk factor than previously thought. There are only few studies that have established the risk of anticoagulant therapy for traumatic intracranial hemorrhage in MHI. [22 , 23] A recent systematic review found a pooled incidence of traumatic findings in MHI patients that used anticoagulants of 8.9%. [22] However, there was a large variation and in the two largest studies in the review this incidence was only 4%. A second reason for the difference could be that referral patterns have changed. Possibly patients on anticoagulant therapy are referred to the ED for less severe trauma than patients without anticoagulant therapy. This potential difference was nonetheless not reflected in the multivariable analysis. Finally we do not know how well anticoagulants were used, it is known that patients on anticoagulants frequently have a sub-therapeutic INR. [24] However, although anticoagulant use was not a risk factor for traumatic findings in the current study, a low threshold for scanning these patients should be considered in our opinion because traumatic findings may have a worse outcome in the presence of anticoagulant use. [25][26][27] Scanning all patients on anticoagulant therapy would (at a 3% predicted-risk scanning-threshold) lead to 81 extra CTs and a reduction of two patients with missed traumatic findings (sensitivity 92%; specificity 25%).
In contrast to the original CHIP-rule we choose to present the detailed results only, the updated decision rule will be integrated into an easy to use app. A simplified decision rule is less reliable and not necessary anymore because everybody uses smart phones and electronic patient records are widespread.
Future research is needed to externally validate this updated CHIP decision rule. Until now the CHIP-model has only been validated in the Netherlands. To increase generalizability validation data should preferably also be collected in other countries.
A limitation of this study is that not all consecutive MHI patients received a CT. This is a result of the current Dutch guidelines for patients with MHI [28] . Patients that were not scanned could possibly have had traumatic findings that would have been missed. To anticipate these possible false negatives, the outcomes of these patients were imputed. Because of different scanning rates in hospitals all different risk profiles of patients were present in the non imputed dataset. Nonetheless, differential patterns of missing data may introduce unknown biases despite multiple imputation.
Study forms were filled in by clinicians as part of their routine clinical care this could have caused missing data. Although we are not aware of any missed inclusions in the study and inclusions were checked on a daily basis we cannot rule out that possible eligible patients could have been missed.
There is some heterogeneity in the results across different study sites, this means that the performance of the model may be overstated in some sites and understated at other sites. Besides this the reported c-statistics and estimates of sensitivity and specificity are based on the selection of centers and patients in our study. To reduce optimism we used shrinkage and cross validation across centers, nonetheless the reported c-statistics and sensitivity and specificity will still be biased. Therefore external validation of the results is needed.
The CHIP-rule predicts the presence or absence of traumatic findings on CT. Nonetheless, the real outcome of interest is the long-term clinical outcome which was not assessed in the current study.
Because there was no follow-up for discharged patients with a negative CT (or without CT) it is possible that some of these patients would have developed traumatic findings on consecutive scans. However development of an intracranial lesion after a normal CT is rare. [29] In summary use of the updated CHIP decision rule should be considered in patients with MHI. Compared to the original CHIP the updated rule seems to be better able to identify patients with (potential) neurosurgical lesions without increasing the CT rate. In the current study anticoagulant use was not identified as independent risk factor for traumatic findings. Nonetheless a low threshold for scanning these patients is advised because of potentially worse outcome of traumatic intracranial hemorrhage in the presence of anticoagulant use. Future research is needed to externally validate the updated CHIP decision rule.

Declaration of Competing Interest
MGMH and DWJD were the principal investigators of the CHIP (CT in head injury patients) development study; MGMH reports the following interests: royalties from Cambridge University press for textbook on Medical Decision Making; EIBIR Scientific Advisory Board member; ESR iGuide Subcommittee Chair. CLB and KJ participated in a biomarker study sponsored by Roche Diagnostics. None of the authors has financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

Grant
Author CLB received a grant from the St Jacobus Foundation, which is a Dutch non-profit organization that supports research. The funder played no role in the design, analysis or presentation of the study.