Abstract
BACKGROUND: Stability in baseline risk and estimated predictor effects both geographically and temporally is a desirable property of clinical prediction models. However, this issue has received little attention in the methodological literature. Our objective was to examine methods for assessing temporal and geographic heterogeneity in baseline risk and predictor effects in prediction models.
METHODS: We studied 14,857 patients hospitalized with heart failure at 90 hospitals in Ontario, Canada, in two time periods. We focussed on geographic and temporal variation in baseline risk (intercept) and predictor effects (regression coefficients) of the EFFECT-HF mortality model for predicting 1-year mortality in patients hospitalized for heart failure. We used random effects logistic regression models for the 14,857 patients.
RESULTS: The baseline risk of mortality displayed moderate geographic variation, with the hospital-specific probability of 1-year mortality for a reference patient lying between 0.168 and 0.290 for 95% of hospitals. Furthermore, the odds of death were 11% lower in the second period than in the first period. However, we found minimal geographic or temporal variation in predictor effects. Among 11 tests of differences in time for predictor variables, only one had a modestly significant P value (0.03).
CONCLUSIONS: This study illustrates how temporal and geographic heterogeneity of prediction models can be assessed in settings with a large sample of patients from a large number of centers at different time periods.
| Original language | English |
|---|---|
| Pages (from-to) | 12 |
| Journal | Diagnostic and Prognostic Research |
| Volume | 1 |
| DOIs | |
| Publication status | Published - 13 Apr 2017 |
Bibliographical note
Funding:This study was supported by the Institute for Clinical Evaluative Sciences (ICES),
which is funded by an annual grant from the Ontario Ministry of Health
and Long-Term Care (MOHLTC). The opinions, results, and conclusions
reported in this paper are those of the authors and are independent from
the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended
or should be inferred. This research was supported by an operating grant from
the Canadian Institutes of Health Research (CIHR) (MOP 86508). Dr. Austin
is supported in part by a Career Investigator award from the Heart and Stroke Foundation. Dr. Lee is supported by a Clinician-Scientist award from
the CIHR. Dr. Steyerberg and Mr. Van Klaveren are supported in part by a U
award (U01NS086294, value of personalized risk information). Mr. van Klaveren
and Dr. Vergouwe are supported in part by the Netherlands Organisation for
Scientific Research (grant 917.11.383). The Enhanced Feedback for Effective
Cardiac Treatment (EFFECT) data used in the study was funded by a CIHR
Team Grant in Cardiovascular Outcomes Research (Grant numbers CTP
79847 and CRT43823).