TY - JOUR
T1 - Prediction accuracy of discrete choice experiments in health-related research
T2 - a systematic review and meta-analysis
AU - Zhang, Ying
AU - Anh Ho, Thi Quynh
AU - Terris-Prestholt, Fern
AU - Quaife, Matthew
AU - de Bekker-Grob, Esther
AU - Vickerman, Peter
AU - Ong, Jason J.
N1 - Publisher Copyright: © 2024
PY - 2025/1
Y1 - 2025/1
N2 - Background: Discrete choice experiments (DCEs) are increasingly used to inform the design of health products and services. It is essential to understand the extent to which DCEs provide reliable predictions outside of experimental settings in real-world decision-making situations. We aimed to compare the prediction accuracy of stated preferences with real-world choices, as modelled from DCE data. Methods: We searched six databases for health-related studies that used DCE to assess external validity and reported on predicted versus real-world choices, up to July 2024. A generalised linear mixed model was used for a meta-analysis to jointly pool the sensitivity and specificity. Heterogeneity was assessed using the I2 statistic, and sources of heterogeneity using meta-regression. This study is registered with PROSPERO (CRD42023451545). Findings: We identified 14 relevant studies, of which 10 were included in the meta-analysis. Most studies were conducted in high-income countries (11/14, 79%) from the European region (9/14, 64%) and analysed using mixed logit models (5/14, 36%). Pooled sensitivity and specificity estimates were 89% (95% CI:77–95, I2 = 97%) and 52% (95% CI:32–72, I2 = 95%), respectively. The area under the SROC curve (AUC) was 0.81 (95% CI:0.77–0.84). Our meta-regression found that DCEs for prevention-related choices had higher sensitivity than treatment-related choices. DCEs conducted under clinical settings and analysed using the heteroskedastic multinomial logit model, incorporating systematic preference heterogeneity and random opt-out utility, had higher specificity than non-clinical settings and alternative models. Interpretation: DCEs are valuable for capturing health-related preferences and possess reasonable external validity to predict health-related behaviours, particularly for opt-in choices. Contextual factors (e.g., type of intervention, study setting, analysis method) influenced the predictive accuracy. Funding: JJO is supported by an Australian National Health and Medical Research Council Emerging Leadership Investigator Grant ( GNT1193955). EBG is supported by the Dutch Research Council (NWO-Talent-Scheme-Vidi-Grant No, 09150171910002). YZ is supported by an Australian Government Research Training Program (RTP) scholarship.
AB - Background: Discrete choice experiments (DCEs) are increasingly used to inform the design of health products and services. It is essential to understand the extent to which DCEs provide reliable predictions outside of experimental settings in real-world decision-making situations. We aimed to compare the prediction accuracy of stated preferences with real-world choices, as modelled from DCE data. Methods: We searched six databases for health-related studies that used DCE to assess external validity and reported on predicted versus real-world choices, up to July 2024. A generalised linear mixed model was used for a meta-analysis to jointly pool the sensitivity and specificity. Heterogeneity was assessed using the I2 statistic, and sources of heterogeneity using meta-regression. This study is registered with PROSPERO (CRD42023451545). Findings: We identified 14 relevant studies, of which 10 were included in the meta-analysis. Most studies were conducted in high-income countries (11/14, 79%) from the European region (9/14, 64%) and analysed using mixed logit models (5/14, 36%). Pooled sensitivity and specificity estimates were 89% (95% CI:77–95, I2 = 97%) and 52% (95% CI:32–72, I2 = 95%), respectively. The area under the SROC curve (AUC) was 0.81 (95% CI:0.77–0.84). Our meta-regression found that DCEs for prevention-related choices had higher sensitivity than treatment-related choices. DCEs conducted under clinical settings and analysed using the heteroskedastic multinomial logit model, incorporating systematic preference heterogeneity and random opt-out utility, had higher specificity than non-clinical settings and alternative models. Interpretation: DCEs are valuable for capturing health-related preferences and possess reasonable external validity to predict health-related behaviours, particularly for opt-in choices. Contextual factors (e.g., type of intervention, study setting, analysis method) influenced the predictive accuracy. Funding: JJO is supported by an Australian National Health and Medical Research Council Emerging Leadership Investigator Grant ( GNT1193955). EBG is supported by the Dutch Research Council (NWO-Talent-Scheme-Vidi-Grant No, 09150171910002). YZ is supported by an Australian Government Research Training Program (RTP) scholarship.
UR - http://www.scopus.com/inward/record.url?scp=85211985096&partnerID=8YFLogxK
U2 - 10.1016/j.eclinm.2024.102965
DO - 10.1016/j.eclinm.2024.102965
M3 - Article
C2 - 39791109
AN - SCOPUS:85211985096
SN - 2589-5370
VL - 79
JO - EClinicalMedicine
JF - EClinicalMedicine
M1 - 102965
ER -