Alternative Outcome Definitions and Their Effect on the Performance of Methods for Observational Outcome Studies

Background A systematic risk identification system has the potential to test marketed drugs for important Health Outcomes of Interest or HOI. For each HOT, multiple definitions are used in the literature, and some of them are validated for certain databases. However, little is known about the effect of different definitions on the ability of methods to estimate their association with medical products. Objectives Alternative definitions of HOI were studied for their effect on the performance of analytical methods in observational outcome studies. Methods A set of alternative definitions for three HOT were defined based on literature review and clinical diagnosis guidelines: acute kidney injury, acute liver injury and acute myocardial infarction. The definitions varied by the choice of diagnostic codes and the inclusion of procedure codes and lab values. They were then used to empirically study an array of analytical methods with various analytical choices in four observational healthcare databases. The methods were executed against prede Results In the three outcomes studied, different definitions of outcomes show comparable ability to differentiate true from false control cases (AUC) and a similar bias estimation. However, broader definitions generating larger outcome cohorts allowed more drugs to be studied with sufficient statistical power. Conclusions Broader definitions are preferred since they allow studying drugs with lower prevalence than the more precise or narrow definitions while showing comparable performance characteristics in differentiation of signal vs. no signal as well as effect size estimation.