Explaining Probability Judgment Inaccuracy: A Lens Model Extended Decomposition of the Brier Score

Identifying sources of inaccuracy in probability judgments can help correct decision errors. We combine the lens model with the Brier Score to decompose this measure of inaccuracy into (a) difficulty of judgment task, (b) outcome predictability, (c) bias, (d) inappropriate weighting of cues, (e) private information, and (f) noise. Private information refers to discriminatory power of judgments uncorrelated with observable, modeled cues. Its identification can be particularly valuable to explain inaccuracy of laypersons’ judgments about personal events, and to gauge the knowledge of experts predicting impersonal events. We illustrate this new decomposition by using it to explain the accuracy of professional forecasters in predicting economic recession and the grossly inaccurate judgments of older Americans about their longevity. In both applications, judgment difficulty makes the largest contributions to inaccuracy and its variation, although this is partially offset by outcome predictability. Inappropri-ate weighting of cues is substantial and helps explain heterogeneity in inaccuracy. In the longevity application, low discriminatory power of the judgments is partly due to insufficient responsiveness to mortality risk factors, particularly among the least educated. In both applications, noise in judgment residuals (net of predictions from cues) is an important source of inaccuracy and its variation. This noise is partially offset by nonnegligible (private) information these residuals contain on the outcome