Causal language jumps and non-alignments between clinical practice guidelines and original studies: a systematic evaluation of diabetes guidelines and their cited evidence

OBJECTIVES: Clinical practice guidelines are designed to guide clinical practice and often make causal claims when making recommendations. Sometimes, guidelines make or require stronger causal claims than supplied in the original studies, a phenomenon we call 'causal language jump'. We aimed to evaluate the strength of expressed causation in guidelines and the evidence they reference to assess the pattern of jumps, taking diabetes as an illustrative example.

DESIGN: This is a systematic evaluation of guidelines and original studies cited by them, using scoping review design with deviations.

DATA SOURCE: Randomly sampled 300 guideline statements (narrative sentences describing evidence to support recommendations) from four selected diabetes guidelines.

ELIGIBILITY CRITERIA: The eligible guidelines should focus on non-pharmacological treatments or preventive strategies for adult type 2 diabetes mellitus management and related complications. The eligible action recommendations and guideline statements should intend to support non-pharmacological treatments or preventive strategies of type 2 diabetes or in a general diabetic context.

DATA EXTRACTION AND SYNTHESIS: We rated the causation strength in the statements and the dependence on causation in recommendations supported by these statements using existing scales. Among the causal statements, the cited original studies were similarly assessed. We then evaluated jumps by checking if the causal claims in guideline statements were stronger than in original studies, and if the causation-dependence in guideline recommendations was stronger than supplied in guideline statements. We also assessed how well they report target trial emulation (TTE) components as a proxy for reliability.

RESULTS: Of the 300 statements, 114 (38.0%) were causal, and 76 (66.7%) expressed strong causation. 27.2% (31/114) of causal guideline statements stated stronger causation than any of their references and demonstrated 'causal language jump'; 34.9% (29/83) of guideline recommendations required stronger causation than provided in statements. Of the 53 eligible studies for TTE rating, most did not report treatment assignment and causal contrast in detail. The prevalence of these jumps could be partially attributed to the suboptimal use of causal and associational words.

CONCLUSIONS: Causal language jumps were common among diabetes guidelines. While these jumps are sometimes inevitable, they should always be justified by good causal inference practices.