Aligning Estimands, Identification, and Conclusions

Presented at the Causal Data Science Meeting (with Graham Goff).

We introduce a simple, reproducible workflow to align regressions with an estimand and the identification it requires. The workflow entails four steps: (1) declare an estimand; (2) draw a Directed Acyclic Graph (DAG); (3) classify control variables by role; and (4) estimate models using minimal and canonical adjustment sets. We implement the workflow in a user-friendly R package, DAGassist. To illustrate its utility, we examine all 2024 American Political Science Review (APSR) articles using observational regressions with controls. Incorporating author feedback on our DAGs, we do not find any colliders. However, 64% of articles include non-admissible mediators or their descendants, and 21% of articles interpret them without qualifications. Removing non-admissible variables also changes results: 19% of articles lose statistical significance, and average standard errors fall by 4%. These rates likely constitute lower bounds given APSR’s flagship status. In turn, drawing assumptions before conclusions with DAGassist can improve empirical practice.