As discussed in this earlier post, to make meaningful inference from the ScotCET dataset, the focus needs to be shifted to the mediated effect. In such cases, following Baron and Kenny’s (1986) influential article, social scientists usually rely on structural equation modelling and the product method to derive the direct and indirect effects. Nevertheless, this approach has serious limitations that are usually overlooked in the applied literature.
The most important limitations can be summarised in four points. First, the product method postulates effect homogeneity for each individual in the population (i.e., the causal effects are constant across cases) which is an untestable and very unlikely assumption. Second, even if this effect homogeneity assumption could be loosened, the presence of an interaction between the treatment and mediator that affects the outcome cannot be handled by the product method, which makes the usual decomposition break down. Third, the product method only works if the linearity assumption holds, which allows for the additivity of effects. In many applied cases it is usually difficult to justify such a restriction, which also means that in case of non-linear (e.g., multinomial) models this method does not offer a possible solution. The fourth and final limitation has more to do with the applied literature. Even when a randomised treatment is present – such as in ScotCET – the indirect effect can still be influenced (and potentially nullified) by unmeasured third variables (i.e., confounders). (On these points see: Jo 2008; MacKinnon et al. 2013; De Stavola et al. 2015).
To improve upon the product method, causal mediation analysis has been offered as an alternative (Imai et al. 2010a 2010b 2011). Relying on the potential outcome framework this approach makes the direct and indirect effects estimable provided that the sequential ignorability assumption is satisfied. To elucidate these assumptions it is worth taking a look at a more general figure of mediation analysis:
Here T stands for the treatment, M for the mediator, and Y for the outcome. C is a vector of pre-treatment covariates, and U is a vector of unmeasured variables that might be able to affect the M-Y relationship. Pathway “c” is the direct effect, whilst the product of “a” and “b” gives the indirect effect relying on the product method.
In causal mediation analysis the sequential ignorabiltiy assumption (XY) postulates that the direct and indirect effects are estimable provided that controlling for pre-treatment covariates C, there is no unmeasured confounder for:
- The relationship between the treatment (T) and outcome (Y),
- The relationship between the mediator (M) and outcome (Y),
- The relationship between the treatment (T) and mediator (M),
- There is no post-treatment mediator-outcome confounder (L) that was affected by the treatment.
For the new decomposition and definitions of direct and indirect effects (usually referred to as “natural” effects) one needs to satisfy all four points from above. Notably, the randomisation of the treatment only satisfies 1 (pathway “c”) and 3 (pathway “a”), but not 2 (pathway “b”). Moreover, assumption 4 essentially means that there cannot be any other mediators transmitting the treatment to the outcome, which is also a very strong assumption. The next post will provide an example using ScotCET on how to derive these new natural effects.
Baron, Reuben M. and David a. Kenny. 1986. “The Moderator-Mediator Variable Distinction in Social The Moderator-Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations.” Journal of Personality and Social Psychology 51(6):1173–82.
Imai, Kosuke, Luke Keele, and Dustin Tingley. 2010a. “A General Approach to Causal Mediation Analysis.” Psychological Methods 15(4):309–34.
Imai, Kosuke, Luke Keele, Dustin Tingley, and Teppei Yamamoto. 2011. “Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies.” American Political Science Review 105(4):765–89.
Imai, Kosuke, Luke Keele, and Teppei Yamamoto. 2010b. “Identification, Inference and Sensitivity Analysis for Causal Mediation Effects.” Statistical Science 25(1):51–71.
Jo, Booil. 2008. “Causal Inference in Randomized Experiments With Mediational Processes.” Psychological Methods 13(4):314–36.
Mackinnon, David P., Yasemin Kisbu-sakarya, and Amanda C. Gottschall. 2013. “Developments in Mediation Analysis Oxford Handbooks Online Developments in Mediation Analysis.” Pp. 1–28 in Oxford Handbook of Quantitative Methods, vol. 2, edited by T. D. Little. New York: Oxford University Press.
De Stavola, Bianca L., Rhian M. Daniel, George B. Ploubidis, and Nadia Micali. 2015. “Mediation Analysis with Intermediate Confounding: Structural Equation Modeling Viewed through the Causal Inference Lens.” American Journal of Epidemiology 181(1):64–80.