I have not been blogging as diligently lately, and one of the reasons for this is that I have been a bit preoccupied with writing resubmissions. As part of this process, I had to revisit the issue of treatment effect consistency which I discussed last summer. This post discusses an alternative way of assessing and quantifying whether treatment effect (in)consistency is present, using methods adopted from meta-analysis.
This post should be considered as an addendum to this previous one that discussed causal mediation analysis with a single mediator. That post ended with the argument that causal evidence could be found that procedural justice indeed mediated the effect of the treatment (previous experiences with the police) towards the outcome (normative alignment with the police). This post will discuss two sensitivity analysis techniques which assess this finding’s robustness to unmeasured confounding. Continue reading “ScotCET and sensitivity analysis for causal mediation analysis with a single mediator”
As discussed in this earlier post, to make meaningful inference from the ScotCET dataset, the focus needs to be shifted to the mediated effect. In such cases, following Baron and Kenny’s (1986) influential article, social scientists usually rely on structural equation modelling and the product method to derive the direct and indirect effects. Nevertheless, this approach has serious limitations that are usually overlooked in the applied literature. Continue reading “ScotCET and causal mediation analysis with a single mediator 2. – Issues with the product method and the sequential ignorability assumption”
This post continues the discussion of the Scottish Community Engagement Trial (ScotCET). In four posts (you can find them here, here, here, and here) I discussed the apparent failure of implementation over the summer. I concluded that despite the unexpected direction of the findings (i.e., those who received the procedurally just treatment had more negative opinion of the police), the treatment effect appeared to be consistent, was not heterogeneous, and thus it could be actually attributed to the research design. Continue reading “ScotCET and causal mediation analysis with a single mediator 1. – Procedural justice and normative alignment with the police”
This final post finishes the discussion of the previous three (you can find them here, here, and here) by looking at treatment effect heterogeneity and design heterogeneity.
Continue reading “Scotcet and implementation failure 4. – Treatment effect and design heterogeneity with a demonstration in R”
This post continues two earlier ones (this one and this one) and assesses the effect consistency across the matched pairs in the ScotCET dataset. Part of the redacted dataset can be downloaded at the bottom of this page which will allow you to carry out the same analysis presented below. Continue reading “ScotCET and implementation failure 3. – Treatment effect consistency in a blocking design with a demonstration in STATA”
How can one evaluate the veracity of the claim made by MacQueen and Bradford (2017), that the treatment effect emerging from the ScotCET RCT is, in fact, attributable to the experimental design, despite the apparent failure of implementation? For this assessment, one needs to discuss some design features of ScotCET. Continue reading “ScotCET and implementation failure 2. – The design features of ScotCET”
As outlined in my earlier post, one of the key propositions in the procedural justice literature is that – at least in most Western countries – procedural justice has an outsized importance in shaping views regarding the police. Yet, there have been only a few studies that established a causal relationship between police practices and the perception of procedural justice (e.g., Lowrey et al., 2016; Sahin et al., 2016), and none, that has explored its mediating role (although see Mazerolle et al. (2013) who used a path analysis approach – more on this in another post). Thus, in my first paper, I decided to “pry open” the black box of causality, and investigate procedural justice’s causally mediating role. Continue reading “ScotCET and implementation failure 1. – An apparent implementation failure”