I have not been blogging as diligently lately, and one of the reasons for this is that I have been a bit preoccupied with writing resubmissions. As part of this process, I had to revisit the issue of treatment effect consistency which I discussed last summer. This post discusses an alternative way of assessing and quantifying whether treatment effect (in)consistency is present, using methods adopted from meta-analysis.
I have been conducting online experiments with MTurk and other similar online platforms for almost four years now. I usually leave a little feedback box at the end of each experiment, where participants can share their thoughts, make some comments, and potential complaints about the study. I have found this a very useful tool, especially during the piloting of the experiments, where many attentive respondents have pointed out several typos and other mistakes over the years. Continue reading “A 32-year old White Alabamian man* and the luxury of quantitative criminology”
This final post finishes the discussion of the previous three (you can find them here, here, and here) by looking at treatment effect heterogeneity and design heterogeneity.
Continue reading “Scotcet and implementation failure 4. – Treatment effect and design heterogeneity with a demonstration in R”
This post continues two earlier ones (this one and this one) and assesses the effect consistency across the matched pairs in the ScotCET dataset. Part of the redacted dataset can be downloaded at the bottom of this page which will allow you to carry out the same analysis presented below. Continue reading “ScotCET and implementation failure 3. – Treatment effect consistency in a blocking design with a demonstration in STATA”
How can one evaluate the veracity of the claim made by MacQueen and Bradford (2017), that the treatment effect emerging from the ScotCET RCT is, in fact, attributable to the experimental design, despite the apparent failure of implementation? For this assessment, one needs to discuss some design features of ScotCET. Continue reading “ScotCET and implementation failure 2. – The design features of ScotCET”