Embedding a Randomized Experiment Within a Regression Discontinuity Design: A Meta-Analytic Approach

Hong, Mindy

doi:https://doi.org/10.21985/n2-d282-bp96

Work

Embedding a Randomized Experiment Within a Regression Discontinuity Design: A Meta-Analytic Approach

Public

Download PDF

Download All Files (.zip)

Randomization is considered the gold standard when it comes to evaluating the effectiveness of interventions, primarily due to its ability to avoid bias. However, in recent years, randomization has been heavily criticized in circumstances where subject randomization may not be ethical. In a randomized controlled trial, patients who are extremely ill and therefore in greater need of a treatment, have the exact same chance of receiving the treatment as patients who may not need to be as prioritized. Because of its ethical issues, complete randomization in trials is sometimes infeasible. As a result, researchers have considered alternative design strategies that do not include any randomization while still generating unbiased causal effects, known as quasi-experiments. One particular quasi-experiment is the regression discontinuity design, which involves assigning individuals to treatment conditions on the basis of a cutoff on an assignment variable. Regression discontinuity designs can act as alternatives to randomized experiments, and although when executed correctly can generate treatment effect estimates similar to those produced by randomization, they require a larger sample size. As a result, there has been an interest in combining regression discontinuity designs and randomized experiments in order to create a hybrid design that consists of both experimental and quasi-experimental features. The randomization portion of the combined design leads to an increase the power of the test of treatment effects, allowing it to be a better alternative than to using a regression discontinuity design on its own. This research considers different methods for researchers to analyze a combined regression and randomized experiment design and presents methods in which both design estimates are combined in order to obtain an overall effect estimate. By treating the regression discontinuity and the randomized experiment as independent studies, meta-analytic techniques can be used to combine the estimates into one that is more statistically efficient. For each approach, the algebraic forms for computing the design's treatment effect estimate and its variance are derived. In order to provide researchers with ways to properly design and implement these trials, power analyses for the proposed methods are shown. Additionally, in order to better understand how these designs perform in comparison to one another, the proposed methods are applied to a dataset on student test scores from a University of California Merced study. By evaluating the different approaches to analyzing the combined regression discontinuity and randomized experiment design, this dissertation aims to address the specific concerns of randomization by providing researchers with an alternative solution that is still considered a statistically powerful design.

Creator