Using multisite instrumental variables to estimate treatment effects and treatment effect heterogeneity
MetadataShow full item record
Multisite randomized trials (MSTs) are an attractive research design to test the efficacy of an educational program at scale. Population models examining data from MSTs can provide information on the range of possible treatment effects that sites (such as schools) can expect from an educational program, even for those sites not included in the study. However, when some individuals at a site do not comply with their treatment assignment, conventional multilevel and meta-analytic estimation methods do not provide information on the effect of actually participating in the educational program. Instrumental variables (IV) is a method that can produce consistent estimates of the causal effect of participating in an educational program for those individuals that comply with their treatment assignment, an estimand called the complier-average treatment effect (CATE). IV methods for single-site trials are well understood and widely-used. Recently multisite IV models have been proposed to estimate the CATE and CATE heterogeneity across a population of sites, but the performance of these estimators has not been examined in a simulation study. Using Monte Carlo simulation, the current study examines the performance of three IV estimators and two conventional estimators in recovering the CATE and CATE heterogeneity under simulation conditions that resemble multisite trials of well-known educational programs.