Computing Power of Tests for the Variability of Treatment Effects in Designs with Two Levels of Nesting (WP-07-04)
Spyros Konstantopoulos
Field experiments that involve nested structures may assign treatment conditions either to entire groups (such as classrooms or schools), or individuals within groups (such as students). Even though typically the interest in field experiments is in determining the significance of the overall treatment effect, it is equally important to examine the inconsistency of the treatment effect in different contexts, and understand how they vary. This study provides methods for computing power of tests for the variability of treatment effects in different clusters in three-level designs, where for example, students are nested within classrooms and classrooms are nested within schools. The power computations take into account clustering effects at the classroom and at the school level, and sample size effects (e.g., number of students, classrooms). The methods can also be applied to quasi-experimental studies that examine the significance of the variation of group differences in an outcome, or associations between predictors and outcomes across different clusters.