Randomized Experiments in Education: Why Are They So Rare? (WP-02-19)
Thomas D. Cook
This paper documents how rare randomized experiments are in work to establish the causal consequences of educational innovations of obvious policy relevance. It also shows that most of those that have been conducted to date were done by people not trained in Schools of Education. The paper also describes the reasons that educational evaluators and theorists of method have adduced for not using experiments to learn “what works” in schools. These reasons are then critically appraised. The conclusions are reached that (1) the objections to random assignment are not cogent enough to justify methods other than the experiment, although (2) they are cogent enough to suggest feasible additions to the basic experimental framework. In particular, experiments need to deal with the quality of implementation of both random assignment and the innovation particulars; and they also need to deal with analyses of the extent to which the theoretical processes specified in the theory of the innovation are responsible for any cause-effect relationship claimed. Criticisms of experimentation in education strongly suggest the inadvisability of conducting black box experiments.