Facilitation of Research Networks & Best Practices
The Importance of Understanding Scope in Social Science Interventions
Social scientists have always studied human behavior and society, but in the last 50 years, they have increasingly turned to helping solve significant social problems by interventions and policies. When evaluating a possible intervention, social scientists look at the outcomes it leads to, its costs, and how easily and affordably it can be enlarged to include many people—its scale. IPR economist Jonathan Guryan, with the University of Chicago’s Monica Bhatt, Jens Ludwig, and Anuj Shah argue in a working paper that another aspect of an intervention’s effectiveness to assess is its “scope.” By scope, they mean how much an intervention can change a large share of the decisions that affect a person’s outcomes. A particular intervention might have a large impact on one decision an individual makes, but many decisions go into changing a given outcome. If one examined intervention shows a large effect, it might seem like an attractive policy change; however, the intervention might make little difference when considered as part of the whole range of decisions people make that produce improvements. The authors present a framework to examine policies that pays special attention to this range of decision making. Using it would enable social scientists to focus more on scope and to determine whether putting together multiple small-impact policies could result in large social changes. Guryan is the Lawyer Taylor Professor of Education and Social Policy.
Educational Research in the COVID Era
Larry Hedges and Elizabeth Tipton, IPR statisticians and co-directors of the STEPP Center, address the difficulties, due to the COVID-19 pandemic, facing educational researchers with experiments in the field. In a working paper, they systematically consider possible strategies for researchers at different stages of conducting funded randomized or longitudinal studies. These studies, meant to assess how well certain educational interventions work, cannot be conducted as planned in schools that closed overnight in March 2020 and are often still conducting classes online. Drawing on their own and others’ research, and their own experience, which includes running the Cluster-Randomized Trials Institute at IPR, Hedges and Tipton present a set of questions and options to assist the researchers—and their grant program officers—to think through next steps. They begin with the principles of quality in educational research found in the Institute of Education Sciences (IES) Standards of Excellence in Education Research (SEER). They start by asking researchers to consider whether a study should continue. If the answer is yes, then the authors offer four strategies to alter the design, methodology, or even focus of the study to address issues that will likely arise. Above all, they urge researchers tackling great uncertainties to base their decisions on scientific experience, practice, and judgment. Hedges is Board of Trustees Professor of Statistics and Education and Social Policy.