Evaluations of Education Interventions & Programs
Improving Communication of Education Research
Over the last 20 years, education research has focused on conducting high-quality causal studies to help school decision makers implement the best interventions, curricula, and practices and communicating them through mechanisms such as the Institute of Educations Sciences’ What Works Clearinghouse. In a 2022 report, the National Academies of Sciences, Engineering, and Medicine (NASEM) recommended a new type of research, “Knowledge Mobilization,” to study the connections between education research and schools. IPR statistician Elizabeth Tipton participated in the committee that developed NASEM recommendations, and she and former IPR graduate research assistant Kaitlyn Fitzgerald, now at Azusa Pacific University, present a three-part framework for the new field in the Journal of Research on Educational Effectiveness. Using their own field of statistical research as an example, they ask, “How should statistical evidence be reported and conveyed to facilitate evidence-based decision making by education practitioners and policymakers?” The three facets of the framework are (1) normative, examining the norms researchers assume and embed in their findings; (2) descriptive, understanding how educators and policymakers reason about evidence; and (3) prescriptive, developing and evaluating communication to enable decision makers to better use evidence. Fitzgerald and Tipton emphasize the disconnect between the message researchers send and what decision makers receive, which is also influenced by their outlook and understanding, as well as the particular situation. The researchers argue for creating an integrated science that focuses on translating and disseminating scientific findings incorporating data visualization and human-computer interaction.
Improving Immigrant Parents' English with a Two-Generation Program
In 2012, the Community Action Project of Tulsa County (CAP Tulsa) created a family-focused English as a Second Language (ESL) program for parents and children in their Head Start program. Immigrant parents seeking to improve their English language skills were enrolled in intensive, free ESL courses with a child-centered curriculum, while their young children attended Head Start. In the first experimental evaluation of this two-generation program, IPR researchers Teresa Eckrich Sommer, Lauren Tighe, Terri Sabol, Lindsay Chase-Lansdale, and their colleagues assess the effects on parents’ English language, parent engagement, and psychological wellbeing, as well as the children’s language and cognitive skills after one year. The randomized control trial of 197 parent-child pairs from Spanish- (89%; mostly from Mexico) and Zomi-speaking (11%; from Myanmar) families, published in Applied Developmental Science, finds that parents in the program reported higher English reading skills and engagement of their child’s teacher compared to parents not in the program. Parents who began the program with lower English proficiency reported more benefits–more positive parenting skills and lower levels of psychological distress. Parents with greater English skills, however, reported fewer benefits–more parenting stress and psychological distress after one year. The researchers suggest that parents with advanced English skills may become more aware of being treated differently, which may lead to distress. No effects on children’s language and cognition were detected, but all children in CAP Tulsa’s Head Start program scored highly on cognitive assessments. The authors emphasize that early investment in immigrant families with young children may have benefits for both parents and children and a long-term study is needed. They recommend creating a national demonstration program to explore how two-generation ESL programs may function in other Head Start contexts.
Affordable 'High-Dosage' Tutoring in High Schools
Can in-school tutoring be effective and affordable on a large scale? In the American Economic Review, IPR economist Jonathan Guryan and his colleagues explore intensive, in-school tutoring as a cost-effective intervention for high-school students who are behind academically. The researchers conducted two randomized controlled trials in Chicago Public Schools in 2013–14 and 2014–15 of Saga Education’s tutoring program, involving 2,633 and 2,645 ninth- and tenth-grade students, respectively. The program, which involves each tutor working with two students at a time for a full class period every day in addition to their regular math class, is integrated into the school day and uses paraprofessionals instead of full-time teachers, making it less costly. In the first trial, participating in the tutoring program caused student learning to increase by the equivalent of more than an additional year of learning, and in the second trial, tutoring generated even larger test score gains. Tutoring also raised grades and graduation rates for all students, who were also less likely to fail math and other courses. The cost for these beneficial outcomes is comparable to many successful early childhood programs, suggesting that effective tutoring may be affordable and scalable in traditional high schools. The research was supported by the National Institutes of Health, MacArthur Foundation, and Arnold Ventures. Guryan is Lawyer Taylor Professor of Education and Social Policy.
An Intervention to Improve Attendance and Academic Outcomes
Does mentoring improve a student’s attendance and academic outcomes? In the Journal of Policy Analysis and Management, IPR economist Jonathan Guryan and his co-authors ran a randomized controlled trial (RCT) on a mentoring program in Chicago Public Schools called Check & Connect (C&C) to examine the effects on attendance and academic outcomes. The researchers randomized the sample by schools, grades, and students. There was also a control group to determine whether the program affected students attending the same school as the intervention but not in the program. C&C mentors met individually with students or in small groups about five times a month, more often for students the mentors thought needed additional support, and had informal check-ins with students during recess and other school environments. Guryan and his colleagues determined that student absences in grades 5–7 decreased by 4.2 days or 22.9%, but the intervention did not improve student attendance in grades 1–4. In addition, the researchers did not find statistically significant effects on test scores or GPA or any detectable spillovers to other students in the schools where the program took place. The authors suggest that C&C may significantly impact older youth because older children have more control over school attendance decisions. While the researchers estimate that C&C costs about $400 per day of improved attendance, they argue higher-cost interventions like this one may be necessary to make more substantial improvements in attendance. Guryan is the Lawyer Taylor Professor of Education and Social Policy.
Does Education Protect Against Job Loss During an Economic Downturn?
Research suggests workers with more education suffer fewer job losses during economic downturns. In an IPR working paper, IPR labor and education economist Kirabo Jackson and his colleagues investigate whether more education had a causal effect of protecting against unemployment during the COVID-19 pandemic recession in Barbados. They linked educational records from 1987 to 2002 to nationally representative surveys of employment status conducted before, during, and after the recession. The educational records showed the students’ scores on the Barbados Secondary School Entrance Examination, which is taken after primary school and determines which secondary school students attend, and their preferred secondary school. The researchers find that when the economy was strong in 2016 and early 2020, men and women who attended their preferred schools were more likely to be employed than their peers. When the economy took a hit in May 2020, women who scored just above the cut off for their preferred school were 66 percentage points more likely to be employed than women who scored below it. Men who scored above the cut off for their preferred school were somewhat less likely to be employed. Women who attended their preferred schools attained more education and were less likely to lose their job during the recession. In contrast, men who attended their preferred school did not attain more education and were no more or less likely to lose a job. The results indicate that education plays a causal role in keeping workers employed during a poor economy because it enhances their skills. Jackson is the Abraham Harris Professor of Education and Social Policy.
Did More Schooling in Romania Lead to Better Health and Longer Life?
Although educated people have better health and live longer, it is not clear that additional education per se leads to improved health and longevity. Evidence on the causal effects of education on health in low- and middle-income countries is especially sparse. In a study published in The Journal of Human Resources, IPR economist Ofer Malamud and his colleagues investigate the impacts of Romania’s educational expansions on health, hospitalizations, and mortality. Between 1956 and 1962, Romania expanded compulsory education from four to seven years. This large and rapid change enables the researchers to compare similar groups of people who had less education to those who had more. Using enrollment, census, and vital statistics records, they examine the health and mortality of over 2.6 million Romanians born between 1945 and 1953 for over 60 years, up through 2016. They find that the educational expansions led to a significant increase in schooling in the population (approximately 2 months on average, since not everyone was affected). However, the researchers do not find any causal effect of education on health and mortality in their analyses. They suggest that free access to healthcare in Romania and the Communist labor market, which enforced similar wages for all workers, may partially explain their results. Their study of a relatively poor country with lower educational attainment is especially valuable since most of the prior work on the causal effects of education on health and mortality has been done in the U.S. and Western Europe.
Immigrants and the Educational Performance of U.S. Born Students
Over the past 50 years, immigration rates in the United States have risen dramatically, and in 2015, almost one out of four public school students came from an immigrant household. In an IPR working paper, Figlio, finance professor and IPR associate Paola Sapienza, and their colleagues study the effect of exposure to immigrants on the academic outcomes of U.S.-born students. The researchers use unique administrative data linking population-level school records from the Florida Department of Education between 2002–2012 with birth records between 1994–2002. The school data contain the results of the Florida Comprehensive Assessment Test (FCAT) in reading and math administered annually to all students in 3rd to 10th grade, disciplinary incidents, country of origin, and the language spoken at home. This dataset allowed the researchers to find siblings who had different exposure to immigrant classmates, follow them over many years, and compare their test scores to one another. The researchers find that immigrant students had a positive impact on U.S.-born students’ academic outcomes, especially for students from disadvantaged backgrounds, including Black students and free-or-reduced-priced lunch eligible students. Immigrant students also do not negatively affect U.S.-born students even when the immigrants’ academic achievement is lower. Despite the fact that White native and affluent students are more likely to flee schools with larger concentrations of immigrants, more immigrant students are associated with higher achievement of U.S.-born students, which suggests the value of diversity in schools. Sapienza is the Donald C. Clark/HSBC Chair in Consumer Finance at the Kellogg School of Management.
Improving Work-Related Social Interactions in Schools
Work-related, often chance, social interactions may lead to improvements in schools, but little is known about the factors that predict these interactions. In Teaching and Teacher Education, education professor and IPR associate James Spillane and his colleague explore the relationship between schools’ physical infrastructures and the work-related social interactions of school leaders. They look at 14 elementary schools in a Midwestern suburban school district of roughly 5,900 students. During 2011–12 and 2014–15, the researchers conducted semi-structured interviews with school staff from five of the district schools to explore staff interactions around math instruction. Teachers and administrations in all the schools were surveyed in the spring of 2010–13 about their work responsibilities, work-related social interactions, and room numbers. And the researchers measured physical proximity between workspaces using school floor plans. They find, first, that staff with leadership positions are located centrally in school buildings compared to teachers without formal leadership positions. Second, leaders understood that they were physically isolated in their schools and would make efforts to be more visible to colleagues by leaving their office doors open, visiting classrooms, and trying to engage in face-to-face interactions. Third, teachers also had to travel further to leaders than to non-leaders for work-related advice. The researchers urge school planners to consider physical placement of leaders, their usual routes through the building, as well as schedules and routines to ease the burden on formal leaders and increase interactions with their colleagues. Spillane is the Spencer T. and Ann W. Olin Professor in Learning and Organizational Change.