Recent Education Policy Research

School Finance, Accountability, and Vouchers

School Finance Reforms and Adult Outcomes

In an effort to ensure equal educational opportunity for all children, most states adopted school finance reforms between 1970 and 1995 that caused some of the most dramatic changes in the structure of education spending in U.S. history, yet little research exists on whether and how these changes have affected the adult well-being of students from poor neighborhoods. With support from the National Science Foundation, IPR economist Kirabo Jackson is examining the long-term effects of school finance reforms on state distributions of school spending, academic achievement, and adult outcomes. Preliminary findings using the Panel Study of Income Dynamics reveal that school finance reforms lead to increased educational attainment, higher incomes, and lower poverty for children from low-income households in areas that saw increases in school spending. Jackson, however, also finds evidence that while both legislative and court-mandated reforms reduced inequality in the short-term, wealthy districts were able to reverse the effects of legislative reforms in the long run.

Competitive Effects of School Vouchers

In a forthcoming American Economic Journal: Applied Economics article, education economist and IPR Director David Figlio and Cassandra Hart of the University of California, Davis, a former IPR graduate research assistant, study the effects of private school competition on public school students’ test scores after the Florida Tax Credit Scholarship Program is introduced. Before the program, some communities had a richer and more diverse set of private school options than others. The two examine whether test scores improved more for students attending public schools with many private schools nearby than for those attending schools with fewer local options. They find that both easier access to private schools and the variety of religious or secular affiliations of private schools are positively linked with public school students’ test scores after the program’s launch. Gains were more pronounced in schools most at risk to lose students, such as elementary and middle schools, where the price to attend private school with a voucher is much lower. But the results also indicate that the program’s introduction led to overall improvements in public school performance, with the gains occurring immediately—before students left the public schools to use a voucher. This implies that competitive threats are responsible for at least some of the voucher program’s estimated effects. Figlio is Orrington Lunt Professor and IPR director.

Cash Incentives and Long-Term Outcomes

Some education reforms use cash incentives to promote better student outcomes. The Advanced Placement Incentive Program (APIP) is one example. With private donors financing 70 percent of costs, the APIP trains AP teachers, and motivates both students and teachers by paying substantial cash bonuses for passing scores on AP exams. To examine if it works, Jackson looked at the APIP in Texas, tracking more than 290,000 high school students from 1993–2008. He compared changes in student outcomes before and after APIP adoption in the 58 participating schools with changes across the same cohorts in comparable schools that did not adopt it. APIP adoption increased taking an AP course by 21 percent and passing an AP exam by 45 percent. More importantly, Jackson finds benefits beyond the program. For those participating in APIP four years after it was adopted, the probability of students persisting in college as sophomores rose by about 20 percent and earnings increased by 3.7 percent. The pay increases erased the Hispanic-white earnings gap and reduced the black-white earnings gap by one-third. The results imply a per-pupil lifetime earnings benefit of $16,650 for a cost of $450. Jackson’s findings indicate it is possible to raise achievement for students “consigned” to low-achieving, urban schools—and that high-quality college-preparatory programs might be a viable alternative to transferring such students to higher-achieving schools. Results from the study were published in Economic Inquiry.

Financing College with FAFSA over Scholarships

IPR education and social policy professor James Rosenbaum and Kelly Iwanaga Becker, an IPR graduate research assistant, are examining the ways different high school counselors handle the college and financial aid application process. Building on work indicating that counselors encourage low-income students to apply for private scholarships, the researchers expanded the project to include data from all Chicago public high schools that have a policy of encouraging students to complete three or more scholarship applications. They find that slightly more than 54 percent of seniors who applied to scholarships reported receiving one. This is likely because many students were applying for private scholarships, which are typically very competitive with smaller financial awards. For instance, the Duck-Tape brand “Stuck at Prom” scholarship awards $20,000 to the couple with the best-designed duct-tape prom outfit. They show evidence that these time-consuming applications might derail some students from completing applications for more reliable sources of aid, such as the student federal-aid form, or FAFSA. Their results suggest a need for improving high school advising on the college financial-aid process. 

School Accountability and Instructional Changes

With Cecilia Rouse of Princeton University, Jane Hannaway of the American Institutes for Research, and Dan Goldhaber of the University of Washington, Figlio continued his study of the impact of school accountability in Florida with a study published in the American Economic Journal: Economic Policy. They surveyed Florida principals about their schools’ instructional policies and practices, before and after changes were made to Florida’s accountability grading scheme. They find that schools with increased accountability pressures appeared to focus on low-performing students, as previous research has shown, but they also find substantial changes in other areas. Schools increase time devoted to instruction, reorganize the structure of the day and the learning environments, increase teacher resources, and decrease principal control. They believe these responses can explain a portion of the test score gains associated with the Florida school accountability system.

School Accountability and Principal Mobility

The move toward increased school accountability initiated by No Child Left Behind (NCLB) could substantially saddle school leaders with more risk and less pay, affecting low-performing schools the most. Since effective school leaders likely have significant scope in choosing where to work, such uncompensated risks might undercut the effectiveness of accountability reforms by limiting the ability of low-performing schools to attract and retain effective leaders. IPR associate and Kellogg economist Danielle Li empirically evaluates how implementing NCLB in North Carolina affected principal mobility across the state's schools and how it reshaped the low- and high-performing schools where high-performing principals go. She demonstrates that NCLB decreases average principal quality at schools serving more disadvantaged students by inducing better qualified principals to move to schools less likely to face NCLB sanctions. These results are consistent with a model of principal-school matching where school districts with historically low-performing students fall farther behind, as they cannot compensate principals adequately for assuming a school's sanction risk.

Education Interventions and Program Evaluations

Using Mentors to Prevent Dropouts

While urban high school dropouts have received a great deal of policy attention, the problem almost always starts much earlier with truancy from school. However, very little is known about the risk and protective factors involved in truancy—and even less about effective remedies. To shed light on this issue, IPR economist Jonathan Guryan and his team of researchers are continuing to implement a new program called Check & Connect that matches students with adult mentors in an effort to increase school attendance and student engagement at 24 public elementary and middle schools in Chicago. Check & Connect is motivated by findings that show a strong relationship with a helpful adult is a highly protective factor against children failing school—something that many of those growing up in distressed family and community environments often lack. More than 3,000 students in 24 Chicago public schools are in the treatment and control groups, with close to 500 students receiving the intervention. The program involves mentoring, monitoring, and enhancing communication between school and home. The program’s potential spillover effects on peers of students in the program are also being measured by looking at outcomes for the more than 6,000 control students. The project is supported by the Institute of Education Sciences, the Eunice Kennedy Shriver National Institute of Child Health and Human Development, and the William T. Grant Foundation.

Lessons from Community Colleges

Nearly 90 percent of high school graduates say they will seek a bachelor’s degree, according to a 2004 national survey, but only 28 percent of community college students who want to earn a bachelor’s degree will actually get one—and it often takes them six to eight years to do so. In an article published with SUNY’s Janet Rosenbaum in the Journal of Economic Perspectives, Rosenbaum looks at evidence suggesting better results for community colleges’ private, two-year counterparts that offer career preparation in occupational fields like healthcare, business, information technology, and others. For many community college students, earning a quick two-year credential (i.e., certificates and associate degrees) that will qualify them for high-demand jobs is preferable to the relatively unlikely pathway from a community college program directly to a four-year program. The results suggest that instead of asking whether every student should attend college, it is more important to ask what type of college they should attend, what credentials they should seek, and in what sequence they should pursue those credentials. The authors stress that both nontraditional colleges and credentials deserve much closer attention from researchers, policymakers, and students.

Interventions for Low-Achieving Students

By the time they reach high school, many low-achieving students in distressed communities have been written off. They can be as many as four to seven years behind their grade level, particularly in math. Guryan and his colleagues at the Urban Education Lab are working on a new intervention that addresses this problem of “mismatch” for those performing below their current grade level. During the last year, the research team carried out a rigorous randomized controlled trial of “math tutoring on steroids,” two-on-one math instruction for an hour each school day. The study involved 106 ninth- and tenth-grade boys at Harper High School in Chicago, which serves mostly low-income students. The tutoring helps the students catch up to their grade level and re-engage in their classes, thereby increasing their chances of graduating. In addition, students also took part in the B.A.M. (Becoming A Man) program, a sociocognitive-skills intervention developed by the Chicago nonprofit organization Youth Guidance. Promising results from the pilot study led to a large-scale study of the two interventions being implemented in 21 Chicago public high schools in fall 2013. The Urban Education Lab was launched in 2011 by Guryan and University of Chicago colleagues Jens LudwigSteve Raudenbush, and Timothy Knowles. It counts more than 40 affiliated researchers from universities around the country, including five IPR faculty.

Usability of Community College Websites

In work with Jonathan Margolin and former IPR RA Shazia Rafiullah Miller at the American Institutes for Research, Rosenbaum evaluated whether community college websites are useful for providing knowledge relevant to degree completion. Ten community students used one of three community college websites to answer 10 questions about occupational degree programs. A facilitator asked participants to think aloud while using the website to answer these questions; their responses were video-recorded and coded in terms of the correctness of their answers and the types of usability problems encountered. The data suggest that participants frequently encountered problems with seeking out and understanding information needed to better comprehend degree selection and completion. The content analysis of these problems yields several suggestions for improving the usability of community college websites in answering common questions about degree completion. Community College Review published the results.

Use of Technology in Early Education

Despite increased access to computers and newer mobile devices, actual classroom technology use remains infrequent, especially in early childhood education. Most prior research has focused on K–12 teachers rather than early childhood educators. A study co-authored by IPR associate Ellen Wartella, the Sheikh Hamad bin Khalifa Al-Thani Professor of Communication, in Computers & Education examines predictors of early childhood educators’ access to and use of traditional technologies and mobile devices. Drawing on surveys from 1,329 early childhood educators, the researchers explore how school environment and attitudes toward the affordances and barriers of technology integration predicted the use of various devices. Their results indicate that while extrinsic barriers influence access to a range of technologies, the educators’ positive beliefs in children’s learning from technology predicted technology use more accurately. The study provides specific practical considerations to help increase quality integration of technology in early childhood education. It also suggests that adjusting teacher attitudes to better appreciate the benefits of technology could prove more effective in increasing its use in pre-K classes.

Online vs. Offline Learning

More than 80 percent of American research institutions now offer online classes, and schools are relying on them more than ever in light of recent financial constraints. In a study published in the Journal of Labor Economics, Figlio, with Mark Rush and Lu Yin from the University of Florida, compared the learning outcomes for students taking an online class versus live instruction. They conducted an experiment—randomly assigning a group of students enrolled in a microeconomics course to either a live or online section of the same course. They find that relatively low-achieving students, male students, and Hispanic students—the groups most likely to enroll in online courses—did better academically when they took a class face-to-face with an instructor rather than online. Until more research on the topic is done, colleges that simply put traditional courses online could potentially jeopardize student learning, especially for these three groups of students.

Generalizing Education Evaluations

If an education intervention proves to be successful for a study’s participants, will it work in schools outside of the study, too? Supported by the National Science Foundation and Institute of Education Sciences, IPR statistician and education researcher Larry Hedges and Paki Reid-Brossard, an IPR research associate, are investigating new methods to improve the generalizability of education research findings so results from one study can be used to make statistical claims for another population or location. Building on propensity score methods and a database of national covariates, they are working on a statistical approach that uses study samples to estimate parameters of the distribution of treatment effects in an inference population. Hedges is also developing methods that can be used to better plan experiments that aim to produce results generalizable to policy-relevant populations. Recent findings related to this work by Hedges and Colm O’Muircheartaigh at the University of Chicago appeared in the Journal of the Royal Statistical Society: Series C (Applied Statistics). Hedges is Board of Trustees Professor of Social Policy and Statistics.

Organizing for Instruction in Education Systems

In a study co-authored with Meghan Hopkins of Pennsylvania State University, IPR associate James Spillane extends earlier work based on a longitudinal mixed-methods study of one American school district and its 14 elementary schools to examine how the school staff organized for instruction in three core elementary school subjects. The Journal of Curriculum Studies article explores how district education leaders and teachers interact with one another in terms of advice and information about teaching and learning in literacy, mathematics, and science. The researchers examine similarities and differences in school staff members’ advice and information networks, considering how these relate to the formal organizational infrastructure intended to support instruction. The analysis reveals that how schools organize for instruction differs depending on the school subject, offering evidence that these differences are tied to differences in the formal organization structure, particularly in how the system deploys resources and in the curriculum design decisions of system leaders. They advise policymakers and practitioners to carefully design infrastructures to support interaction among school staff that will foster learning and cater to teacher’s learning needs in particular subject areas. Spillane is Olin Professor in Learning and Organizational Change.

Impact of Small Schools in Chicago

IPR economist Diane Whitmore Schanzenbach is part of a team that evaluated the performance of the Chicago Public Schools small high schools initiative. Using a quasi-experimental design and data from the Consortium on Chicago School Research, Schanzenbach and her colleagues analyzed student enrollment patterns and test scores for students entering high school at one of 22 new small schools, comparing them with their eighth-grade classmates who did not attend small schools. They find students who attend small schools are much more likely to persist and eventually graduate, although there was no effect on student test scores. Their results add evidence to the growing consensus that small schools improve academic achievement but do not raise standardized test scores—and that educational interventions aimed at older students are more effective at improving their noncognitive skills than their cognitive skills. The research was conducted with Amy Claessens of the University of Chicago and Lisa Barrow of the Federal Reserve Bank of Chicago.

Expanding Access to Preschool Education

In a forthcoming article with Dartmouth economist Elizabeth Cascio, Schanzenbach examines the effects of the introduction of universal preschool programs in Georgia and Oklahoma in the 1990s, comparing the children and families in those states with children and families elsewhere in the country. They reveal stark differences in preschool enrollment patterns by family background, with children whose mothers have no more than a high school diploma being much more likely to enroll their children in preschool at age 4—experiencing an 18–20 percentage-point enrollment gain versus a 12–15 percentage-point gain in preschool enrollment rates for children whose mothers have more education. The authors also find some academic benefits, with modest, sustained increases in eighth-grade math test scores for the lower-income children. Conversely, among higher-income children, they see no positive impacts of the program on student achievement. The researchers suggest it might be more cost-effective to design a preschool program to target those most in need to reduce the extent of crowd-out. The Brookings Papers on Economic Activity will publish the study.

Teacher and Principal Characteristics

Tenure-Track Professors and Teaching

As the higher education landscape changes and colleges and universities rely increasingly on a combination of nontenure- and tenure-track faculty, IPR education economists David Figlio and Morton Schapiro, also Northwestern University president, analyzed data on more than 15,000 Northwestern freshmen from 2001–2008 to compare the impacts of tenure-track versus nontenure-track faculty on student learning outcomes. With Kevin Soter, a Northwestern alumnus and consultant for The Greatest Good, they find students were relatively more likely to take a second course, and to earn a higher grade in that subsequent course, when the introductory course had been taught by a nontenure-track instructor. Their findings held consistently across subjects, and the benefits of taking introductory courses with nontenure-track faculty were strongest for the incoming freshmen with lower academic indicators. Interestingly, the beneficial effects of adjunct faculty were bigger for two groups of Northwestern students—those who were less academically prepared and those who tended to take more difficult classes. The working paper, which generated a wave of media coverage, also called into question whether the rise of hiring full-time designated teachers in U.S. higher education is “cause for alarm.” Rather, the three researchers suggest that such a trend might offer colleges and universities a way to be great institutions of research and of undergraduate learning at the same time. The Smith Richardson Foundation supported the project.

Impact of High School Teachers

Research has shown that elementary school teachers matter, but what about high school teachers? Some have just extrapolated the findings from studies of elementary school teachers and applied them to those in high school, but Jackson advises against this. In a forthcoming Journal of Labor Economics article, he argues that in high schools, even with random assignment of students to teachers, bias exists due to “track treatment” effects. This happens when different teachers teach in different tracks, and students in different tracks are exposed to different treatments. These “track treatment” effects might arise due to other teachers, the content of other courses, or explicit track-level treatments, such as honors courses or college-prep courses. To counter this, Jackson outlines a new method for identifying teacher quality effects in high schools, testing it with data on all North Carolina ninth graders from 2005–2010. He shows that high school Algebra I and English I teachers have much smaller effects on student test scores than elementary school teachers. The Spencer Foundation is supporting the project.

Student Abilities, Test Scores, and Teacher Quality

In an IPR working paper, Jackson develops a new model to measure long-term outcomes that combines student cognitive and noncognitive ability and teacher effects to evaluate students’ outcomes. Conditional on cognitive scores, an underlying noncognitive factor associated with student absences, suspensions, grades, and grade progression is strongly correlated with long-run educational attainment, arrests, and earnings in survey data. In administrative data, teachers have meaningful causal effects on both test scores and the noncognitive factor. The calculations show that teacher effects based on test scores alone fail to identify many excellent teachers—and might greatly understate the importance of teachers on adult outcomes. The Spencer Foundation provides funding for the project.

Importance of Teacher-School Matching

Published in the Review of Economics and Statistics, a recent study by Jackson investigates the importance of the match between teachers and schools for student achievement using data from North Carolina. From a sample of mobile teachers, he documents that teacher effectiveness—as measured by improvements in student test scores—increases after a move to a different school. He then estimates the importance of teacher-school match quality for the resulting improvement in student outcomes. Preliminary results reveal that between one-quarter and one-half of what is typically measured as a teacher effect is, in fact, due to the specific teacher-school pairing and does not carry across schools. Further, he establishes that match quality is as economically important as teacher quality in explaining student achievement.

High-School-to-College Transitions

Class Size and College Completion

In work with Susan Dynarski and Joshua Hyman of the University of Michigan, Schanzenbach looks at the effects of reducing elementary school class sizes on college enrollment and getting a degree. Using Tennessee Project STAR data, they find being randomly assigned to attend a smaller class in kindergarten through third grade increases a child’s probability of attending college. Assignment to a small class increases the probability of attending college by 2.7 percentage points, with effects more than twice as large among African Americans. Among those with the lowest projected probability for attending college, the effect is 11 percentage points. In addition, small classes in the early grades improve the likelihood of earning a college degree, and majoring in a more technical and high-earning field, such as a STEM field (science, technology, engineering, and mathematics), business, or economics. The article documents short- and long-term effects of early education interventions. The actual long-run impacts were larger than what short-run test score gains alone would have predicted. This implies that cost-benefit analyses based on short-run impacts might misestimate the true long-run effectiveness of interventions. The Association for Public Policy Analysis and Management awarded Schanzenbach and her co-authors the 2013 Vernon Memorial Prize for the article, given annually for the best research article published in their Journal of Policy Analysis and Management.

College for All

With a grant from the Spencer Foundation, Rosenbaum continues research on his “College for All” project. While the nation’s high schools have embraced the idea of trying to get all high school seniors into college, little attention has been paid to the processes that increase the number of students who actually complete their degree. Rosenbaum’s research team is conducting a longitudinal study of all seniors at 82 public high schools in Chicago from 2004–2008 seeking to extend understanding of the varied institutional procedures that shape the high-school-to-college transition process for students.

Gaps in Academic Achievement

Addressing the Summer Reading Gap

Once children enter school, a reading gap between students from high- and low-socioeconomic-status (SES) backgrounds appears and begins to grow. It is likely exacerbated by summer vacation, as low-SES students are less likely to receive continued reading instruction over the break. Guryan and James Kim of Harvard University are in the midst of a five-year, multidistrict randomized controlled trial to implement and evaluate Project READS, Reading Enhances Achievement During the Summer. The program, developed by Kim, is being administered to approximately 10,000 students in 70 North Carolina elementary schools over the course of the study. It is already showing promising results, moderately reducing “summer loss” and improving reading skills. Students are sent two books every two weeks over summer break. Matched to student interests and reading level, the books are also paired with family activities to support summer reading. Members of the control group receive the books and activities at the start of school. Pre-and post-tests, as well as reading tests, are used to measure impact. In addition to monitoring student achievement and overall progress, Guryan is also examining different variations of READS to improve effectiveness, measure cost- effectiveness, and seek how best to replicate and further expand the program. The U.S. Department of Education is providing support for the project.

Benefits of Attending Selective Schools

A growing body of evidence indicates that attending selective schools might improve student outcomes, but researchers have little understanding of why. Using data from Trinidad and Tobago, Jackson investigates the extent to which the positive school effects can be attributed to the fact that selective schools contain higher-achieving peers. He explains that attending a school with higher-achieving peers is associated with substantial improvements in academic outcomes, with larger effects seen for girls. However, on average, improvements to incoming peer achievement within a school are associated with small improvements. The effect of improvements in peer achievement within a school is largest at selective schools, providing further evidence that direct peer effects are responsible for at least some of the effect of attending schools with higher-achieving peers. Jackson concludes that direct peer effects do not explain the benefits of attending a more selective school among the bottom three-quarters of schools, but among the top-quarter, at least one-third can be attributed directly to how well their peers are performing. These results highlight the importance of understanding how schools seek to improve student outcomes. Jackson calls the results “sobering” since very little of the success of these highly selective schools can be translated to average schools. He notes, however, that the relative successes at average schools—since they are not attributed to peer achievement—could be scalable to low-performing schools. The article was published in the Journal of Public Economics, and project support came from the Spencer Foundation.

Wealth Effects on School Achievement

In an Economics of Education Review article, social psychologist and IPR associate Mesmin Destin describes various approaches to understanding how wealth seems to influence children’s educational experiences, which are generally categorized as either resource-based or person-based. Resource-based approaches prioritize the importance of investments made to enrich the contexts, expand opportunities, and improve outcomes for youth. Person-based approaches focus on how beliefs, values, and perceptions shape achievement. Destin suggests that a combination of resource-based and person-based approaches might reveal a more complete model of the complex relationship between wealth and educational achievement. He proposes an integrative framework that uses identity as a unifying construct to connect the observable influence of resources to the more subtle role of motivation and cognitive factors that drive achievement. This basic conceptual model allows room for overlap between resource-based and person-based perspectives. These have the potential to inform one another in important ways that can advance research regarding wealth and asset effects on child development.

Social Distribution of Achievement

In a project with funding from the Spencer Foundation, Hedges and his colleagues seek to document the social distribution of academic achievement in the United States. By examining various achievement gaps in different ways, they come to understand how the social distribution of achievement has changed over the last few decades. A major part of this study evaluates patterns of between- and within-school variability of student achievement. They also examine whether different sources of evidence lead to the same conclusions; that is, they seek to triangulate whenever possible. Finally, the researchers are studying the somewhat limited longitudinal evidence, attempting to coordinate it with repeated cross-sectional evidence. They expect that combining such data might help understand the emergence of differences in patterns of academic achievement between important population subgroups.