Education Policy

Introduction

Failing public schools, tumbling U.S. student achievement, declining school funding, persistent achievement gaps, and recruiting and retaining effective teachers are just a few of the critical issues that school districts across the nation face every day. More rigorous research is needed to understand the issues facing schools and educators and to create effective solutions to address them. IPR’s Education Policy program regroups faculty fellows from a variety of disciplines and aligns with other programs, including Quantitative Methods for Policy Research.

Overview of Activities

Retooling No Child Left Behind

Therese McGuire moderates an IPR policy research briefing on Capitol Hill evaluating the impact of No Child Left Behind.

At a February 2010 IPR policy research briefing on Capitol Hill, three researchers discussed what lawmakers should consider when they retool the federal No Child Left Behind Act (NCLB), passed under President Bush in 2002.

Starting with whether NCLB has raised achievement, IPR social psychologist Thomas D. Cook provided one of the most scientifically rigorous sources of evidence to date to show that NCLB has indeed raised standardized test scores in public schools. Using data from the National Assessment of Educational Progress, or NAEP, from 1990 to 2009, Cook presented multiple analyses. Some compared public schools in the nation at large with both Catholic and non-Catholic private schools, and others compared states that chose to implement NCLB in ways that led to larger or smaller amounts of school reform being required.
All these national and state analyses show that after 2002, NCLB affected fourth- and eighth-grade performance in math but not in reading. Cook, who is Joan and Sarepta Harrison Chair in Ethics and Justice, co-authored the study with former IPR colleagues Manyee Wong, now of the American Institutes for Research, and Peter Steiner, now at the University of Wisconsin–Madison.

Despite NCLB’s goal to “leave no child behind,” IPR economist Diane Whitmore Schanzenbach has found that many were. She and colleague Derek Neal of the University of Chicago followed two groups of
children in Chicago Public Schools (CPS) before and after the introduction of NCLB. Different cohorts took either a low- or high-stakes test before and after NCLB’s implementation. Grouping students by their baseline math and reading achievement decile, they find scores remained the same, or worsened, for the highest- and lowest-scoring students, but improved for those children in the middle who scored close to the passing margin. These “bubble kids” provide additional evidence that teachers and schools engage in “educational triage,” giving extra attention to those low-performing students with the best chances of passing while the rest are left behind. Schanzenbach noted that, looking across all grades in Chicago, this practice affects 25,000 to 50,000 low-achieving students. The study was published in the Review of Economics and Statistics.

In the ongoing quest for ever-higher test scores, some schools try to “game the system” and reach for a special score boost on test day. IPR education economist David Figlio has documented several such scenarios, from feeding students high-calorie snacks, with positive, short-term cognitive effects, to suspending or expelling low-achieving students on test days. His studies confirm the need for a high-proficiency threshold as confirmed by Cook and his colleagues, but schools also need to measure gains for those students who do not achieve it, as Schanzenbach’s study suggests. Thus, he recommends a hybrid system for accountability. Figlio is Orrington Lunt Professor of Education and Social Policy.

School Finance Reform and State Taxes
One of the most important external shocks to state government finances over the past 40 years has been court-ordered school finance reform. In ongoing research with Nathan Anderson of the University of Illinois at Chicago, IPR public finance economist Therese McGuire is studying whether states change their tax revenue structures to pay for these reforms. The researchers look at 19 states that had court-ordered school finance reforms handed down at least once between 1980 and 2007, comparing them with states that did not experience such reforms. Preliminary findings suggest that for those states that experienced reform and had a state income tax, income taxes became more progressive. This trend indicates that states were not shifting the costs of such measures back to lower-income taxpayers. McGuire is ConAgra Foods Research Professor in the Kellogg School of Management.

Demographic Changes and School Finance
A new study by Figlio and Deborah Fletcher of Miami University, forthcoming in the Journal of Public Economics, links demographics and education funding in 20 suburban school districts around Northeastern and Midwestern cities between 1970 and the 1990s. Their data reveal that—all else equal—once the Baby Boomers’ children were out of school, suburban school districts that developed earlier and with consequently older populations tended to cut back on school spending faster than did later-developing suburbs with younger families. This effect was mitigated, however, in districts where voters held less local control over revenues and spending. Finally, the biggest differences appeared in districts with the widest racial and ethnic gaps between older and younger residents. The authors posit what such demographic change portends for local education policies.

Vouchers and Public School Test Scores
Competition from private-school vouchers might lead to small, but significant, academic gains in surrounding public schools. That is one conclusion from a new study by Figlio and IPR graduate research assistant Cassandra Hart. In examining low-income students in Florida’s tax credit voucher system, they find students in public schools with a number of private schools in the area had better test scores than students in other public schools. Pressure to improve increased as the number and variety of private schools increased. The researchers also find that the Florida students using vouchers differed little in academic performance compared with students in public schools. Figlio cautions against seeing vouchers as a cure-all, however, since the gains are small, though significant. This finding stems from Figlio’s ongoing evaluation of Florida’s Tax Credit Scholarship Program, the largest school voucher program of its kind in the United States.

Student Pay and Test Scores

Kirabo Jackson presents evidence that match quality between teachers and schools has a significant impact on student achievement.

IPR labor and education economist Kirabo Jackson and Dartmouth economist Bruce Sacerdote are examining the short- and long-term impact of a New York City program that pays high school seniors for achieving a score of 3, 4, or 5 on Advanced Placement (AP) exams. Jackson has already evaluated a similar program in Texas, which pays low-income and minority students for scoring well on their AP tests. The Texas study shows that schools offering students $100 to $500 for scores of 3 or higher have more students taking AP courses, more scoring well, and 8 percent more going to college. In addition, Texas students’ SAT and ACT scores rose by 30 percent. In a follow-up study tracking students through college, he finds that those students who received the incentives in high school attend college in greater numbers, are more likely to remain in college beyond their freshman year, and have better GPAs. He also uncovers evidence of increased college graduation rates for black and Hispanic students. Findings from the Texas study were published in the Journal of Human Resources.

School Context and Teacher Effectiveness
Using data from North Carolina, Jackson is investigating the importance of the match between teachers and schools for student achievement. From a sample of mobile teachers, he documents that teacher effectiveness—as measured by improvements in student test scores—increases after a move to a different school. He then estimates the importance of teacher-school match quality for the resulting improvement in student outcomes. Preliminary results show that between one-quarter and one-half of what is typically measured as a teacher effect is, in fact, due to the specific teacher-school pairing and is not portable across schools. Moreover, he finds that match quality is as economically important as teacher quality in explaining student achievement.

In related work, he is also analyzing how the opening of a charter school affects teacher turnover, hiring, effectiveness, and salaries at nearby traditional public schools. By analyzing a variety of teacher outcomes, he hopes to paint a relatively comprehensive picture of how charter school entry affects both the demand for and supply of teachers at pre-existing traditional public schools.

School Accountability and Teacher Mobility
In a 2010 IPR working paper, Figlio and his colleagues provide the first causal evidence that school accountability measures affect the teacher labor market. They exploit a 2002 change in Florida’s statewide accountability system, which exogenously shocked some schools to receiving higher accountability grades and others to receiving lower accountability grades. Using microdata from the universe of Florida public school teachers, they then measure whether teachers in these “shocked” schools were more or less likely to keep their current jobs or move. It turns out that after a school faces a downward shock—especially to the lowest grade—its teachers are much more likely to leave, even for another school in the same district. Meanwhile, schools that see an upward shock are more likely to retain more teachers. These patterns affect the distribution of teacher quality, in terms of value-added measures for students, although the average differences are not large across all schools. The study was co-authored by Li Feng of Texas State University and Tim Sass of Florida State University.

Accountability Pressures and Children’s Health
Under NCLB, schools facing increased pressure to produce academic outcomes might reallocate their efforts in ways that inadvertently affect the health of their students, for example, by cutting back on recess and physical education in favor of more classroom time. Schanzenbach, Patricia Anderson of Dartmouth College, and Kristin Butcher of Wellesley use data from schools in Arkansas to test the impact of NCLB rules on student obesity rates. Schools on the margin of meeting NCLB requirements see the obesity rates among their students increase, and the evidence suggests that such schools have changed their health-related policies in a manner that might lead to the observed increase.

Student Teachers and Placement in Public Schools
With funding from the Joyce Foundation, education researcher and IPR faculty affiliate Michelle Reininger is completing a longitudinal study of 3,000 Chicago Public School (CPS) student teachers who have entered and exited the district’s student-teaching program. As the nation’s third largest urban school district, CPS struggles each year to fill its classrooms with high-quality teachers, especially in hard-to-staff subjects such as math, science, and bilingual education. Preliminary data from two online surveys indicate that more than half of student teachers plan to continue working as a teacher or in education generally following the program. Yet after their student-teaching placement, just 25 percent say they have a greater interest in teaching in CPS and 30 percent show less interest in staying within the district. More student teachers expressed having “no plans” of continuing in CPS after they finished their placement than before, suggesting that the district needs to improve its placement program if it hopes to retain participants.

	James Spillane's research looks at the organizational routines of school leaders, most recently through a large-scale study of school prinicipals.

Principal Policy and Practice
The Principal Policy and Practice (P3) Study relies on the research strengths of education professor and IPR associate James Spillane on school leadership and Reininger on teacher training and preparation to examine Chicago Public School principals. Studying the issue from a supply-and-demand perspective, they are looking at the principals’ routes to preparation, recruitment, and retention, in addition to tracking their career paths. The two investigators hope the study will shore up a significant lack of data and contribute to better school policies, given the principal’s importance in shepherding school improvements. Spillane is Spencer T. and Ann W. Olin Professor of Learning and Organizational Change.

Leadership Development in Schools
Spillane is investigating an approach to leadership development that centers on school practice rather than focusing exclusively on principals’ knowledge and skills. Specifically, he has launched a four-year research study with Brian Junker of Carnegie Mellon University and Richard Correnti of the University of Pittsburgh to determine the impact of a structured “walk-through” routine for school leaders. This routine, called the Learning Walk®, involves principals and other school leaders conducting brief visits to classrooms on a regular basis for the purpose of observing classroom instruction.

Online vs. In-Class Learning
The rise of online universities and the explosion of online class offerings has been driven to some extent by advances in technology and budget cuts at universities, but does online learning confer the same educational benefits as live lectures for college students? In a study forthcoming in the Journal of Labor Economics, Figlio and his University of Florida colleagues Mark Rush and Yu Lin are the first to compare how live versus online delivery of large lectures affects student learning. Their data come from an experiment in which 327 students at a major university were randomly assigned either to an online section or to a live section of an introductory economics course taught in an identical manner by the same instructor. The only difference was that one group only viewed the lectures live and the other only on the Internet. The researchers find a slight effect for better achievement with live lectures, but when stratifying the results, they show that low-achieving and male students in the online sample tend to have worse grades. While online and distance learning courses seem to be more cost-effective, the authors caution universities to proceed carefully. They conclude that in the rush to supplant live instruction with online courses, universities could leave some behind.

College Attendance and “Coaching” Programs

James Rosenbaum(l.) talks with Stanford education researcher Sean Reardon, who presented his study of the widening rich-poor achievement gap at an IPR colloquium.

More than 80 percent of Chicago public high school students say they want to go to college, yet only 64 percent actually attend. IPR education researcher James Rosenbaum is looking at possible causes for the gap, in particular for low-income students, who are less likely to enroll in college. Starting in 2005, he, IPR postdoctoral fellow Jennifer Stephan, and graduate student Michelle Naffziger began gathering ethnographic and administrative data from a new college-counseling program in Chicago Public Schools (CPS) that targets disadvantaged students, many of whom are the first in their families to attend college. The program helps students overcome cultural barriers by pairing them with “college coaches,” who advise them on their options for college, demonstrate how to work with admissions counselors, and assist with scholarship applications. Using comprehensive surveys of nearly all CPS seniors from their senior year through the fall after high school and comparing outcomes before and after the onset of the program, Rosenbaum and Stephan find significant improvements in outcomes for students in the coaching high schools over comparable students in other high schools in the district. In detailed observations, Rosenbaum and Naffziger describe how coaches engage students and improve the types of colleges students attend.

Unlike many programs, this reform benefits disadvantaged students more than relatively advantaged students. The researchers’ presentation to CPS administrators, including then-CEO Ron Huberman, helped convince district administrators to save the program from cuts.

Can College Be for All?
Rosenbaum received a new Spencer Foundation grant to continue research on his “College for All” project. While the nation’s high schools have embraced the idea of trying to get all high school seniors into college, little attention has been paid to the processes that increase the number of students who actually go and complete their degree. Rosenbaum’s research team is conducting a longitudinal study of all seniors at 82 public high schools in Chicago, examining the role of guidance counselors and other college-oriented programs and indicators of college attendance. The study seeks to extend understanding of the varied institutional procedures that shape the college search and application process for students. It will examine procedures by multiple actors, students’ relevant college activities, and student outcomes, including college persistence.

“Gainful Employment” and For-Profit Colleges
In 2010, the U.S. Department of Education was considering a rule change called “gainful employment,” which would have discontinued federal student aid at for-profit colleges whose graduates rarely earn enough to pay back their student loans—and thus are prone to higher default rates. As an economic expert for the Association of Private Sector Colleges and Universities, IPR economist Jonathan Guryan wrote a report on how the proposed regulation would affect for-profit colleges and universities and their students. Analyzing data for 600,000 students in more than 10,000 programs at the association’s member institutions, he found that the proposed regulation would have had potentially devastating consequences: As many as 2,000 for-profit programs, representing one-third of students in the study, would have become ineligible for federal financial aid. Furthermore, he found that a disproportionate number of the 200,000 students potentially affected by the rule were low-income and/or minority. After the report’s release, Guryan briefed many policymakers on the research, such as U.S. Secretary of Education Arne Duncan, several of his senior staff, and senior congressional staff, including those whose members sit on the congressional black, Hispanic, and Asian caucuses. Subsequently, the U.S. Department of Education released a revised version of the rules that Guryan’s research suggests would have a reduced but still significant negative impact on college access. The final rule, published in June 2011, addresses certain criticisms outlined in Guryan’s public comment as submitted to the Department.

Mothers’ Education and Child Development

Lindsay Chase-Lansdale presents her pilot study of a dual-generation education intervention for mothers and their preschool-aged children

IPR developmental psychologist Lindsay Chase-Lansdale and IPR research scientist Teresa Eckrich Sommer are expanding a unique dual-generation intervention that links postsecondary education and career training of low-income mothers to their children’s development through early childhood education centers (ECEs). Their recent exploratory work indicates that such a program could harness mothers’ hopes for their children as motivation for educational progress and provide the mothers with supportive staff and peers. It also intervenes at a critical developmental point in the preschoolers’ lives, likely providing more benefits than if the mothers were to wait until their children were in school to further their own education. The researchers will study and expand a pilot program, CareerAdvance, through the Community Action Project of Tulsa, Okla., with an IPR seed grant and additional support from the Administration for Children and Families.

Investment in Early Childcare Centers
Interest in childhood education is on the rise among global business and economic leaders, including Fortune 500 CEOs, Nobel laureates, Federal Reserve bankers, and even Grammy Award-winning artists. During the 2008 presidential campaign, Barack Obama pledged $10 billion
for early childhood education. IPR economist Sergio Urzúa and graduate student Grace Noboa Hidalgo have examined the impact of public investments in early childhood education in Chile on a range of cognitive and noncognitive child development outcomes. Deriving average causal effects, their results reveal gains in nearly all areas of child development, particularly those concerning motor and cognitive skills. The positive effects are particularly high for children 7 to 12 months old who attend a public childcare center for seven months. Though some negative effects are found with respect to adult interactions, the authors highlight the policy challenges and rewards of expanding ECE coverage, in particular for low-income and young children, all the while maintaining high-quality, yet expensive, services.

Kindergarten Teachers and Future Earnings
Does it matter who your teacher is in kindergarten? In an innovative new study, Schanzenbach and her colleagues at Harvard and the University of California, Berkeley, test whether kindergarten classroom quality and student test scores make a difference in adult outcomes. They use data from the Tennessee Project STAR experiment from the 1980s, which randomly assigned nearly 12,000 children to kindergarten classrooms with varying class sizes and followed the student’s progress through third grade. The researchers use tax data to link the students’ kindergarten class experience and test scores to their adult wages, education, and other later outcomes. Their analysis shows that being randomly assigned to a high-quality kindergarten class improves wages, as well as a variety of other outcomes, in adulthood.

Abilities, Education, Race, and the Labor Market
For many years, economists focused on the role of cognitive ability as a determinant of schooling, labor market, and behavioral outcomes. Yet Urzúa and others are uncovering evidence that noncognitive measures of ability might be more important—and relevant—than IQ and other indicators of cognitive intelligence. In one study using data from the 1979 National Longitudinal Survey of Youth (NLSY79), Urzúa created a model of the relationship between abilities, schooling choices, and racial gaps in labor market outcomes. In his analysis, he distinguishes observed cognitive and noncognitive measures from unobserved cognitive and noncognitive abilities. He analyzes schooling decisions based on future earnings, family background, and unobserved abilities. The results indicate that, even after controlling for abilities, significant racial labor market gaps exist. They also suggest that the standard practice of equating observed test scores might overcompensate for differentials in ability.

Gaps in Academic Achievement

	Larry Hedges talks to journalists at an education writers conference about how to better report the results of randomized studies.

IPR education researcher and statistician Larry Hedges and his colleagues are documenting the social distribution of academic achievement in the United States by examining various achievement gaps, including those of gender, race, ethnicity, and social class, in different ways. A major part of this study evaluates patterns of between- and within-school variability of student achievement. The researchers also examine whether different sources of evidence lead to the same conclusions and attempt to coordinate the limited longitudinal evidence with repeated cross-sectional evidence. They expect that combining such data might help to explain differences in patterns of academic achievement between minorities. They hope to find out more about how large achievement gaps are when students enter school and how they evolve over time. Hedges is Board of  Trustees Professor of Statistics and Social Policy.

Assessing Spatial Learning
Workers in a high-tech, global economy need adequate scientific, technical, engineering, and mathematical skills. Northwestern’s Spatial Intelligence and Learning Center (SILC) aims to achieve a better understanding of spatial relationships, which serve as a basis for many of these skills. Hedges is leading the SILC project to develop and distribute a spatial assessment battery. One key element in the battery is CogSketch software, a sketch-understanding software that can be downloaded for free from SILC’s website (www.spatiallearning.org). Researchers are gathering users’ sketches to analyze how people reason and learn and how sketching might improve student learning.

Achievement Gap and Summer Reading
Once children enter school, a reading gap between students of high and low socioeconomic status (SES) appears and begins to grow, likely exacerbated by summer vacation, as low SES students are less likely to receive continued reading instruction over the break. Guryan is leading the evaluation of a five-year, multidistrict, randomized controlled trial of the READS program, Reading Enhances Achievement During the Summer. Already, the program is showing promise, moderately reducing “summer loss” and improving reading skills.

The program, developed by James Kim of Harvard University, will be administered to approximately 10,000 students in 70 North Carolina elementary schools over the course of the study. Students are sent two books biweekly over summer break. Matched to the student’s interests and reading level, the books are also paired with family activities to support summer reading. Members of the control group receive the books and activities at the start of school. Pretests and posttests, as well as reading tests, are used to measure impact. In addition to monitoring student achievement and overall progress, Guryan will examine different variations of READS that could improve its effectiveness. He will also measure its cost-effectiveness and identify those elements useful for replicating and further expanding the program.

Evaluating After-School Programs
After-school programs have come to be seen as a catchall solution for reducing a host of teenage social ills, from staggering dropout rates to bad attitudes. But do they work? Hedges and IPR associate Barton Hirsch, an education and social policy professor, are currently evaluating After School Matters—a nationally recognized nonprofit organization providing out-of-school activities for teens in the Chicago area. The program operates at more than 60 Chicago public high schools and offers programs and apprenticeships in various domains, such as science, sports, performing arts, and new technologies. Hirsch and Hedges use random assignment to evaluate the effects of the program for 535 youth at 10 high schools and 13 apprenticeships (for the treatment group). The multiyear study, for which results were recently released, measures four outcomes—positive youth development, marketable job skills, academics, and problem behavior. Their work is supported by the William T. Grant and Wallace foundations.

Preventing Truancy in Urban Schools

Jonathan Guryan (l.) talks with Asim Khwaja of Harvard after his joint IPR/Economics Labor & Education Policy Seminar on the market impact of providing school and child learning reports.

While urban high school dropouts have received a great deal of policy attention, the problem almost always starts much earlier with truancy from school. However, very little is known about the risk and protective factors that lead to truancy—and even less about effective remedies. To shed light on this issue, Guryan is leading the first large-scale randomized effectiveness trial of Check & Connect, a structured mentoring, monitoring, and case management program. This intervention focuses on reducing chronic absenteeism and improving school engagement by pairing a mentor with students at risk for dropping out of school. It is one of the few interventions where positive effects for staying in school have been found, based on two small trial studies.

Guryan is studying students in the second, third, fourth, and ninth grades who have a record of chronic absences in Chicago Public Schools (CPS). The study uses two-level randomization, across and within schools, to identify the intervention’s causal effects on the target students and other students within the same school. Researchers will start by randomly assigning between 450 and 700 CPS students to a Check & Connect mentor for two years. They will then compare the assigned students to control groups of more than 4,000 CPS students within the treatment and control schools. Other data, such as arrest records, will also be included, and researchers will conduct personal surveys with students and their parents to pinpoint changes in family structure and dynamics that might contribute to the student’s truancy.

Workshops on Education Data Sets, Methods
In recent years there has been a dramatic increase in the number and quality of student-level administrative data sets collected locally and statewide, but researchers are not necessarily aware of how best to make use of them. Thus, Figlio and Rosenbaum organized a one-day workshop in May 2010 to highlight how such data sets have been creatively and effectively used to study policy-relevant research questions for K-12 and higher education. Presentations were made by Stanford University’s Eric Bettinger on college remediation and Eric Hanushek on teacher job matching, Jane Hannaway of the Urban Institute on value-added measures of student achievement, and Melissa Roderick of the University of Chicago on the Consortium for Chicago School Research. It was sponsored by the Multidisciplinary Program in Education Sciences in Northwestern’s School of Education and Social Policy.

At a December 2 bootcamp for more than 30 journalists of the Education Writers Association, Hedges spoke about research, design, and analysis in education, explaining how the writers can distinguish good studies from bad ones and improve their reporting on studies’ data and results.

Joint IPR/Economics Labor & Education Policy Seminars
IPR continues its joint Labor & Education Policy Seminar Series with the economics department at Northwestern. Speakers in 2010 included Asim Khwaja of Harvard, who discussed how providing school and child test scores affects educational markets, and Magne Mogstad of Statistics Norway and the University of Oslo, who demonstrated the potential for universal childcare to “level the playing field.” IPR’s Jackson also presented his study showing that student outcomes improve when teachers are better “matched” with the right school.