Win-Win Partnerships to Improve Education

IPR experts and Chicago Public Schools plug into research opportunities

Becoming a Man
During a visit to Chicago’s Hyde Park Career Academy in February, President Barack Obama participates in a roundtable with students in the B.A.M. program, which is being studied by IPR faculty.

Northwestern University has a long history of working with Chicago Public Schools (CPS) to improve student outcomes. On December 3, Northwestern President and Professor Morton Schapiro and Chicago Mayor Rahm Emanuel announced the creation of Northwestern Academy. This new tutoring program is designed to help bright, financially disadvantaged Chicago public high school freshmen clear some of the hurdles to attending top-tier universities and colleges.

In addition to academic partnerships, Northwestern and IPR faculty are also involved in creating research partnerships with CPS. As the nation’s third largest urban school district, CPS operates in a rapidly changing environment of rising expectations and standardized testing. Implementing new programs and policies to improve student achievement is a constant. And like all other school districts, CPS often faces a dearth of information about whether a particular program will work.

“School districts want to know what is effective,” said IPR economist Jonathan Guryan, “but they often lack the resources and expertise to design and carry out rigorous experiments.”

Since researchers who do know how to run high-quality experiments are interested in answering questions of scientific interest that could serve to inform others, these research partnerships are a “win-win” situation, Guryan said.

“This is why we are working with CPS to find opportunities to conduct high-quality, randomized experiments of their programs,” said Guryan, who chairs IPR’s Education Policy research program, “and why many of IPR’s education experts are currently involved in CPS research projects.”

Some of the projects IPR education researchers are currently running in Chicago’s public schools include examining how a program based in cognitive behavioral therapy might prevent dropouts and violence, the effects of small high schools on student outcomes, how college “coaches” can improve college attendance and completion, and how stress affects academic achievement (see below for project descriptions and the sidebar).

Some of the projects are also part of the Urban Education Lab, an important research initiative launched in 2011 by Guryan and his University of Chicago colleagues Jens Ludwig, Stephen Raudenbush, and Timothy Knowles. They seek to deploy more randomized experiments in evaluations of metropolitan education policies and programs. The lab counts more than 40 affiliated researchers from universities around the country, including IPR education researchers Kirabo Jackson, David Figlio, Diane Whitmore Schanzenbach, and James Spillane. IPR's Education Policy faculty are affiliated with Northwestern's School of Education and Social Policy, the Kellogg School of Management, and the departments of sociology and economics in the Weinberg College of Arts and Sciences.

Guryan also points to how such partnerships could create an informative and timely research “pipeline.”

“If we set up the experiments correctly, then there will be a continuous supply of rigorous findings that could serve not just that particular school district after they take programming decisions, but also others before they take theirs,” Guryan said. “They wouldn’t have to wait for years for the results to come through, they would have the results when they need them.”

Well-run experiments can provide much better information about what does—or does not—work over crunching data, Guryan continued. This is because they compare different treatment groups with counterfactuals—or what was expected to happen had there not been an intervention.

Finally, he underscores that generating research opportunities through collaboration ensures that districts are interested in what is being studied and the findings direct a spotlight on what can be done.

“Collaborating means that policymakers find out about it and use it right away,” Guryan said.

Highlights of IPR Research Projects with CPS

Check & Connect

To shed light on factors that influence truancy, Guryan is leading a team of researchers in an ongoing implementation and evaluation of a program called Check & Connect. It matches students with adult mentors in an effort to increase school attendance and student engagement.

Jon Guryan
Jonathan Guryan

Check & Connect is motivated by findings that show a strong relationship with a pro-social adult helps to shield children from school failure—something that many children, particularly those growing up in distressed family and community environments, often lack. More than 3,000 students in 24 Chicago public schools are involved in  the study, with close to 500 students receiving the intervention. The program involves mentoring, monitoring, and enhancing communication between school and home. Its potential spillover effects on peers of students in the program will also be measured by looking at outcomes for the more than 6,000 control students at schools where the intervention is not being implemented.

Addressing 'Mismatch' for Low-Achieving High School Students

By the time they reach high school, many low-achieving students in distressed communities have been written off. They can be as many as four to seven years behind their grade level, particularly in math. Guryan and his colleagues, including Ludwig, at the Urban Education Lab, housed at the University of Chicago, are working on a new intervention that addresses this problem of “mismatch” for those performing below their current grade level. During 2012-13, the research team carried out a rigorous randomized controlled trial of “math tutoring on steroids,” two-on-one math instruction for an hour each school day. The study involved 106 ninth- and tenth-grade boys at Harper High School in Chicago, which serves mostly low-income students. The tutoring helps the students catch up to their grade level and re-engage in their classes, thereby increasing their chances of graduating. In addition, students also took part in the B.A.M. (Becoming A Man) program, a sociocognitive-skills intervention developed by the Chicago nonprofit organization Youth Guidance. The results from the pilot study will be released soon, but they are promising enough that in fall 2013 the team implemented a large-scale study of the two interventions in 21 Chicago public high schools. In this study, the researchers are seeking to disentangle how each intervention component affects student outcomes, generating better understanding of potential policies and how to better inform future scale-ups.

Small High Schools

Schanzenbach is part of a team that evaluated the performance of the CPS small high schools initiative, using a quasi-experimental design. Using data from the Consortium on Chicago School Research, Schanzenbach and her colleagues analyzed student enrollment patterns and test scores for students entering high school at one of 22 new small schools, comparing them with their eighth-grade classmates who did not attend small schools. Their findings add evidence to the growing consensus that small schools improve academic achievement but do not raise standardized test scores, and that educational interventions aimed at older students are more effective at improving their noncognitive skills than their cognitive skills. The research was conducted with Amy Claessens of the University of Chicago and Lisa Barrow of the Federal Reserve.

College “Coaching”

IPR education and social policy professor James Rosenbaum uses CPS data in his ongoing work examining the processes that increase the number of students who attend college and complete their degree. His most recent study of this transition involves a longitudinal study of nearly all CPS seniors in 58 high schools to determine whether a new counseling model could improve the high school-to-college transition for disadvantaged students. With Jennifer Stephan of the American Institutes for Research, Rosenbaum compared outcomes of schools with the new coaching model to those of schools with the traditional counseling model. They find that students in schools with coaches were more likely to attend four-year colleges versus two-year colleges. Coaches also appear to increase enrollment by increasing the number of students who apply to three or more colleges and complete the federal financial aid form, FAFSA. The study was published in Educational Evaluation and Policy Analysis.

Better Measures of Teen Stress

More than 300 CPS middle and high school students are helping IPR psychobiologist Emma Adam better understand how daily and long-term stress affect children and teens. In an ongoing study with IPR clinical psychologist Edith Chen and DePaul’s Kathryn Grant, Adam is developing a new and more comprehensive measure of adolescent stress. It includes dimensions not normally captured such as poverty, discrimination, and neighborhood stress, as well as family, peer, and academic stressors. The students participated in a full day of on-site testing that includes questionnaires, one-on-one interviews and biological measures of stress. A subset of participants continued with a four-day follow up study. Adam sent iPods home with teens to test their cognitive functioning each morning at home to see how the stressors of the day before affect their stress hormones, sleep, and cognition the next morning.

After School Matters

After School Matters (ASM) is a large-scale apprenticeship program, partnered with CPS, designed to bridge the gap in employment experiences for urban high school students. IPR statistician and education researcher Larry Hedges and youth expert and IPR associate Barton Hirsch and colleagues conducted a rigorous mixed-methods evaluation of ASM using a mock interview assessment they developed with help from senior-level human resource professionals. The evaluation reveals that ASM did not increase marketable job skills compared with a randomly assigned control group, but the data obtained do provide clues as to how apprenticeship experiences might be strengthened. Students who participated in apprenticeships with communication norms and work-based customs that more closely mirrored mainstream standards fared better in the mock interview than control-group students. In response to the findings and a request from CPS, the researchers have piloted a curriculum for interviewing skills in career and technical education classrooms.

Jonathan Guryan is associate professor of human development and social policy and of economics, an IPR fellow, and chair of IPR's Education Policy Program.

Photo credits: Official White House photo by Pete Souza; Jonathan Guryan photo by D. Dry.