Colloquia - Spring 2012


IPR Colloquia

Larry Hedges

Board of Trustees Professor of Statistics & Social Policy and IPR Fellow

“Improving the Generalizability of Education Field Experiments”

Abstract: In educational policy research, how does one determine if the results of a well-designed experiment will apply to policy-relevant populations? Hedges begins by formalizing some subjective notions of generalizability in terms of estimating average treatment effects in well-defined inference populations. The problem is how to use a study sample to estimate parameters of the distribution of treatment effects (e.g., the average treatment effect) in an inference population. When study samples are not probability samples, the inference process relies on matching the study sample to the inference population on a potentially large number of covariates that are related to variation in treatment effects. Hedges outlines methods that can, under definable assumptions, yield estimates of the population average treatment effects that are unbiased (or nearly so) with a standard error that depends largely on how well the study sample matches the inference population. If the standard error is reasonably small, the study sample yields generalizable effects, but if it is large (or even infinite, as it can be), the evidence in the study sample has little or no generalizability to the inference population. He ends his discussion with some examples.

April 2, 2012 • Chambers Hall, 600 Foster St. • IPR


Steven Epstein

John C. Shaffer Professor in the Humanities, Professor of Sociology, and IPR Associate, Northwestern University

“Sexual Health as Buzzword: Competing Stakes and Proliferating Agendas”

Abstract: “Sexual health” is one of the great buzzwords of the early 21st century. The recent, exponential growth of discourses, practices, techniques, and industries that reference or profess the goal of sexual health marks a new moment in the history of engagement by health institutions within the domain of sexuality. At the same time, the convergence around the specific term masks a remarkable diversity of scientific, political, economic, and practical agendas that sometimes coexist, and at other times directly compete. Epstein seeks to understand the contexts in which the term has arisen, the consequences of attempts to lay claim to it, the kinds of bodies and embodied subjectivity that are linked to its uses, and its implications for what we imagine sexuality to be.

April 9, 2012 • Chambers Hall, 600 Foster St. • IPR/CS2


Lori Beaman

Assistant Professor of Economics and IPR Fellow

“Why Don’t Women Get Job Referrals? Evidence from a Recruitment Experiment in Malawi”

Abstract: Women are less likely to use friends and relatives as part of their job search. Therefore, the use of social connections in job referrals may disadvantage female job seekers, contributing to women’s worse labor market outcomes. Beaman and her colleagues designed a field experiment in Malawi where applicants took part in a competitive selection process to become qualified for future surveyor positions with a local firm. The researchers investigated whether skilled women are in fact left out of social network-based referrals and whether this gender gap is attributable either to relative difficulties in identifying qualified women, taste-based discrimination, or statistical discrimination. They find that women do not benefit as much from job referrals as men. The constraint on attracting high quality female candidates through referrals appears to be twofold: Men prefer not to refer women, and women make poorer quality referrals when they do refer women.

April 16, 2012 • Chambers Hall, 600 Foster St. • IPR


Morton Schapiro

President, Professor (Economics/Kellogg/SESP), and IPR Fellow, Northwestern University

“Educational ‘Goodwill’: Measuring the Intangible Assets at Highly Selective Private Colleges and Universities”

Abstract: Seeking to learn more about the factors associated with the loss of admitted students to other competing schools, Schapiro and his colleagues utilize data on the head-to-head loss rate for students accepted at Williams College, but who opt to enroll elsewhere. For example, they employ data that measure the fraction of students admitted to Williams and Amherst (or Harvard or Yale, etc.) but who opt to attend Amherst (or Harvard or Yale, etc.) instead of Williams. They then model this head-to-head loss rate using data from a variety of sources. A better understanding of the head-to-head loss rate can assist an institution in the competition for high quality students. Importantly, it can also shed light on the degree to which some part of the loss rate might be due to “intangible” differences between the schools being compared. These intangibles (positive or negative) might grant a school greater success (or failure) in the market for students than an objective accounting of its characteristics might suggest. Such an advantage (or disadvantage) is closely aligned with the business concept of “goodwill.” The researchers present preliminary evidence on how a quantitative measure of educational goodwill can be computed.

April 23, 2012 • Chambers Hall, 600 Foster St. • IPR


Christopher Kuzawa

Associate Professor of Anthropology and IPR Fellow, Northwestern University

“Fatherhood and Testosterone in the Philippines: Policy Implications of the Human Strategy of Cooperative Child Rearing”

Abstract: Humans are among the rare mammals in which fathers are involved in rearing offspring, which recent work suggests has left its mark on male biology and behavior. One example is the hormone testosterone, which is known to be lower in fathers than in non-fathers. Conventional wisdom has suggested that a drop in testosterone in human fathers could be the body’s way of helping fathers shift priorities from mating activities to parenting. However, past human research on the relationship between hormones and fatherhood has been cross-sectional, making it unclear whether fatherhood leads to a decline in testosterone, or alternatively, whether men with lower testosterone are more likely to enter stable pair-bonds and become fathers. Kuzawa will discuss findings from the first longitudinal test of this hypothesis conducted in a large population-based cohort in the Philippines. The findings strongly support the hypothesis that fatherhood lowers a man’s testosterone. Evidence that low testosterone facilitates care—while care further lowers testosterone—suggests important biosocial feedbacks in the maintenance of male caregiving bonds. The policy implications of these findings will be discussed.

May 7, 2012 • Chambers Hall, 600 Foster St. • IPR


Brian Mustanski

Associate Professor of Medical Social Sciences, IPR Associate, and Director, IMPACT LGBT Health and Development Program, Northwestern University

“Moving Along the Translational Spectrum to Address Health Disparities in LGBT Youth”

Abstract: Lesbian, gay, bisexual, and transgender (LGBT) youth experience a number of health disparities relative to their heterosexual and cisgendered peers. These disparities are found in mental, physical, and sexual health outcomes. While the existence of some of these disparities has been well documented in the research literature, there has been less research to characterize risk and protective mechanisms, and very little research focused on the development, testing, and implementation of clinical and prevention interventions with this population. Mustanski will describe his program of translational research on the health of LGBT youth. This program has included epidemiological studies of the prevalence of health issues, longitudinal studies to characterize developmental patterns and identify risk and protective processes, the development and testing of prevention intervention, and the collaborative transfer of promising interventions to the community for implementation. In addition to describing some of his team’s research, he will talk about the obstacles and benefits of moving research along the translational spectrum.

May 14, 2012 • IPR Conference Room, 617 Library Place • IPR/CS2


David Uttal

Director, Cognitive Psychology Program, and Professor of Psychology and Education, Northwestern University

“Spatial Thinking and Science, Technology, Engineering, and Mathematics (STEM) Education: When, Why, and How”

Abstract: Are spatial skills important for STEM education and practice? The answer seems to be both yes and no. Some research has revealed strong and consistent correlations between spatial abilities and STEM career choice. However, other studies have shown that spatial skills do not seem to predict performance among experts. Uttal begins by trying to resolve this seeming paradox, suggesting that spatial skills serve more as a barrier or gateway to STEM entry, rather than as a critical part of STEM practice. This argument then lays the foundation for a case in which spatial training and experiences might facilitate STEM achievement. Uttal presents the results of a meta-analysis that reveals that spatial training is effective, that it transfers, and that it can endure. Taken together, the findings he discusses help to constrain the possible answers to the questions regarding when, why, and how spatial thinking matters in STEM education—and also suggest what we can and cannot expect to happen if we attempt to improve students’ spatial thinking. Training in spatial cognition might be a low-cost, high-benefit intervention to increase the number of students who can succeed in STEM fields.

May 21, 2012 • IPR Conference Room, 617 Library Place • IPR


Q-Center Colloquia

John Ioannidis

C.F. Rehnborg Professor in Disease Prevention, Director of the Stanford Prevention Research Center, and Professor of Medicine, Health Research & Policy, and Statistics, Stanford University

“Empirical Testing of Excess Significance Bias”

Abstract: Empirical evaluations show that almost all biomedical studies highlight statistically significant results with p-values of less than 0.05, yet the large majority of these statistically significant claims fail to be replicated when larger and better studies are conducted. Publication bias, though often thought to be the primary explanation for excess significance, actually accounts for only a minority of the problem. Ioannidis will critique conventional methods for detecting publication bias and present a new approach that tests for excess significance in wider domains or whole fields of research. He will provide examples from diverse fields including clinical trials, genetics, and studies of brain volume abnormality.

April 24, 2012 • IPR Conference Room, 617 Library Place • Q-Center


Christopher Winship

Diker-Tishman Professor of Sociology, Harvard University

“Sexual Health as Buzzword: Competing Stakes and Proliferating AgendasDoes Going to School Make You Smarter? New Methodology and New Estimates”

Abstract: Winship presents evidence demonstrating that nearly all of the literature examining the effects of education on IQ has mismeasured change by using an age-standardized measure of IQ. His research shows how synthetic cohorts can be used to estimate education’s effect, and in doing so, examines three issues: the potentially differential impact of education and age, the distinction between permanent versus transitory effects, and the problem of endogeneity. He finds that, on average, a year of schooling increases IQ by 3.48 points, although only 1.95 points of this effect is permanent.

May 8, 2012 • IPR Conference Room, 617 Library Place • Q-Center


Linda Collins

Professor of Human Development & Family Studies and Statistics, and Director of The Methodology Center, Penn State

“The Multiphase Optimization Strategy (MOST) for Engineering Better Behavioral Interventions”

Abstract: Interventions are used widely today for the prevention and treatment of health problems and the promotion of educational attainment. Behavioral and educational interventions are typically developed and evaluated using a treatment package approach where the intervention is assembled a priori and evaluated by means of a two-group randomized controlled trial (RCT). Refinement of the intervention is often done by conducting post-hoc analyses on data from the RCT. Collins will suggest an alternative framework for building and evaluating behavioral interventions, called the Multiphase Optimization Strategy (MOST), which is a principled approach to intervention optimization inspired by ideas from engineering. MOST includes the RCT for intervention evaluation, but also includes other steps before the RCT aimed at intervention optimization. Using MOST, behavioral and educational interventions can be optimized using criteria chosen by the intervention scientist. The MOST framework relies heavily on resource management by strategic choice of highly efficient experimental designs. Collins proposes that MOST offers several benefits, including more rapid long-run improvement of interventions, without requiring a dramatic increase in intervention research resources.

June 5, 2012 • IPR Conference Room, 617 Library Place • Q-Center


IPR Seminar on Performance Measurement

Devin Pope

Assistant Professor of Behavioral Science and Robert King Steel Faculty Fellow, Booth School of Business, University of Chicago

“Improving College Performance and Retention the Easy Way: Unpacking the ACT Exam”

Abstract: Improving college performance and retention can be difficult. Pope and his colleagues propose a simple and low-cost change in the way colleges use the ACT exam in their admission decisions that can greatly increase their ability to identify students at a high risk of under-performing and dropping out. Specifically, the researchers show that only two of the four subtests of the ACT — English and Mathematics— can effectively predict outcomes in college. This result is robust across various samples, specifications, and outcome measures. The study demonstrates that by eliminating the noise associated with the two non-predictive subtests, student-college matches can be significantly improved.

April 4, 2012 • IPR Conference Room, 617 Library Place • IPR/SPM


Jill Horwitz

Professor of Law and Professor of Health Management & Policy, University of Michigan

“Challenges to Regulatory Decentralization: Lessons from State Health Technology Regulation”

Abstract: Decentralized regulation is thought to better reflect the views of local residents and to encourage experimentation than centralized regulation. When a state’s regulatory choices affect other states, however, these benefits can be compromised. To examine the implications of such externalities, Horwitz and her colleagues consider the case of certificate of need laws (CON), state laws that require providers to obtain licenses before adopting various types of healthcare technology. In particular, they analyze the cross-border effects of these laws on the number and location of Magnetic Resonance Imaging (MRI) providers. The researchers find a large effect on the number and location of MRIs near borders between unregulated and regulated states. These results may help explain conflicting studies on whether and why CON regulation may have failed to control costs and quantity.

April 18, 2012 • IPR Conference Room, 617 Library Place • IPR/SPM


Eugene Steuerle

Richard B. Fisher Chair and Institute Fellow, The Urban Institute

“Restoring Fiscal Democracy: Can the Budget Squeeze Give Us Another Shot at Greatness?”

Abstract: Today the nation (and much of the developed world) is at a major fiscal turning point, presaged but misunderstood by focus on one symptom—deficits. In point of fact, the major political parties have increasingly conspired to create a series of public programs that automatically grow so fast that they claim every future dollar of revenue. At the same time, they have reduced taxes far below what is necessary to pay the bills, largely by passing burdens onto future taxpayers. Among the deadly economic and political consequences are a lack of fiscal space to respond to new recessions or needs and a budget for a declining nation that underinvests in its children. The solution rests not just in deficit reduction to maintain the budget priorities of dead and retired legislators, but in restoring to future generations the fiscal freedom to choose their own priorities, allocate their own resources, and reach for their own stars.

April 25, 2012 • IPR Conference Room, 617 Library Place • IPR/SPM


Iftikhar Hussain

Research Associate, Centre for Economic Performance, London School of Economics and Political Science; Assistant Professor of Economics, University of Sussex

“Subjective Performance Evaluation in the Public Sector: Evidence from School Inspections”

Abstract: Performance measurement in the public sector is largely based on objective metrics, which might be subject to gaming behavior. This paper investigates a novel subjective performance evaluation system where independent inspectors visit schools at very short notice, publicly disclose their findings, and sanction schools rated “fail.” First, Hussain demonstrates that inspection ratings can aid in distinguishing between more and less effective schools, even after controlling for standard observed school characteristics. Second, exploiting a natural experiment, he shows that a “fail” inspection rating leads to test score gains for primary school students. Although there is some fade-out, test score gains for the treated group persist even after these students move into the secondary phase of their education. He finds no evidence to suggest that “fail” schools are able to inflate test score performance by gaming the system. Oversight by inspectors might play an important role in mitigating such strategic behavior.

May 2, 2012 • IPR Conference Room, 617 Library Place • IPR/SPM


Joint IPR/Economics Education & Labor Seminars

Till Marco von Wachter

Associate Professor of Economics, Columbia University

“The Effects of Extended Unemployment Insurance over the Business Cycle on
Employment and Earnings”

Abstract: One goal of extending the duration of unemployment insurance (UI) in recessions is to increase UI coverage in the face of longer unemployment spells. Although it is a common concern that such extensions may themselves raise nonemployment durations, it is not known how recessions would affect the magnitude of this moral hazard. To obtain causal estimates of the differential effects of UI in booms and recessions, this paper exploits the fact that, in Germany, potential UI benefit duration is a function of exact age which is itself invariant over the business cycle. The researchers implement a regression discontinuity design separately for 20 years and correlate their estimates with measures of the business cycle. They find that the nonemployment effects of a month of additional UI benefits are, at best, somewhat declining in recessions. Yet, the UI exhaustion rate, and therefore the additional coverage provided by UI extensions, rises substantially during a downturn. The ratio of these two effects represents the nonemployment response of workers weighted by the probability of being affected by UI extensions. Hence, their results imply that the effective moral hazard effect of UI extensions is significantly lower in recessions than in booms. Using a model of job search with liquidity constraints, they also find that, in the absence of market-wide effects, the net social benefits from UI extensions can be expressed either directly in terms of the exhaustion rate and the nonemployment effect of UI durations, or as a declining function of their measure of effective moral hazard.

April 5, 2012 • 3245 Anderson Hall • IPR/Joint Economics


Elizabeth Cascio

Associate Professor of Economics, Dartmouth College

“Valuing the Vote: The Redistribution of Voting Rights and State Funds Following the Voting Rights Act of 1965”

Abstract: The Voting Rights Act of 1965 (VRA) has been called one of the most effective pieces of civil rights legislation in U.S. history, having generated dramatic increases in black voter registration and black voter turnout across the South. Cascio and her colleague show that the expansion of black voting rights in some Southern states was brought about by one requirement of the VRA—the elimination of literacy tests at voter registration. This was accompanied by a shift in the distribution of state aid toward localities with higher proportions of black residents, who held newfound power to affect the re-election of state officials. The researchers also found relatively large increases in voter turnout in areas with higher black population shares in treated states over this time period. Their estimates imply an elasticity of state transfers to counties with respect to turnout in presidential elections, the closest available measure of enfrachisement, of roughly 1.

April 12, 2012 • 3245 Anderson Hall • IPR/Joint Economics


Jonah Rockoff

Sidney Taurel Associate Professor of Business, Graduate School of Business, Columbia University

“The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood”

Abstract: Are teachers’ impacts on students’ test scores (“value-added”) a good measure of their quality? This question has sparked debate largely because of disagreement about whether value-added (VA) provides unbiased estimates of teachers’ impacts on student achievement and whether high-VA teachers improve students’ long-term outcomes. Rockoff and his colleagues address these two issues by analyzing school district data from grades three through eight for 2.5 million children linked to tax records on parent characteristics and adult outcomes. The researchers find no evidence of bias in VA estimates using previously unobserved parent characteristics and a quasi-experimental research design based on changes in teaching staff. Students assigned to high-VA teachers are more likely to attend college, attend higher-ranked colleges, earn higher salaries, live in higher SES neighborhoods, and save more for retirement. They are also less likely to have children as teenagers. On average, a one standard deviation improvement in teacher VA in a single grade raises earnings by about 1 percent at age 28. Replacing a teacher whose VA is in the bottom 5 percent with an average teacher would increase the present value of students’ lifetime income by more than $250,000 for the average classroom in our sample. The researchers conclude that good teachers create substantial economic value and that test score impacts are helpful in identifying such teachers.

April 19, 2012 • 3245 Anderson Hall • IPR/Joint Economics


Hilary Hoynes

Professor of Economics, University of California, Davis

“Income, the Earned Income Tax Credit, and Infant Health”

Abstract: This paper evaluates the health impact of a central piece in the U.S. safety net for families with children: the Earned Income Tax Credit (EITC). Using tax-reform-induced variation in the federal EITC and the presence and generosity of state EITCs, Hoynes and her colleagues examine the impact of the credit on infant health outcomes. They find that increased EITC income reduces the incidence of low-weight births and increases mean birth weight. For single, low-education (12 years or less) mothers, an increase of $1,000 in EITC income is associated with a 7 percent reduction in the rate of low-weight births. These impacts are evident in difference-in-difference models and event study analyses. The researchers conclude that the sizeable increase in income for eligible families significantly improved birth outcomes for both whites and African Americans, with larger impacts for births to African American mothers.

May 7, 2012 • 3245 Anderson Hall • IPR/Joint Economics


Jesse Rothstein

Associate Professor of Public Policy and Economics, University of California, Berkeley

“Teacher Quality Policy When Supply Matters”

Abstract: Many districts, encouraged by the U.S. Department of Education, are moving to implement new teaching contracts that provide bonuses to effective teachers and/or allow for the removal of ineffective teachers. An important hoped-for mechanism is self-selection—these policies should make teaching more attractive to those who are effective and less attractive to those who are not. But extant models of the selection effects are extremely simplistic. Rothstein develops a model of the teacher labor market, incorporating self-selection through entry and voluntary exit, noisy performance measurement, Bayesian learning, risk aversion, and on-the-job search. He then uses this model to study the effects of alternative contracts on teaching effectiveness and on the teacher wage bill. Simulations with reasonable parameter values indicate that both performance pay and firing policies can raise student achievement, albeit at significant cost. Accounting for labor market interactions makes each policy notably less attractive than when these interactions are ignored.

May 31, 2012 • 3245 Anderson Hall • IPR/Joint Economics