Data Use, Quality & Cost in Policy Research
How Schools Are Chosen for Research
As education policymakers increasingly rely on evidence, researchers have conducted more cluster-randomized trials. But much work remains be done in reviewing those trials’ methodology, specifically how researchers choose which schools to include as what works for one school or district may not work for another. In a recent working paper, IPR statistician Beth Tipton and her co-authors examine 34 such trials to determine whether their samples are truly representative of particular populations of schools and students. They compare the sample data from those studies, funded by the Institute of Education Sciences and selected and evaluated by Tipton and her co-authors, to general population data from the U.S. Department of Education. They find that recruitment for studies is heavily dependent on pre-existing local relationships between researchers and the schools that are ultimately selected. Therefore, those schools skew, like the universities, larger in size and more urban than the population being studied. They find this poses major challenges to any generalizations drawn from such studies. The researchers recommend three major changes to recruitment: including the sample-collection methodology in the grant proposal, increasing training for sample collection, and establishing best practices for school recruitment.
Predicting Homeowners' Gender
Homeownership is an essential tool to building wealth, but who owns a home? In Population Research and Policy Review, IPR sociologist Julia Behrman and Doron Shiffer-Sebba note that shared ownership between married couples is an assumption in most surveys, creating higher joint ownership estimates and misleading patterns in wealth. To get a clearer picture of who actually holds wealth in the form of homeownership, the researchers apply gender R’s algorithms to distinguish gender through names based on data from the Social Security Administration. They examined 257,764 properties from tax assessor data in Detroit and Philadelphia and combined the data with Census Zip code-level data from 2013–2017 American Community Survey 5-year estimates to examine race, marriage status, education, and income. The researchers categorized homeownership by female-sole owners, male-sole owners, mixed ownership, or other, which meant more than two owners or owners of the same gender. Female-sole owners were the largest category in Detroit (49%) and Philadelphia (36%), and mixed male-female ownership was the lowest in Detroit (12%) and somewhat lower in Philadelphia (33%). Even though the researchers found more female owners, women were more likely to own smaller and lower-value homes than males. In Philadelphia, women also tend to be in Zip codes with lower education and fewer high earners. The results provide new insights into understanding wealth and homeownership by highlighting gender differences, specifically that more women, not couples, own homes.
What Is a Realistic Estimate of COVID-19 Infection Rates?
Epidemiologists and policymakers accept that due to imperfections in testing and data collection, the actual rate of COVID-19 infection is likely higher than reported. Given those uncertainties, how can researchers more accurately estimate the true infection rate? In a working paper, IPR economist Charles F. Manski and Cornell University’s Francesca Molinari (PhD 2003) explore new upper and lower limits for those rates by combining existing data with assumptions about the infection rate in the untested population, as well as those about the accuracy of current tests. To explain the difficulty of setting those bounds, they examine three key hotspots, using data on the numbers of individuals tested and numbers of positive test results in Italy, Illinois, and New York. They find that due to the lack of information about the infection rate in the large untested population, as well as issues with the accuracy of current testing methods, the infection rate might be substantially higher than reported. They also find the infection fatality rate in Italy is substantially lower than reported. While the bounds can be narrowed by imposing stronger assumptions about the infection rate, random testing would significantly narrow the bounds, along with the development of a better understanding regarding how testing predicts infection rate. Manski is Board of Trustees Professor in Economics.