Breakthroughs in Policy Measurement
Get all our news
From left: IPR Director Diane Whitmore Schanzenbach discussed the “arduous work” of policy measurement with Burton Weisbrod, Emma Adam, Bruce Spencer, and Sera Young.
“Policy measurement is something that is long and slow, arduous work,” IPR Director Diane Whitmore Schanzenbach said in opening the IPR@50 panel on policy measurement with four IPR fellows. “[But] what we measure and how we allocate resources for those measurements are extremely important.”
IPR economist Burton Weisbrod set the stage for discussion by pointing to how measurement underlays everything at IPR. Pinpointing the start of performance measurement to the publication of Adam Smith’s Wealth of Nations, he argued that the key question is the same then as now—how to align individuals’ incentives with social goals.
“The belief that we can measure performance implies that we really know what good performance means,” Weisbrod said.
He gave the example of standardized tests. We think they measure what we care about, Weisbrod said, but as so often happens, it is easy to measure and reward that which we readily observe, but much harder to measure—and reward—those things we cannot.
Beyond rewarding performance, accuracy is also a key concern in measurement, and statistician Bruce Spencer has analyzed the accuracy of forecasts in many areas from jury verdicts to earthquake hazard maps and the census.
In discussing the 2020 Census, he explained how its lower funding and the addition of a citizenship question could impact its accuracy.
“The predicted impact of the citizenship question is going to be to depress the responses to the census of the Latino population, among others, and the depressed responses are going to lead to undercounts,” Spencer said. This will especially hurt representation and funding for states like California, Texas, and Florida, but it will also lower in-state representation of undercounted subgroups in every state, blue or red.
Measurement can also “get under the skin.” Emma Adam, a developmental psychobiologist, highlighted her work using biomarkers in one of IPR’s newest research areas, social disparities and health. She and other IPR researchers have pioneered methods to measure human functions from immune system functioning to stress levels through samples of saliva or blood.
“Stress is not just a feeling but has biological implications,” Adam said, underscoring how biomarkers can help pinpoint effects on human health, cognition, and long-term outcomes.
She also pointed to how they can be incorporated in, and benefit, many types of evaluations by revealing the real-life implications of how a particular policy or intervention impacts biology in both small and population-level samples.
Young thanked Weisbrod for his light-hearted recasting of the panel title as “‘Breakdowns’ in Policy Measurement,” as that is exactly what led her to an unexpected area of research, water insecurity.
She was studying food insecurity in Kenya, and during the course of her research, Young discovered how necessary water was for participants in feeding their children. At the time, there was not a scale to measure water insecurity, but today she and a team of researchers have validated the first cross-cultural scale to measure water insecurity globally, HWISE, and are working with institutional partners such as UNESCO and Gallup to launch it.
This scale will help determine what standards countries hold themselves to and provide data to create a policy agenda for water, Young said, noting that performance measurement is “not for the faint of heart.”
Diane Whitmore Schanzenbach (moderator) is the Margaret Walker Alexander Professor and IPR director. Burton Weisbrod is the Cardiss Collins Professor of Economics. Bruce Spencer is professor of statistics. Emma Adam is professor of human development and social policy. Sera Young is assistant professor of anthropology. All are IPR fellows.
Photo credits: Rob Hart.
Published: June 28, 2019. Updated: July 12, 2019.
