Treating Ambiguity with Diversification
IPR economist shows how to make more informed policy choices

Fall, 2009, Volume 31, Number 1

Charles Manski

Ambiguity is part of our everyday lives—consider the recent performance of your 401(k) portfolio or the likelihood of your company announcing layoffs due to the current economic crisis. Yet dealing with ambiguity in research and policymaking circles is problematic.

“The scientific community rewards those who produce unambiguous findings, and the public rewards those who offer clear-cut policy recommendations,” said Charles F. Manski, IPR faculty fellow and Board of Trustees Professor of Economics.

“But all too often, policy choices are made with no clear understanding of which policy would provide the best outcomes,” Manski continued. “Research typically only provides part of the knowledge needed to make an informed decision.”

This uncertainty stems from inherent statistical imprecision and identification issues—or trying to apply what one might have learned from a relatively small sample group to a larger or a different population.

Some of these limits are linked to measurement problems. “We often want to understand the long-term outcomes of treatments, but studies often reveal only immediate outcomes,” Manski clarified. “So, for example, trying to extrapolate how preschool policies might affect adult outcomes, such as college enrollment, employment, and criminal behavior, can be extremely challenging.” Plus, Manski noted that researchers often interpret their data using assumptions that have little or no foundation.

Manski illustrated the problem with a what-if scenario: Imagine that a virulent new disease, X-Pox, sweeps through a community, infecting everyone in its path. So far every infected person has died, and the community’s remaining inhabitants seem certain to follow. Two possible treatments are proposed—only one is life-saving. Yet no one knows which one, and administering the two in combination to each person would prove fatal. Treatment must begin immediately if there is to be any hope. How can health officials intelligently pick a treatment course for the entire community when they do not know which treatment saves lives and which one kills?

Manski suggested using a diversified treatment plan, or “adaptive diversification,” as one would for a financial portfolio. “You’ve heard that you should diversify to avoid having all your financial nest eggs in one basket,” Manski said. “This can also apply to policy treatments.”

In this case it means dividing the community’s entire population, say into two halves, and administering Treatment A to one and Treatment B to the other. Half of the population will die, but the other half will be saved. The alternative would be administering only one of the two treatments to the entire population such that all would either live or die.

One could reasonably argue for either option, Manski said, but the argument for using diversification strengthens if the infection occurs in two waves instead of one. In this case, those falling ill in the first wave are split into two groups who receive different treatments. By the second wave, policy makers might have gleaned enough knowledge to choose the life-saving treatment for everyone.

“This amounts to conducting a randomized experiment that will yield hard evidence on treatment response, thus allowing health officials to save the remaining population with lower loss of life,” Manski continued. “It copes with ambiguity in the short run and reduces it in the long run.”

Manski’s idea of adaptive diversification holds potential application for myriad social issues, from how to treat disease to providing assistance to the unemployed to sentencing juvenile offenders. However, ethical considerations might inhibit wide adoption of the idea.

“It violates the democratic idea of equal treatment for equals
that is exemplified in the U.S. Constitution’s 14th Amendment, the Equal Protection Clause,” Manski said. But random tax audits, drug testing, and airport screening show that Americans are sometimes willing to accept “unequal” treatment, at least when they face equal probabilities for treatment.

Manski concluded that while choosing the optimal policy with limited knowledge of outcomes is generally not feasible, researchers and policymakers can try to make more reasoned choices. “They should not hide behind shaky assumptions, but face up to ambiguity in their decision making and seek to reduce it over time,” he urged.

For more information, see the related working paper online at www.northwestern.edu/ipr/publications/workingpapers/wpabstracts09/wp0902.html.