INSTITUTE FOR POLICY RESEARCH

In a lecture at the National Academy of Sciences, IPR economist Charles F. Manski called for scientists and agencies to report uncertainty when communicating policy analysis and research.

In an era of questioning of all facts, it is more important than ever to face up to and report uncertainty when communicating policy analysis and scientific research, according to IPR economist Charles F. Manski.

Speaking at the National Academy of Sciences on November 17, Manski focused on uncertainty in policy analysis and communication. He pointed out how common it is to see discussions of public policy or predictions about the future that never express any uncertainty in their conclusions. He dubs this “incredible certitude.”

It is routine, Manski said, for the Congressional Budget Office (CBO) and federal statistical agencies to report exact numbers for what are estimates that should include margins of error or ranges of possibilities. For example, the CBO scores—that is, predicts—the federal debt implications of pending legislation. These are expressed as precise dollar figures, such as $138 million, despite how difficult it is to foresee new laws’ impact. The CBO gives no quantitative measure of the uncertainty of their prediction. Uncertainty is also missing in other official statistics, such as GDP growth and the poverty rate.

One U.S. agency earned praise from Manski, however, for broadcasting the “best expression of uncertainty” he has seen. The National Weather Service tweeted as Hurricane Harvey began moving ashore in Texas on August 27, 2017: “This event is unprecedented & all impacts are unknown & beyond anything experienced.”

Why do scientists, agencies, and journalists report scientific findings and policy predictions with incredible certitude? Manski underscores how the scientific community rewards strong, novel findings, and the public wants and prefers unequivocal policy recommendations.

Manski has advocated for more acknowledgement of uncertainty since his 2013 book on the subject, Public Policy in an Uncertain World: Analysis and Decisions (Harvard University Press). But he has steadily encountered resistance from government statisticians who believe that members of Congress are unwilling or even unable to deal with uncertainty. 

It is possible to present predictions with a measure of uncertainty. Manski points to the Bank of England’s use of fan charts in its estimations of GDP growth as an excellent way to express uncertainty. 

If more policy analysts revealed their level of uncertainty, Manski said, they would be more trustworthy as well as more honest: “Express the uncertainty because if you want someone to believe what you do know, then you have to be upfront about what you don’t know.”

Charles F. Manski is Board of Trustees Professor in Economics and an IPR fellow. Sources for this talk to the NAS Arthur M. Sackler Colloquium are his book, Public Policy in an Uncertain World: Analysis and Decisions, Harvard University Press, 2013, and his 2015 article, “Communicating uncertainty in official economic statistics: An appraisal fifty years after Morgenstern” in the Journal of Economic Literature.