Skip to main content

Estimating Impact With Surveys Versus Digital Traces: Evidence From Randomized Cash Transfers in Togo (WP-23-38)

Emily Aiken, Suzanne Bellue, Joshua Blumenstock, Dean Karlan, and Christopher Udry

Do non-traditional digital trace data and traditional survey data yield similar estimates of the impact of a cash transfer program? In a randomized controlled trial of Togo’s COVID-19 Novissi program, endline survey data indicate positive treatment effects on beneficiary food security, mental health, and self-perceived economic status. However, impact estimates based on mobile phone data – processed with machine learning to predict beneficiary welfare – do not yield similar results, even though related data and methods do accurately predict wealth and consumption in prior cross-sectional analysis in Togo. This limitation likely arises from the underlying difficulty of using mobile phone data to predict short-term changes in well-being within a rural population with fairly homogeneous baseline levels of poverty. The researchers discuss the implications of these results for using new digital data sources in impact evaluation.

Emily Aiken, School of Information, University of California, Berkeley

Suzanne Bellue, Assistant Professor of Economics, CREST–ENSAE

Joshua Blumenstock, Chancellor’s Associate Professor, School of Information and Goldman School of Public Policy, University of California, Berkeley

Dean Karlan, Frederic Esser Nemmers Distinguished Professor of Economics and Finance, and IPR Associate, Northwestern University

Christopher Udry, Robert E. and Emily King Professor of Economics and IPR Associate, Northwestern University

Download PDF