Estimating Impact With Surveys Versus Digital Traces: Evidence From Randomized Cash Transfers in Togo (WP-23-38)
Emily Aiken, Suzanne Bellue, Joshua Blumenstock, Dean Karlan, and Christopher Udry
Do non-traditional digital trace data and traditional survey data yield similar estimates of the impact of a cash transfer program? In a randomized controlled trial of Togo’s COVID-19 Novissi program, endline survey data indicate positive treatment effects on beneficiary food security, mental health, and self-perceived economic status. However, impact estimates based on mobile phone data – processed with machine learning to predict beneficiary welfare – do not yield similar results, even though related data and methods do accurately predict wealth and consumption in prior cross-sectional analysis in Togo. This limitation likely arises from the underlying difficulty of using mobile phone data to predict short-term changes in well-being within a rural population with fairly homogeneous baseline levels of poverty. The researchers discuss the implications of these results for using new digital data sources in impact evaluation.