RISE_WP-042-Myths_Offical_Measurement_Auditing_Improving_Data_Singh.pdf (4.68 MB)
Myths of Official Measurement:Auditing and Improving Administrative Data in Developing Countries
preprintposted on 2022-09-15, 11:16 authored by Abhijeet Singh
A central, yet understudied, component of state capacity is the quality of administrative data, which is a key enabler of effective policy making. I study this in the context of education systems in India. Using direct audit evidence, I show that levels of student achievement in Madhya Pradesh, from a large census covering approximately 7 million students annually, are severely inflated due to cheating. This distortion affects students at all levels of achievement but is particularly severe for low-performing students. In a follow-up randomized experiment, covering over 2400 schools in a different state (Andhra Pradesh), I evaluate whether tablet-based testing, which makes cheating harder, could reduce distortion. I find paper-based assessments proctored by teachers severely exaggerate achievement, in both private and government schools, but find no evidence of such distortion in tablet-based assessments. These results suggest that business-as-usual learning assessments may be compromised even without high-powered incentives, limiting their usefulness for policy action or research, although it may be possible to curtail such manipulation even at scale. This challenge of data corruptibility, and potentially evidence on reforms to address it, is likely to generalize across sectors.
RISE FundingFCDO, DFAT and the Bill & Melinda Gates Foundation