Search

Begin New Search Topic Education Content Type Working Paper Author Lant Pritchett Publication Year within 5 Years

1. Let’s Take the Con Out of Randomized Control Trials in Development: The Puzzles and Paradoxes of External Validity, Empirically Illustrated

Author:
Lant Pritchett
Publication Date:
05-2021
Content Type:
Working Paper
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
The enthusiasm for the potential of RCTs in development rests in part on the assumption that the use of the rigorous evidence that emerges from an RCT (or from a small set of studies identified as rigorous in a “systematic” review) leads to the adoption of more effective policies, programs or projects. However, the supposed benefits of using rigorous evidence for “evidence based” policy making depend critically on the extent to which there is external validity. If estimates of causal impact or treatment effects that have internal validity (are unbiased) in one context (where the relevant “context” could be country, region, implementing organization, complementary policies, initial conditions, etc.) cannot be applied to another context then applying evidence that is rigorous in one context may actually reduce predictive accuracy in other contexts relative to simple evidence from that context—even if that evidence is biased (Pritchett and Sandefur 2015). Using empirical estimates from a large number of developing countries of the difference in student learning in public and private schools (just as one potential policy application) I show that commonly made assumptions about external validity are, in the face of the actual observed heterogeneity across contexts, both logically incoherent and empirically unhelpful. Logically incoherent, in that it is impossible to reconcile general claims about external validity of rigorous estimates of causal impact and the heterogeneity of the raw facts about differentials. Empirically unhelpful in that using a single (or small set) of rigorous estimates to apply to all other actually leads to a larger root mean square error of prediction of the “true” causal impact across contexts than just using the estimates from non-experimental data from each country. In the data about private and public schools, under plausible assumptions, an exclusive reliance on the rigorous evidence has RMSE three times worse than using the biased OLS result from each context. In making policy decisions one needs to rely on an understanding of the relevant phenomena that encompasses all of the available evidence.
Topic:
Development, Economics, Education, and Public Policy
Political Geography:
Global Focus

2. Learning Equity Requires More than Equality: Learning Goals and Achievement Gaps between the Rich and the Poor in Five Developing Countries

Author:
Maryam Akmal and Lant Pritchett
Publication Date:
02-2019
Content Type:
Working Paper
Institution:
Center for Global Development
Abstract:
The Sustainable Development Goals (SDGs) for education include the goal that “all youth...achieve literacy and numeracy” (Target 4.6). Achieving some absolute standard of learning for all children is a key element of global equity in education. Using the Annual Status of Education Report (ASER) data from India and Pakistan, and Uwezo data from Kenya, Tanzania, and Uganda that test all children of given ages, whether in school or not, on simple measures of learning in math, reading (local language), and English, we quantify the role of achieving equality between the richest 20% and the poorest 40% in terms of grade attainment and learning achievement toward accomplishing the global equity goal of universal numeracy and literacy for all children. First, excluding Kenya, equalizing grade attainment between children from rich and poor households would only close between 8% (India) and 25% (Pakistan) of the gap to universal numeracy, and between 8% (Uganda) and 28% (Pakistan) of the gap to universal literacy. Second, children from the poorest 40% of households tend to have lower performance in literacy and numeracy at each grade. If such children had the learning profiles of children from rich households, we would close between 16% (Pakistan and Uganda) and 34% (India) of the gap to universal numeracy, and between 13% (Uganda) and 44% (India) of the gap to universal literacy. This shows that the “hidden exclusion” (WDR, 2018) of lower learning at the same grade levels—a gap that emerges in the earliest grades—is a substantial and often larger part of the equity gap compared to the more widely documented gaps in enrollment and grade attainment. Third, even with complete equality in grade attainment and learning achievement, children from poor households would be far from the equity goal of universal numeracy and literacy, as even children from the richest 20% of households are far from universal mastery of basic reading and math by ages 12-13. Achieving universal literacy and numeracy to accomplish even a minimal standard of global absolute equity will require more than just closing the rich-poor learning gap, it will take progress in learning for all.
Topic:
Development, Education, Sustainable Development Goals, and Language
Political Geography:
Pakistan, Kenya, Africa, Middle East, India, Asia, and Tanzania

3. The Limits of Accounting-Based Accountability in Education (and Far Beyond): Why More Accounting Will Rarely Solve Accountability Problems

Author:
Dan Hoing and Lant Pritchett
Publication Date:
05-2019
Content Type:
Working Paper
Institution:
Center for Global Development
Abstract:
Accountability is rightly at the center of the conversation regarding how to improve governance systems, particularly health and education systems. But efforts to address accountability deficits often focus primarily on improving what can be counted and verified—what we term “accounting-based accountability.” We argue that introducing greater accounting-based accountability will only very rarely be the appropriate solution for addressing accountability problems. We illustrate this by exploring the role of Accountability ICT in (not) improving education system performance. Strengthening “real” accountability is not the same as improving data systems for observation and verification, and often attempts at the latter undermine the former. The development discourse’s frequent semantic misunderstanding of the term “accountability” has pernicious real-world consequences with real effects on system reform efforts and ultimately global welfare.
Topic:
Education, Governance, Accountability, and Welfare
Political Geography:
Global Focus