Search

Begin New Search Topic Development Political Geography Global Focus Publishing Institution The John F. Kennedy School of Government at Harvard University Publication Year within 10 Years Publication Year within 5 Years

1. How Pratham Learns While Scaling: A Case Study of Adaptive Design and Evaluation

Author:
Jossie Fahsbender, Siddhant Gokhale, and Michael Walton
Publication Date:
11-2023
Content Type:
Working Paper
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
Developing scalable innovations is a central challenge in development. Organizations that succeed in scaling often prioritize measurement, learning, and evaluation, but how this is done is poorly understood. This paper explores the case of Pratham, one of the largest NGOs in primary education and an exceptional learning organization. While Pratham is renowned for its randomized control trial sequence with J-PAL, we underscore this is just one aspect of a wider array of learning activities. These include an iterative process to continuously refine and protocolize solutions as they scale, the use of on-field qualitative insights and quantitative tracking, open feedback channels between the field, state, and central offices, information exchange among content and evaluation teams, and a nationwide assessment of learning levels (ASER). The RCTs were effective because they were embedded within this broader learning process and culture. Pratham learns at three levels: learning to improve children's basic skills, learning as an organization about what does and doesn’t work, and fostering learning by others in the system. Its learning capacity is rooted in deep-seated values and a culture of openness, trust, problem-solving, and the freedom to experiment and learn from failure, inculcated and nurtured by its leadership. Throughout, we use the prism of an Adaptive Evaluation to provide a systematic framework for mirroring and understanding Pratham's organic learning processes (as affirmed by its own leadership). Pratham actively engages with the three main pillars of an Adaptive Evaluation, involving understanding systems, theorizing how to effect change, and iterating its designs. While Pratham’s culture will often not be transferable, the systematic analysis of how Pratham learns can provide a framework for other organizations aspiring to replicate Pratham's success as a learning institution.
Topic:
Development, Economy, Innovation, and Adaptation
Political Geography:
Global Focus

2. Process and Implementation Evaluations: A Primer

Author:
Patricia J. Rogers and Michael Woolcock
Publication Date:
05-2023
Content Type:
Working Paper
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
Beyond assessing whether or not interventions have achieved their stated goals, evaluations can also provide greater understanding—in real-time and at completion—about how, where, for whom, over what time frame, and which aspects of an intervention may (or may not) have succeeded, and thus where improvements might be sought. Moreover, to the extent any intervention is only as good as its implementation, evaluations can also help identify where breakdowns in the delivery system may have occurred and spaces wherein frontline implementers were able to find innovative solutions to local (“binding constraint”) problems. Process and implementation evaluations thus serve the vital purpose of jointly promoting accountability and learning, thereby expanding the common perception of evaluations as external instruments of compliance and discipline to internal procedures for promoting partnerships, innovation, and improvement (organizationally or technically). In this chapter, we describe six different types of process and implementation evaluations and describe their respective strengths and weaknesses in various contexts, for various purposes. As part of collective efforts to enhance the effectiveness of all classes of interventions, impact and process evaluations should be regarded as necessary complements.
Topic:
Development, Accountability, and Intervention
Political Geography:
Global Focus

3. What is public policy success, especially in development?

Author:
Matt Andrews
Publication Date:
09-2022
Content Type:
Working Paper
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
Public policy work is hard, especially when one works in developing countries. It is even difficult to define what success looks like, and thus how to manage towards success. Literature helps manage such difficulty, providing studies that define the concept and show how it can be achieved. A core message from such is that success is multi-dimensional, and practitioner need to focus on multiple criteria when doing their policy work. But what dimensions and criteria matter? And do development practitioners really adopt this multi-dimensional view? Tackling such questions, the current paper reviews 45 applied studies from the public policy, project management and development evaluation literatures to see what they identify as key success criteria and if the practical studies (about development evaluation) are in sync with the more academic messages. Reading across all three literatures, I identify 30 potential success criteria in 6 categories or dimensions (program, impact and endurance, capability, political, stakeholder, and process). I find that the development evaluation literature focuses on a narrow set of 7 criteria, mostly in one dimension (program success) as compared to broader perspectives in the other literatures. This suggests that development practitioners have a narrow view on success, which is out of step with academic views on the topic. A conclusion proposes a broader approach for these practitioners.
Topic:
Development, Public Policy, and Project Management
Political Geography:
Global Focus

4. Managing for Motivation as Public Performance Improvement Strategy in Education & Far Beyond

Author:
Dan Honig
Publication Date:
03-2022
Content Type:
Working Paper
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
People management has an important role to play in improving public agency performance. This paper argues that a ‘Route Y’ managerial approach focused on supporting the empowered exercise of employee judgment will in many circumstances prove superior to conventional reform approaches steeped in ‘Route X’ monitoring and incentives. Returns to Route Y are greater when employees are or can become more “mission motivated” – that is, aligned with the goals of the agency in the absence of monitoring and extrinsic incentives. Returns to Route Y are also greater when monitoring is incomplete or otherwise likely to unproductively distort effort, thus lowering the returns to using performance-linked rewards and penalties. I argue that education systems are one (but far from the only) setting where Route Y is a lever worth focusing on in efforts to improve public performance in the developed and developing world alike.
Topic:
Development, Education, Public Policy, Performance Evaluation, and Project Management
Political Geography:
Global Focus

5. National Development Delivers: And How! And How?

Author:
Lant Pritchett
Publication Date:
05-2021
Content Type:
Special Report
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
Core dual ideas of early development economics and practice were that (a) national development was a four-fold transformation of countries towards: (i) a more productive economy, (ii) a more responsive state, (iii) more capable administration, and (iv) a shared identity and equal treatment of citizens and (b) this four-fold transformation of national development would lead to higher levels of human wellbeing. The second idea is strikingly correct: development delivers. National development is empirically necessary for high wellbeing (no country with low levels of national development has high human wellbeing) and also empirically sufficient (no country with high national development has low levels of human wellbeing). Three measures of national development: productive economy, capable administration, and responsive state, explain (essentially) all of the observed variation in an omnibus indicator of wellbeing, the Social Progress Index, which is based on 58 distinct non-economic indicators. How national development delivers on wellbeing varies, in three ways. One, economic growth is much more important for achieving wellbeing at low versus high levels of income. Two, economic growth matters more for “basic needs” than for other dimensions of wellbeing (like social inclusiveness or environmental quality). Three, state capability matters more for wellbeing outcomes that depend on public production than on private goods (and for some wellbeing indicators, like physical safety, for which growth doesn’t matter at all). While these findings may seem too common sense to be worth a paper, national development--and particularly economic growth—is, strangely, under severe challenge as an important and legitimate objective of action within the development industry.
Topic:
Development, Economics, and Economic Growth
Political Geography:
Global Focus

6. Let’s Take the Con Out of Randomized Control Trials in Development: The Puzzles and Paradoxes of External Validity, Empirically Illustrated

Author:
Lant Pritchett
Publication Date:
05-2021
Content Type:
Working Paper
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
The enthusiasm for the potential of RCTs in development rests in part on the assumption that the use of the rigorous evidence that emerges from an RCT (or from a small set of studies identified as rigorous in a “systematic” review) leads to the adoption of more effective policies, programs or projects. However, the supposed benefits of using rigorous evidence for “evidence based” policy making depend critically on the extent to which there is external validity. If estimates of causal impact or treatment effects that have internal validity (are unbiased) in one context (where the relevant “context” could be country, region, implementing organization, complementary policies, initial conditions, etc.) cannot be applied to another context then applying evidence that is rigorous in one context may actually reduce predictive accuracy in other contexts relative to simple evidence from that context—even if that evidence is biased (Pritchett and Sandefur 2015). Using empirical estimates from a large number of developing countries of the difference in student learning in public and private schools (just as one potential policy application) I show that commonly made assumptions about external validity are, in the face of the actual observed heterogeneity across contexts, both logically incoherent and empirically unhelpful. Logically incoherent, in that it is impossible to reconcile general claims about external validity of rigorous estimates of causal impact and the heterogeneity of the raw facts about differentials. Empirically unhelpful in that using a single (or small set) of rigorous estimates to apply to all other actually leads to a larger root mean square error of prediction of the “true” causal impact across contexts than just using the estimates from non-experimental data from each country. In the data about private and public schools, under plausible assumptions, an exclusive reliance on the rigorous evidence has RMSE three times worse than using the biased OLS result from each context. In making policy decisions one needs to rely on an understanding of the relevant phenomena that encompasses all of the available evidence.
Topic:
Development, Economics, Education, and Public Policy
Political Geography:
Global Focus

7. Horrible Trade-offs in a Pandemic: Lockdowns, Transfers, Fiscal Space, and Compliance

Author:
Ricardo Hausmann and Ulrich Schetter
Publication Date:
07-2020
Content Type:
Working Paper
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
In this paper, we develop a heterogeneous agent general equilibrium framework to analyze optimal joint policies of a lockdown and transfer payments in times of a pandemic. In our model, the effectiveness of a lockdown in mitigating the pandemic depends on endogenous compliance. A more stringent lockdown deepens the recession which implies that poorer parts of society find it harder to subsist. This reduces their compliance with the lockdown, and may cause deprivation of the very poor, giving rise to an excruciating trade-off between saving lives from the pandemic and from deprivation. Lump-sum transfers help mitigate this trade-off. We identify and discuss key trade-offs involved and provide comparative statics for optimal policy. We show that, ceteris paribus, the optimal lockdown is stricter for more severe pandemics and in richer countries. We then consider a government borrowing constraint and show that limited fiscal space lowers the optimal lockdown and welfare, and increases the aggregate death burden during the pandemic. We finally discuss distributional consequences and the political economy of fighting a pandemic.
Topic:
Development, Government, Political Economy, Inequality, Economic Growth, Fiscal Policy, Pandemic, and COVID-19
Political Geography:
Global Focus

8. Creative Destruction or Idiot Winds: Schumpeterian Theory Meets the Educational Sector in Developing Countries

Author:
Mark H. Moore
Publication Date:
08-2019
Content Type:
Working Paper
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
This is one of a series of working papers from “RISE"—the large-scale education systems research programme supported by the UK’s Department for International Development (DFID), Australia’s Department of Foreign Affairs and Trade (DFAT), and the Bill and Melinda Gates Foundation.
Topic:
Development, Education, Governance, and Developing World
Political Geography:
Global Focus

9. When Do Development Projects Enhance Community Well-Being?

Author:
Michael Woolcock
Publication Date:
06-2019
Content Type:
Working Paper
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
Many development agencies and governments now seek to engage directly with local communities, whether as a means to the realization of more familiar goals (infrastructure, healthcare, education) or as an end in itself (promoting greater inclusion, participation, well-being). These same agencies and governments, however, are also under increasing pressure to formally demonstrate that their actions ‘work’ and achieve their goals within relatively short timeframes – expectations which are, for the most part, necessary and desirable. But adequately assessing ‘community-driven’ approaches to development requires the deployment of theory and methods that accommodate their distinctive characteristics: building bridges is a qualitatively different task to building the rule of law and empowering minorities. Moreover, the ‘lessons’ inferred from average treatment effects derived from even the most rigorous assessments of community-driven interventions are likely to translate poorly to different contexts and scales of operation. Some guidance for anticipating and managing these conundrums are provided.
Topic:
Development, Government, Infrastructure, and International Development
Political Geography:
Global Focus and United States of America

10. Smart Development Banks

Author:
Eduardo Fernández-Arias, Ricardo Hausmann, and Ugo Panizza
Publication Date:
04-2019
Content Type:
Working Paper
Institution:
The John F. Kennedy School of Government at Harvard University
Abstract:
The conventional paradigm about development banks is that these institutions exist to target well-identified market failures. However, market failures are not directly observable and can only be ascertained with a suitable learning process. Hence, the question is how do the policymakers know what activities should be promoted, how do they learn about the obstacles to the creation of new activities? Rather than assuming that the government has arrived at the right list of market failures and uses development banks to close some well-identified market gaps, we suggest that development banks can be in charge of identifying these market failures through their loan-screening and lending activities to guide their operations and provide critical inputs for the design of productive development policies. In fact, they can also identify government failures that stand in the way of development and call for needed public inputs. This intelligence role of development banks is similar to the role that modern theories of financial intermediation assign to banks as institutions with a comparative advantage in producing and processing information. However, while private banks focus on information on private returns, development banks would potentially produce and organize information about social returns.
Topic:
Development, Industrial Policy, Markets, and Banks
Political Geography:
Global Focus and Global Markets