"Well, one thing that did certainly affect it is the tactics of how to reform, in the sense that, certainly in academia, you are basically told you need to think deeply. Then there are a lot of pressure groups, lobbies, so you need to talk to them. You need to use the media for communicating the benefits of reform, and so on. Some of the reformers, successful reformers that I spoke with, before I joined the Bulgarian government, basically I said, 'You go, and on Day 1, you surprise everybody. So, you go in every direction you can, because they will be confused what's happening and you may actually be successful in some of the reforms. So, this is what I did. I went to Bulgaria in late July 2009; the Eurozone Crisis had already started around us. Greece was just about to collapse a few months later. So, there was kind of a feeling that something is to happen. But, instead of going, 'Let's now do labor reform,' then, 'Let's do business entry reform,' in the government we literally went 6 or 7 different directions hoping that Parliament will be, you know, confused or too happy to be elected--they were just elected. And we actually succeeded in most of these reforms. When I tried to do meaningful, well-explained reforms two years after, they all got bogged down, because lobbying will essentially take over and, 'Not now; let's wait for next year's government,' and so on."From the always interesting Econ Talk.
30 November 2024
How to achieve public policy reform by surprise and confusion
26 September 2024
JOB: Research Assistant on Global Education Policy
- Collaborated with JPAL on a large-scale field experiment on school accountability in Madhya Pradesh, India
- Commissioned a randomized evaluation by IPA of Liberia’s public-private partnership in primary schooling
- Led a five-year randomized trial of a school voucher programme in Delhi
- Helped the Ugandan National Examinations Bureau create new value-added measures of school performance
- Commissioned scoping studies of non-state education provision in Kenya and Uganda
Reporting to the Head of Research and Evaluation, the Research Assistant will contribute to EPG’s work through a mixture of background research, data analysis, writing, and organizational activities. S/he will support and participate in ongoing and future academic research projects and EPG project monitoring and evaluation activities.
The role is based in Ark’s London office with some international travel.
The successful candidate will perform a range of research, data analysis, and coordination duties, including, but not limited to, the following:
- Conduct literature and data searches for ongoing research projects.
- Organize data, provide descriptive statistics, and run other statistical analysis using Stata and preparing publication quality graphics
- Collaborate with EPG’s project team to draft blogs, policy briefs, and notes on research findings.
- Support EPG’s project team in the design and implementation of project monitoring and evaluation plan
- Provide technical support and testing on the development of value-added models of school quality
- Coordination and update of the EPG/GSF research repository
- Organise internal research and policy seminars
- Perform other duties as assigned.
The successful candidate will have the following qualifications and skills:
- Bachelor’s (or Master’s) degree in international development, economics, political science, public policy, or a related field.
- Superb written and verbal communication skills.
- Competence and experience conducting quantitative research. Experience with statistical software desired.
- Familiarity with current issues, actors and debates in global education
- Proven ability to be a team player and to successfully manage multiple and changing priorities in a fast-paced, dynamic environment, all while maintaining a good sense of humor.
- Outstanding organization and time management skills, with an attention to detail.
- Essential software skills: Microsoft Office (specifically Excel) and Stata
- Experience working in developing country contexts or international education policy -- a plus
- Experience designing or supporting the implementation of research evaluations and interpreting data -- a plus
- Fluency or advanced language capabilities in French -- a plus
21 June 2025
Grit: Probably not that important in developing countries
“Angela Duckworth’s new book Grit: The Power of Passion and Perseverance has been launched with great fanfare, reaching number two on the NY Times Nonfiction bestseller list. She recently gave a very polished and smooth book launch talk to a packed audience at the World Bank, and is working with World Bank colleagues on improving grit in classrooms in Macedonia.”That’s David McKenzie in a great book review, considering what development economists can learn from this hot psychology research trend. Grit - the ability to keep going when things get tough and you aren’t successful straight away - can help explain all sorts of individual outcomes beyond tests of skill or ability. David notes amongst other things how U.S. - centric the research on grit is, and questions how large the effect of grit is even in this context.
So what do we know about the importance of grit in developing countries?
Fortunately, a separate team at the World Bank has recently been rolling out a series of surveys measuring psychological traits including grit alongside measures of skills, income, and other demographics. Data is currently available for 10 countries; Armenia, Bolivia, Colombia, Georgia, Ghana, Kenya, Laos, Sri Lanka, Vietnam, and Yunnan Province (China).
Here’s what I found from some very quick analysis.
Grit and income
I started by looking at the relationship between adults’ grit and their earnings. For seven countries there is no relationship. For the three where some relationship exists, grit explains very little of the variation between the income of individuals. (That is, in the table below, the r2 statistic is less than 0.005.) Adding in a few basic control variables (age, parents’ education and socio-economic status) makes even that weak correlation disappear altogether.


Grit and schooling
Moving next to years of schooling, something more substantial emerges. Grit has a statistically significant relationship with schooling in every country, and throwing in a bunch of control variables doesn’t seem to make it go away. I’m not sure what to make of the magnitude though - less than half a year of extra schooling for a 1-standard-deviation increase in grit. Maybe that’s a lot, maybe it isn't. I suppose the policy-relevant question is how responsive grit might be to interventions.


Grit and learning outcomes
Finally, looking at the correlation with answers on a reading comprehension test. Grit scores pretty poorly here too; one positive correlation, one negative, and eight statistically insignificant. Other control variables by comparison do have the kind of statistically significant relationships you might expect - people tend to score better with more schooling and if they grew up in wealthier families.


I’ll spare you some of the snarkier comments from the office, needless to say that unsurprisingly to some, from a quick look the data does not seem to suggest that grit is all that important in explaining important outcomes in developing countries. Unsurprising, because all the grit and resilience and perseverance in the world is unlikely to help a child succeed at school if they haven’t eaten that day and their teacher hasn’t turned up due to a dysfunctional school system. Similarly in the labour market, individual motivation and grit by itself isn’t going to create any well paying jobs in places where the demand for labour is low because of systemic factors such as bad infrastructure and bad governance.
I will offer one caveat - this measure of grit is based on only three questions rather than Duckworth’s preferred ten, so it is possible that a better measure of grit would matter more. But I doubt it.
And finally - this is not meant as a counsel of despair. For individuals living in low-income countries, of course they should try and persevere as hard as they can to try and achieve their goals. But when it comes to making policy - we should focus on the systemic constraints that are critical to shaping people’s opportunities, rather than just telling them to try harder. Bad schools, infrastructure, and governance, are all fixable public policy problems.
10 July 2025
How would you spend a billion dollars on children?
Jere Behrman: Start a program to improve information about education systems - including costs as well as impacts
Karthik Muralidharan:
1: appropriate curricula and pedagogical materials for teachers,
Lant Pritchett: The key is really moving political focus from access to learning outcomes. Do country studies which are a detailed plan of how to reach a set country-specific learning standard. Frame this as a problem that we can solve.
Paul Glewwe:
2. Pay for performance - e.g. cash on delivery for african gvts if they reach learning outcome standards
3: More Research
Pedro Carneiro: Need some solid plans from ECD people about practical details.
Richard Morgan (UNICEF): Coincidentally, exactly what UNICEF is doing now - including focusing on underexplored areas at this conference - child survival, stunting (with WASH as an input), and child protection.
The audience vote at the end was unclear (though apparently Richard got most of the non-economists...), so I'm stealing Duncan Green's polling habit - Make your choice in the side-bar!
---
Update: The final results:
Costas
|
0 (0%)
|
Jere
|
1 (5%)
|
Karthik
|
8 (40%)
|
Lant
|
4 (20%)
|
Orazio
|
1 (5%)
|
Paul
|
3 (15%)
|
Pedro
|
0 (0%)
|
Richard
|
2 (10%)
|
These are all rubbish, something totally different
|
1 (5%)
|
19 February 2025
The Girl Effect?
"Our analysis in this paper has highlighted several possibly important patterns in gender-based inequalities in the four study countries [Ethiopia, India (Andhra Pradesh), Peru, and Vietnam]. The most important of these is that there is no common thread that can be used to characterize gender inequalities across these different countries or indeed even across different dimensions of child well-being in the same country or across different ages.
we find no evidence of a common narrative of gender bias that is valid across all four countries and all dimensions; recognition of this heterogeneity in the patterns of inequality is, in our opinion, of central importance to effective policy-making, i.e., policy-making that is targeted toward reducing the specific biases that do exist in different contexts."From a new paper by Stefan Dercon and Abhijeet Singh
15 February 2025
When unintended consequences go... well!
They find that the "National Rural Employment Guarantee Scheme" (NREGA) in India has a positive impact on private sector wages by bidding up the price of labour. The indirect gains to poor labourers from the higher private sector wages are big - about half of the value of the direct gains to participants from the public works programme. Of course, this increase in wages represents a loss for buyers of labour, but these tend to be people in the top 20%.
A couple of interesting implications that the authors note - first, this is evidence against the Lewis model of surplus labour which can be cheaply tapped for capitalist expansion.
Second - differences in the political power and organisation of landlord farmers may help explain differences in the implementation of the scheme across states.
Finally, a reminder that this is based on nationally representative data in a country of 1.2 billion people, and a programme which spends $9 billion a year and reaches millions of households. Take that, randomistas.
21 January 2025
When rigorous impact evaluation *does* make quite a big difference
The policy - pay a private provider for each unemployed person that they get into a job.
The result (part 1) - the policy was successful at getting unemployed participants into jobs.
The result (part 2) - almost all of these jobs were just taken from other people who would otherwise have got them. Pure displacement. No net change in unemployment.
Most impact evaluations don't measure such "spillover" effects or "externalities", because they are really hard to measure (neither do most non-randomised evaluations.., this is not a criticism of RCTs).
Ignoring externalities, we would have thus concluded, for example, that 100,000 euros invested in the program would lead 9.7 extra people to find a job within eight months. Since the eff ect disappears by 12 months, this already appears to be quite expensive, at about 10,000 euros for a job found on average four months earlier. But at least, it is not counterproductive. With externalities, investing 100,000 euros leads to no improvement at all.Bruno Crepon, Esther Duflo, Marc Gurgand, Roland Rathelot, and Philippe Zamoray (2012), Do labor market policies have displacement effects? Evidence from a clustered randomized experiment
09 January 2025
Does Policy Work?
Next time you hear "does aid work?" think "does policy work?". It's a silly question, and obvious when you put it like that.So I thought it might be worth some elaboration. The idea of course, is from Duflo & Banerjee's "Poor Economics" and Karlan and Appel's "More Than Good Intentions", but I'm not sure they stated it so succinctly or hit upon the direct metaphor of domestic policy. It's also been further embedded in me from directly working on a wide range of aid-financed policy at OPM. And most proximately, inspired by yet another abysmal right-wing aid attack article in the Spectator (which I won't bother link to because it is awful and you shouldn't waste your time and frustration on it like I did).
— Lee Crawfurd (@rovingbandit) January 8, 2025
The question "does policy work" is jarring, because we immediately realise that it makes little sense. Governments have about 20-30 different Ministries, which immediately implies at least 20-30 different areas of policy. Does which one work? We have health and education policy, infrastructure policy (roads, water, energy), trade policy, monetary policy, public financial management, employment policy, disaster response, financial sector policy, climate and environment policy, to name just a few. It makes very little sense to ask if they all collectively "work" or are "effective". Foreign aid is similar. Aid supports all of these different areas of policy. My colleagues and I at OPM work on aid-financed projects that support most of these different policy areas in different developing countries.
There are of course reasons that aid is different to domestic policy, and why we might want to think about it a bit differently. Classic concerns are about dutch disease, and undermining domestic accountability. But here there is actually not any really robust evidence pointing either way. The really robust evidence is all to be found at the project-by-project, policy-by-policy level.
A common concern is about the impact of aid on growth. But do we judge UK health policy by whether it has an impact on growth? Admittedly, we might expect an external infusion of spending to a poor country to have at least some temporary impact on growth regardless of what it is spent on. But is that really the outcome by which it should be judged? Some aid is specifically targeted at growth - such as financing infrastructure or private sector development. But much of it is not. One of the few papers which looks at the macroeconomic impact of aid and actually bothers to disaggregate even a little the different types of aid, finds that the aid that could be considered to have growth as a target, does increase growth. It's the aid that was never intended to impact growth at all, such as humanitarian assistance, which doesn't have any impact on growth.
So. Africa is not a country. It is many. And aid is not a policy. It is many.
04 January 2025
Policy job vacancies, South Sudan and Indonesia
27 September 2024
Livelihoods 2.0
how do these projects differ from traditional income generation? For decades, NGOs have been showing up in communities and persuading people to raise chickens or rabbits, open tailors, or plant the latest new wonder crop. The record is decidedly mixed. What’s different about this latest round?
- Involve local government and private sector from the outset - they are the only long term guarantors of ‘sustainability’.
- Scale - it’s no use just running a pilot and then crossing your fingers. From the outset, you have to think how your intervention needs to be designed to benefit 100,000s of people, rather than 100s
- It’s about value chains, not just production. Often the real barrier is not growing or making stuff, but finding the credit you need to keep you going between planting and harvest, getting the product to market (the roads here are terrible, gulleyed by rain and gouged by illegal logging trucks), or finding a reliable buyer who pays decent prices. Multiple actors need to be involved - it’s no use just funding a local NGO to hand out seedlings. Systems analysis is essential
- Advocacy: a systems approach resembles a micro version of Dani Rodrik’s bottlenecks to growth. Resolving one bottleneck (eg supplies of decent seeds), allows the effort to move on until it hits the next one (roads, access to finance, quantity and quality). Some of these can be incorporated into the programme, but many require local level advocacy (eg lobbying the public works department to do something about the roads, or the state bank to start lending to long gestating crops like rubber).
21 September 2024
How do we scale up personalized support? (policy response to heterogeneity)
The need seems to be clear. Perhaps the key bottom line from the microfinance impact literature is heterogeneity - people are different, their needs are different, and one-size-fits-all policy has all sorts of different impacts, both positive and negative, on different kinds of people.
Similar results seem to be emerging on support for small businesses (a lit review by David McKenzie here and and evaluation from Ghana here), and in education and early child development (see this great episode of This American Life covering Paul Tough's new book How Children Succeed).
Meanwhile in the UK, Iain Duncan Smith has decided that the benefits system is far too complicated (it is complicated) and so it needs to be simplified, rolling 6 different benefits into one "Universal Credit." But maybe, just maybe, complicated people need complicated support? And is that a realistic goal for developing countries with weak government systems?
03 September 2024
Aid for Infrastructure in Fragile States
Development sometimes feel like a bit of a catch- 22. Economic growth requires decent "institutions" and political stability, but in many ways good institutions require a decent level of economic development to begin with. Nowhere is this more relevant in aid policy than large scale infrastructure development. Infrastructure projects have the potential to spur development through crucial inputs such as employment, connectivity and capacity building. But mismanaged they can do more harm than good, by fuelling corruption and environmental degradation.
14 July 2025
Secretaries without borders
Investment is not just needed in getting the policy ‘right’ but also in making the ‘paper move’. Money needs to be spent on building the capacity of governments to maintain basic administrative and management processes, on training administrators and establishing functional IT systems. No matter how good policies are, without the basics in place - and the people to administer them - implementation will continue to be hindered by chaotic and ineffectual management.Well worth reading in full. They have also just published a couple of briefing notes on the new government Aid Strategy in South Sudan and the 2009 Donor-Government compact.
15 June 2025
A quantitative history of RCTs
Randomised controlled trials (RCTs) are the best way of determining whether a policy is working. They are now used extensively in international development, medicine, and business to identify which policy, drug or sales method is most effective. They are also at the heart of the Behavioural Insights Teamʼs methodology. However, RCTs are not routinely used to test the effectiveness of public policy interventions in the UK. We think that they should be.(HT: Tim Harford's twitter feed)
14 May 2025
Millennium Villages: impact evaluation is almost besides the point
"What is the impact of the Millennium Village package of interventions on the area in question?"
The really depressing part though is that this is actually the least interesting question. Chances are that throwing in a whole bunch of extra inputs to a community will create some outputs, and some impact. The whole point of the Millennium Villages though is to provide a model for the rest of rural Africa to follow. The really interesting question is whether African governments have the desire and capability to manage a massive and complex scaling up of integrated service delivery across rural Africa.
A point which basically belongs to Bill Easterly.
Mr. Easterly argues that the Millennium approach would not work on a bigger scale because if expanded, “it immediately runs into the problems we’ve all been talking about: corruption, bad leadership, ethnic politics.”
He said, “Sachs is essentially trying to create an island of success in a sea of failure, and maybe he’s done that, but it doesn’t address the sea of failure.”
Mr. Easterly and others have criticized Mr. Sachs as not paying enough attention to bigger-picture issues like governance and corruption, which have stymied some of the best-intentioned and best-financed aid projects.
Another challenge in some sites is insufficient capacity of local government to take full ownership of MV activities. This is manifested in unfulfilled pledges to perform mandated roles, unsatisfactory maintenance of infrastructure, and insufficient involvement of local elected officials. MV site teams are addressing these challenges by agreeing to jointly implement interventions targeted at improving the performance of sub‐district governments, increasing sensitization and engagement of local government officials, increasing joint monitoring of MV activities in communities, and developing training plans in technical, managerial, and planning skills for local government officials.Or : "we have no clue how to fix the systemic implementation challenges"
An anonymous aidworker writes on his blog Bottom-up thinking
I’ve noticed around here, normally sloth-like civil servants who won’t even sit in a meeting without a generous per diem rush around like lauded socialist workers striving manly (or womanly) in the name of their country when a bigwig is due to visit, working into the night and through weekends, all without any per diems...
I fear all the achievements of the MVP will wash up against the great brick wall that is a change resistant bureaucracy.None of this is to say that the situation is hopeless. It isn't. In particular there are elements of the Millennium Village package which are proven to be effective, cheap, and don't require complicated systems of governance and accountability. Namely distributing insecticide-treated bednets. Aid money can provide them easily, sustainability is less of a concern than other interventions, and you can buy them right now. Check out Givewell for a rigorous independent assessment (and recommendation) of the Against Malaria Foundation. Probably the single best way you could spend some money today.
22 February 2025
When the counterfactual *really* matters
The Minister in charge of the scheme has proudly trumpeted:
The fact is that 13 weeks after starting their placements, around 50 per cent of those taking part have either taken up permanent posts or have stopped claiming benefits.
Off-flows from JSA remain high - almost 60% of claimants leave within three months
Now this is not definitive - without a proper control group and a counterfactual, we do not know what would have happened to the participants without the programme. Maybe I am wrong, and in fact those who go on the programme have very poor characteristics, and would have done even worse without it. Without proper evaluation, we just don't know. But certainly the evidence and analysis so far published by DWP does not make a good case.Bottom line #1: if you're interested in smart and well-presented UK economic policy analysis you should really be reading Jonathan Portes.
Bottom line #2: There is probably a very good business case to be made for sending Mr. Iain Duncan Smith on the J-PAL Exec Ed course on evaluating social programs. Training budgets must be tight with all the cuts going on and that though, so - genuine offer - if you're interested Mr. Smith I'll pay your tuition fees out of my own pocket :)
13 February 2025
Urbanization in Africa
The popular mental model is that there is massive rural-urban migration in search of jobs.
"Since the mid-1970s, African governments have increasingly adopted policies designed to inhibit or discourage people from moving into urban areas. Today, approximately 80% of African countries have policies in place to prevent rural-urban migration. At the same time, international development organisations increasingly withdrew support for urban development initiatives in favour of rural development projects, often justified by the argument (among others) that improving standards of living in rural areas will help to mitigate the growth of urban poverty."The trouble is, most urban population growth is actually just natural growth. And the rural-urban migration that does exist is driven by a range of factors, not just jobs, such as a "desire to escape age or gender discrimination in their communities, to find a wife or husband, to seek adventure in the “bright lights” of the big city, or to escape rural serfdom."
And the last word from Sean;
"Simply put, the rapid growth of Africa’s urban population is being driven primarily by rapid population growth in urban areas, not rural-urban migration ... For those interested in easing demographic pressure in urban areas, the only humane policy option is to try to reduce population growth by promoting fertility decline through voluntary family planning initiatives. And for those interested in promoting economic development in the region, investment in urban areas should be top of the policy agenda."Read the whole thing here.
23 May 2025
The Wrong Expert
Based on participant observation in a team of British policy-making civil servants carried out in 2009, this article examines the use that is made of evidence in making policy. It shows that these civil servants displayed a high level of commitment to the use of evidence. However, their use of evidence was hampered by the huge volume of various kinds of evidence and by the unsuitability of much academic research in answering policy questions. Faced with this deluge of inconclusive information, they used evidence to create persuasive policy stories. These stories were useful both in making acceptable policies and in advancing careers. They often involved the excision of methodological uncertainty and the use of ‘killer charts’ to boost the persuasiveness of the narrative. In telling these stories, social inequality was ‘silently silenced’ in favour of promoting policies which were ‘totemically’ tough. The article concludes that this selective, narrative use of evidence is ideological in that it supports systematically asymmetrical relations of power.Well here’s another ethnographic study, from “The Thick of It” (a fantastic show, buy the DVD here) (NSFW).
20 March 2025
Economic Policy in Southern Sudan
The SPLM have some pretty respectable economic advisers in Shanta Devarajan, Lant Pritchett, and Tony Venables.
04 November 2024
2010 Commitment to Development Index
The 2010 Commitment to Development Index is out, put together by David Roodman at CGD. Lots of fancy graphics and charts to play with.
The key messages:
- Rich-country policies matter.
- Development is about more than aid.
- Aid is about more than money.
- Coherence matters.
- Partnerships are powerful.
- No one is perfect.
The UK scores pretty poorly, coming in at 16th out of 22 countries. We do particularly badly on migration policy (no surprises), security policy - my guess due to all those weapons sold to undemocratic regimes - and on technology policy - due to either low R&D spending or overly-restrictive intellectual property rights.
Sigh.