Street kids in Sierra Leone

A fascinating new report from Sierra Leone of a census of street children (via the Guardian): . Some headline results:

• In total there are just under 50,000 kids living and working on the streets in Sierra Leone.
• About half live in Freetown and half in other towns.
• There is a roughly even gender split with slightly more boys than girls.
• These kids make up around 5% of the urban population.
• 4,388 children under 6 years old were counted.
• 2,699 children were found sleeping rough.
• 1,821 under-age girls were recorded as working in the commercial sex industry.
• 633 disabled children were counted.

I would have loved to have seen a slightly more detailed age breakdown, and more detail on the types of economic activities these kids are doing. I’m a lot more worried about 8 year olds washing cars and polishing shoes than 17 year olds selling mobile phone credit. 

Scaling up Proven Interventions

If you are in the business of piloting development policies with NGOs, this chart should be keeping you awake at night. If you like to think about "sustainability" and "scale", about handing over your activities to the government, you need to be really really worried. 

Researchers persuaded World Vision and local government in Kenya to both implement that exact same intervention at the same time. The program as implemented by World Vision found a large impact on test scores. The exact same program, as implemented by the local government, found zero impact. 

For more see Gabriel Demombynes.

His conclusion:

Evaluation skeptics may try to cite this as evidence that RCTs are a waste of time, since it suggests that successful interventions implemented by NGOs, as they often are in experiments, may not be replicated at scale by governments. Others might take the paper to indicate that NGOs should be the preferred vehicle for interventions. I think these readings would be mistaken, and I take two reflections from the paper. First, we should do many more rigorous studies working with governments where we vary forms of service delivery to better understand what can work in practice. Second, the World Bank’s approach to public services—the long, difficult slog of working to improve government systems—is the right one, because it’s the only way to ultimately make services work for the poor at large scale.

 I agree. Clearly working through government systems is essential. Innovations for Poverty Action are doing just this - scaling up the exact same contract teacher program tested with NGOs in Kenya and India, but doing it with the government in Ghana, and doing an RCT as they go.

Addendum: Here is the link to the full paper: http://www.cgdev.org/doc/kenya_rct_webdraft.pdf

Herbert Gintis reviews Taleb's "The Black Swan"

I was shocked that OJ Simpson was acquitted of a murder he obvious committed. I am shocked that most Americans do not believe in evolution. And I am dumbfounded at the widespread success of this really stupid piece of pseudo-literature ... 

It would be lovely to have a sensitive exposition of great unexpected events, such as the fall of the Soviet Union, Black Monday, and the shooting of Robert Kennedy. This woeful volume is not it. I can't image what anyone finds valuable in The Black Swan. This too is a completely unforseen event, but is also difficult to explain in hindsight.

Ouch.

The Impact of IPA

Measuring policy influence is hard, but this looks like a slam dunk for IPA: Concern Worldwide are launching a new Gates Foundation funded initiative focused on Innovation, Evidence, Action, and What Works.

Compare and contrast the Concern website;

and the IPA website;

Congrats all round. 

Why Rwanda is like Apple?

Andrew Mwenda, who has been fierce advocate for democracy in Uganda, seems to be quite a big fan of Kagame;

Rwanda/Kagame has been branded by its achievements as a successful case of post conflict reconstruction. The more the positive Rwanda brand has grown, the more it has attracted opportunistic groups that want to ride on it to enhance their own brand. By attacking an attractive brand, you are able to generate attention to your own brand. Human rights groups therefore have little incentive to focus on some obscure - even though murderous regime like Equatorial Guinea - because it will not make them visible in the human rights advocacy market. 

Assume you have a consumer protection advocacy organization and you want to build your global brand. It does not give you sufficient visibility if you focus your campaign on some obscure company called Filiopa Cranta (what a difficult name!) that manufactures drugs and sells them in the rural areas of Papua New Guinea. However, if you can pitch your case against GloxoSmithKline, Novartis or better still Microsoft or Apple, you are likely to attract a lot of attention even if your case is weak. 

Rwanda’s greatest asset in this war is actually the people of Rwanda whom these groups claim to speak for. In all opinion surveys by the most respectable polling organizations like Gallup Poll and World Values Survey, Rwandans say they feel free to speak, associate and express themselves by a margin of 85% - as good as one finds in democracies like Norway and Sweden. It will be humbling to see the advocates for freedom in Rwanda being told by ordinary Rwandans that they are actually free.

What has UNMISS achieved?

That's a genuine question. The UN Mission in South Sudan cost £500 million in its first year, about half of the entire aid budget (link from Nick Travis). The activities that they are engaged in supporting: disarmament, governance, rule of law, and human rights, are clearly important. They are also some of the most difficult in which to achieve results.

My view is that these activities are a high-value high-risk bet. Whereas investing in health and education is a very safe bet - we can achieve measurable but modest results with reasonable certainty, investing in peace and governance is very risky but with potentially enormous returns. What matters is the expected value of the pay-off and our attitude to risk. So I'd like to see some numbers, even if they are totally made up.

~*Totally made-up numbers alert*~

So lets say that a £500 million aid investment in education and health has a guaranteed pay-off of £750 million.

We get a 50% return on investment.

And a £500 million aid investment in peace and governance has a risky pay off - either
     - a - 10% chance we successfully avert a war - massive massive return - maybe £25 billion* ?
     - b - 90% chance our investment has zero impact, and zero return.

The expected value is a 500% return on investment.

This is where our preferences for risk come in. Do you take the guaranteed 50% return, or do you gamble it all for a 1 in 10 chance of winning a 5000% return?** Or are my numbers totally stupid?

---

* Paul Collier does some guesswork on the costs of civil war in low-income countries - for a country with an average GDP of $19 billion, he estimates an average total cost of around $64 billion (in lost GDP for the country and its neighbours, wasted military expenditure, and loss to life and health). So for South Sudan's GDP of $12 billion, the cost might be $40 billion, or £25 billion.

** A poker player with enough chips to play with would obviously go for the highest expected value return no matter how risky. The decision gets trickier when you are running short of chips, and it becomes more costly to take low probability bets. So our rational attitude to risk depends on how many chips we have to play with. You might argue that the aid budget is small compared to expenditure needs for basic service provision, so we are really short stacked...

Addendum: Frontier Economics estimate the cost of war for all of Sudan (North and South?) at $75 billion.

Addendum 2: These numbers will change quite a bit when you consider that the £500 million peacekeeping investment is per year and the war costs are calculated over the lifetime of a war and its aftermath.

Addendum 3: All of this is besides the point I started with in my head, which is that UNMISS seems to have nothing on its website about any actual results. I'd like to see some explicit discussion of what they think that their outcomes are, or if there aren't any, if this is because there is an implicit calculation similar to the one above that they are doing, any a total lack of results is actually completely consistent with spending large sums of money on bets with very very large potential payoffs. 

Cash Rules Everything Around Me (South Sudanese Pound Music Video Edition)

HT: Bored in Post Conflict

Bhutan: Don't believe the hype

On Monday the Guardian kindly explained "Why we'd all be happier in Bhutan." I'm giving Sarah Boseley the benefit of the doubt and assuming that she didn't actually write the headline, given that the report her article is about puts mean happiness in Bhutan at 6.05 (on a scale of 1 to 10), compared to a mean of 7.2 from similarly worded survey questions in Britain. Just because Bhutan measures happiness and its Prime Minister talks a lot about Gross National Happiness, doesn't mean its people are actually happy.

The GDP-lead development model that compels boundless growth on a planet with limited resources no longer makes economic sense. It is the cause of our irresponsible, immoral and self-destructive actions," [Prime Minister] Thinley said

Which is fine and everything yeah, but sometimes nice things like health services cost money. Which is why Bhutan, a lower middle income country, has 12% child malnutrition rates. Unlike Britain.

And a life expectancy of 65. Britain is 80 just hit 100.

So it shouldn't really be too much of a surprise to see this kind of relationship 

Finally, a commenter on the Guardian article notes;

Why do so many articles on this subject fail to mention the government-supported suppression of religious and political freedoms in Bhutan, not to mention the hundred thousand UN-registered refugees now living in camps in Nepal and abroad? Is this the example we want to follow?

None of which is to say that measuring happiness or thinking deeply about the goals of policy are bad ideas, but neither should we go ga-ga for weak democracies. 

NGOs, RCTs, and Institutions

There is a malicious and perverse relationship between the force of NGOs and the weakness of the Haitian state.

That is Ricardo Seitenfus, the Brazilian head of the Organization of American States mission to Haiti during the 2010 earthquake - quoted by Acemoglu and Robinson.

This is the real challenge to aid project impact evaluations, whether randomised or not, and is really the key argument made by the critics from Bauer to Easterly to Moyo. 

What isn't clear to me is how we should expect the impact of NGO activity on state strength to vary by the effectiveness of the project. Does the measured impact of the project on individuals have any relationship with the impact of the project on broader governance? It's at least plausible that a very effective health intervention (measured by outcomes for individuals) could also be effective in building state capacity for service delivery. Or on the contrary, the same measured improvement in outcomes for individuals could be consistent with undermining state capacity, by encouraging the state to spend less on health and more on other goodies, and also by poaching the best staff from the government into the NGO sector. Tricky.

Answers / more data sources on a postcard. 

Cash Transfers in Africa

The World Bank has a new book out reviewing the state of cash transfer programs in Africa, which gives a nice high-level overview, and lots of juicy detail on targeting, coverage, and transfer mechanisms, 

including a timeline ... 

... the cost of programs (they get easier to fund as countries become wealthier) ...

... admin costs are a relatively fixed cost - they become a shrinking share of the program budget as coverage increases ...

... and as a sensible response to the limited capacity of the poorest countries to fund their own programs, donors play a key role in funding systems in the poorest countries ...



Yes all I did was skim the report for the prettiest charts. What!

Evidence on moral hazard / fungibility of aid

I'm not sure if moral hazard is really the right term, but in any case, when we give aid we worry about what effect the transfer will have on the behaviour of the recipient - will they just become lazy and wait for more transfers - or make a health  and education transfer "fungible" by reducing their own initial health and education spend in order to buy more booze. Turns out there is some evidence on household response to schools providing more supplies. 

Das, Dercon, Habyarimana, Krishnan, Muralidharan, & Sundararaman find that an unexpected injection of spending at schools in Zambia and India does increase test scores that year. But the following year parents anticipate the additional spending by the school, and decide to reduce their own education spending so that they have more money for other things. And test scores go right back to where they were before.

The researchers offer a couple of potential solutions to this problem 

1 - just increase the size of the grant such that it is larger than household's own education spending - if you increase per student spend by $20 and households only spent $10 themselves to begin with, they can cut all of their spending and you still get a net increase. 

2 - focus additional spending on things that households don't directly fund themselves anyway - "public" goods like teachers rather than "private" goods like books and stationary - so they can't easily substitute away their own spending. Which in a weird way reminds me of Dan Ariely's holiday gift giving advice 

A paternalistic gift ignores the preferences of the person getting the gift, which tends to drive economists crazy, but it may actually change those preferences for the better.

Yawn.... more RCT debates

Two very smart folks, Mark Rozenzweig and Martin Ravallion have reviews of Poor Economics in the latest Journal of Economic Literature (thanks to Abhi and Andrea for the papers). Obviously self-recommending when smart economists review smart economists. But there does seem to be a bit of a rehashing.

Martin's biggest score is the "where the hell is China?" line. Some of the other criticisms are a bit weaker.

Another likely bias in the learning process is that J-PAL’s researchers have evidently worked far more with nongovernmental organizations (NGOs) than governments.

Which is a bit of a cheap shot, and a bit innacurrate. Researchers have worked with whoever will let them experiment, which yes initially was NGOs but is increasingly governments - see Peru's Quipu commission, Chile's Compass commission, the teaching assistant initiative in Ghana, working with the planning Ministry in South Africa, experimenting with police service reform in Rajasthan, even Britain's Behavioural Insights Unit.

Then

how confident can we really be that poor people all over the world will radically change their health-seeking behaviors with a modest subsidy, based on an experiment in one town in Rajasthan, which establishes that lower prices for vaccination result in higher demand?

Ummmm... well thats why J-PALs policy recommendation for health pricing is based on 6 different studies....

Mark scores his biggest hit in the final footnote on the last page of his article;

Also absent is a discussion of the standard but major problem in the implementation of any programs or transfers targeted to the poor and that do not really spur development—moral hazard.

"Moral hazard" works at both the individual and national government level. If you get aid, you are probably less likely to work hard. The critical question is the magnitude of this effect. I think that on balance the positive value of effective aid outweighs the moral hazard, but that is more of a feeling than an evidence-based proposition. This is also one of the key points made by aid critics Bauer/Easterly/Moyo. Not necessarily that aid doesn't work, as Banerjee/Duflo would like to present their argument, but that even if aid does work, the negative moral hazard effect might outweigh the positive. I haven't seen this argument really addressed at all.

The other serious and neglected criticism for me is on general equilibrium, raised by Daron Acemoglu in the Journal of Economic Perspectives. What if you measure a positive impact of a program on earnings, but those are coming at the expense of others? A training program that increases earnings might just be equipping some individuals to out-compete others in the market, rather than necessarily increasing aggregate productivity, in which case scaling the program ain't gonna work.

So maybe I've missed them - but has anyone seen a convincing rebuttal to the moral hazard and general equilibrium critiques of micro aid project impact evaluation?

-----

Update: A couple of things I missed in my haste - Abhi points out that Rosenzweig makes good points on the sometimes tiny effect sizes lauded in Poor Economics (e.g. where "15% increase" translates to something like 2 weeks schooling or 50 cents), and that RCTs can focus our attention away from the big (important?) questions, but I felt this criticism is pretty well rehearsed.

Update 2: Also Ravallion loses points for his cliched title: "Fighting Poverty One Experiment at a Time". "x one y at a time" is a boring, tired, tired, catchphrase.

Update 3: Ravallion gains points for coining "regressionistas." 

Maasai Land

Last week I was mostly in Kajiado County. Maasai-land. The County is apparently the third richest in Kenya (by the KIHBS, but this number is disputed), though this wealth is driven by "outsiders" building new developments along the recently completed tarmac road to Nairobi. It can now take as little as an hour from Kajiado Central. The Maasai struggle to compete in the new economy, with limited education and skills.

I spoke to an educated professional (middle class?) Maasai who expressed his dismay at the erosion of the traditional Maasai way of life, as the government's land privatisation policy had led to the parcelling off of land, obstructing traditional migratory herding practices based on traditional beliefs that land is communally owned.

I sat there wondering how I could register some disagreement without sounding like a dick, and how this place epitomised the tension between modernisation and traditional ways of living, transported back to a development studies classroom of tedious debates on "what really is development?" I couldn't help but think that perhaps it was a little hypocritical for this well-educated man to sit in comfortable surroundings and talk about how tough it is for his people to have to give up their cattle herding.

Ultimately, for the Kenyan government to be able to afford to provide better health and education services for its people, it is going to need the revenues that come from the kind of economic development that comes with urbanisation and modernisation.

But this process is disruptive, and the best solution for the "losers" is not to put the brakes on development, but to provide compensation, safety nets, services, and the skills that they will need to engage in the growing economy.

Assistant Consultant Job at Oxford Policy Management

There a few days left to apply for the Assistant Consultant vacancy at Oxford Policy Management in the "cross-cutting" team that I am part of. I can honestly say that it's a really fantastic place to work with lots of smart interesting people (including tons of ODI Fellows) doing all sorts of smart interesting work. From the job ad:

Oxford Policy Management (OPM) is a leading development consultancy with offices in Oxford, Islamabad, Delhi, Pretoria, Dhaka and Jakarta. We provide rigorous analysis, policy advice, management and training services to governments, international aid agencies and other public sector and non-government organisations. OPM aims to contribute in innovative ways to enhance economic and social progress in developing and transition economies, with a focus on the needs of the poorest people. We have worked in over 90 low and middle income countries over the last 30 years. 

OPM is seeking to recruit an Assistant Consultant to work in its cross-cutting portfolio. The cross-cutting team is OPM's entry point for talented and passionate individuals with limited experience or no particular specialisation to work in development consultancy, and progress by either specialising in one of the other technical areas or remaining generalist. Skills are learnt through project work with senior consultants (including overseas fieldwork and ministry work as possible), a year-round training programme, and mentorship from an experienced senior consultant. We expect high performance, and reward it with promotions, salary increments and responsibility. The cross-cutting portfolio currently contains four assistant consultants and four consultants who work on and sometimes lead consulting and research projects across the rest of OPM’s specialist portfolios.

This role is full-time based in our Oxford office and the anticipated salary range is between £20,000 and £23,000.

Closing date is 01 April 2025

For more see here. There are also a few other positions open, including an Assistant Consultant based in Jakarta and a Senior Health Economist.