Evidence-based policy-making US-style

Based on our rough calculations, less than $1 out of every $100 of government spending is backed by even the most basic evidence that the money is being spent wisely.

...

Since 1990, the federal government has put 11 large social programs, collectively costing taxpayers more than $10 billion a year, through randomized controlled trials, the gold standard of evaluation. Ten out of the 11—including Upward Bound and Job Corps—showed “weak or no positive effects”

Just in case you thought that there was any danger of the whole results agenda and RCT-fetishism taking over in American politics. From an excellent piece in last month's The Atlantic, which by the way is generally fantastic, I just bought a paper copy for the first time and the whole thing was full of interesting - a critical look at the evidence on over 35 female fertility, a thing about how much health food is actually really unhealthy, a note about how recycling can actually increase carbon emissions because it needs more trucks on the streets, and a piece discussing relationships and gender politics and family from the perspective of a man who has sacrificed his career for his wife's. 

The one thing that does get delivered in India's government schools

Excellent writing by Abhijeet in the Guardian providing some perspective on the recent deaths in Bihar from school meals.

the midday meals, which reach about 120 million children on every school day, are probably the most successful of all interventions in education that the Indian state has delivered in the past decade. On any school day, a quarter of teachers are absent from government schools (pdf), only 45% of those in school are teaching, but in 87% of schools, a hot meal is served (pdf).

The revolution might not be televised, but what about development?

One of the latest big things in education research is the importance of what economists have labelled "non-cognitive skills," psychologists call "psychosocial competencies", and humans have been known to call "character." Various studies have found that "cognitive skills" - literacy, numeracy, etc, actually don't do a fantastic job of explaining success later in life, and that it is a bunch of other soft skills which really do matter - things like perseverance, curiosity, conscientiousness, optimism, and self-control (perhaps someone should tell Michael Gove?).

All of this was the subject of Stefan Dercon's plenary at Young Lives last week (slides here). The good news is that these characteristics appear to be malleable even into adulthood.

Apparently there is one question to measure "locus of control" which correlates highly with aspirations and investing in the future. The question is simple - which of the following two statements do you most agree with:

1. “Each person is primarily responsible for his/her success or failure in life.”
2. “One’s success or failure in life is a matter of his/her destiny/fate.”

If how you think about this question determines how much you save and invest for the future and your attitudes towards this can be positively shifted, then maybe there is a "fatalism trap" which intervention can break. Stefan presented results from an experiment run in Ethiopia to see if optimism could be cultivated by showing poor remote villagers mini-documentaries of inspirational success stories on TV. The shows told the stories of local people who had worked their way out of poverty. The experiment also included both a regular control group and a "placebo" regular TV show, without the particular success stories.

Six months later - the villagers were still talking excitedly talking about the TV show, and more importantly actually spending more on investing in education for their kids.

Of course, the tragic danger is whether these newly raised aspirations will turn out to be a false hope if the economic opportunities to make these investments pay off aren't available.

Yet another reason why TV will save the world?

How would you spend a billion dollars on children?

The final session at Young Lives yesterday was really fun, and put this question to all of the plenary speakers - given everything you know from all your research, what would you do? A video of the session is up online here along with the other sessions here, and with apologies to the speakers for butchering their arguments into 140 character chunks, I thought I would reproduce my tweeted summaries here: 

Costas Meghir: An integrated preschool programme - from prenatal, stimulation, ECD, preschool

Jere Behrman: Start a program to improve information about education systems - including costs as well as impacts

Karthik Muralidharan:

First get rid of the things that don't work - starting with teacher education, then
1: appropriate curricula and pedagogical materials for teachers,

2: contract teachers - The challenge is how to manage political and legal barriers - the solution is teaching assistants. Build a 4-year teacher training programme into teaching assistant contracts. At end of 4 yrs, some assistants pass exam to become full teachers, ones that fail get an exit payment 3 months salary. 

And a general point - for scale-up - need to work through government systems but need an intermediate step - start small with a district or a state. 

Lant Pritchett: The key is really moving political focus from access to learning outcomes. Do country studies which are a detailed plan of how to reach a set country-specific learning standard. Frame this as a problem that we can solve.

Orazio Attanasio: No easy answers. We don't yet really understand what works, why, and how to do things at scale. (implying more research?)

Paul Glewwe: 

1. Do TIMMS and PISA for every developing country
2. Pay for performance - e.g. cash on delivery for african gvts if they reach learning outcome standards
3: More Research

Pedro Carneiro: Need some solid plans from ECD people about practical details.

Richard Morgan (UNICEF): Coincidentally, exactly what UNICEF is doing now - including focusing on underexplored areas at this conference - child survival,  stunting (with WASH as an input), and child protection.

The audience vote at the end was unclear (though apparently Richard got most of the non-economists...), so I'm stealing Duncan Green's polling habit - Make your choice in the side-bar! 

---

Update: The final results:

Costas	0 (0%)
Jere	1 (5%)
Karthik	8 (40%)
Lant	4 (20%)
Orazio	1 (5%)
Paul	3 (15%)
Pedro	0 (0%)
Richard	2 (10%)
These are all rubbish, something totally different	1 (5%)

Measuring learning

Here’s the key slide from Karthik Muralidharan’s presentation at Young Lives that I mentioned yesterday (full ppt here).

I was actually looking for a chart like this a couple of months ago for a policy brief we were working on about education and labour markets. I’m not sure my colleagues really believed me that this kind of chart just did not exist, and weren’t really happy with the alternative proxies, so I was quite relieved to hear that this really is amongst the first of its’ kind. But also kind of depressed that work this important hasn’t happened yet on a wider scale.

In India, the average child just about reaches grade 1 standard in maths by the time they reach grade 4. For the bottom 10%, they are learning absolutely nothing from grade 2 onwards. 

Hey! Teacher! Leave them kids alone!

Matt has a post up on the clear highlight of the first day of Young Lives 2013 - the final plenary by Lant "Dude is so famous he doesn’t even bother wearing a name tag" Pritchett.

Pritchett’s point was fairly simple: in many settings school can be a pretty awful place to be, especially if the curriculum is moving faster than you can keep up with it. Eventually, all but a select few are left behind, leading to a “flattening out” of the learning curve. At this point, you can’t really learn anything when you are this far behind, so why stick around? At one point - and without warning - Pritchett presented an entire slide in Spanish, to give the audience a sense of how this must feel.

The bottom line is really quite depressing - there are thousands and thousands of kids out there sitting in classrooms learning absolutely nothing.

The other highlight for me was Karthik Muralidharan's plenary - apparently one of the first papers to measure and illustrate the learning progress (or lack of) of individual children as they progress through school years - on a comparable ordinal scale. The approach is smart, borrowing from the "Item Response Theory" used in GRE tests, and allows you to estimate for example whether grade 5 students can answer grade 1 questions without having to ask them. The key policy take-away was that clearly we need more of this kind of testing being done with the same kids on an annual basis. At present, we have a few snapshot surveys of learning outcomes in random years in random countries, and almost nothing in most countries that can reliably tell you something meaningful about the progress that children are making. Part of this will hopefully be solved in a few years as countries sign up to a new post-2015 development goal on learning outcomes and then realise that they have committed to figuring out a way of actually measuring them. This stuff is important. As Karthik noted, all of the RCT randomista experimental literature looks at how much an intervention impoves treatment schools compared with control schools, but misses the larger point that nobody has a clue how much progress the control schools are making over time if any (as you might expect given general economic growth).

Naureen Karachiwalla and Abhijeet Singh both presented really interesting papers, documenting in detail the role of caste in determining learning outcomes in Pakistan, and differences between public and private schools in India, respectively (bottom line: they perform similarly, but private school teachers cost around a fifth of public school teachers so private schools are a lot cheaper to run).

The nonparametric bayesian econometrics (I think that's what it is...) was maybe a bit much for me first thing this morning, but the point to note for survey designers emphasised by Costas Meghir was that the cutting edge Heckman "latent factor" model tools for estimating human capital, cognitive and noncognitive skills, or whatever you want to call it, are data hungry. You need a few (at least 3) different measures of each concept that you are trying to proxy for.

That's all for now, time to sleep. 

Young Lives 2013 (pre-match analysis)

Good morning from Oxford. As you'll already know from reading AidThoughts, the inaugural Young Lives Conference is happening today and tomorrow, and Matt and I will be furiously blogging (like this guy) and tweeting all the good bits at #younglives. You can see the programme and papers here.

First a bit of background: Young Lives is a project at Oxford University which is following thousands of children from Ethiopia, India, Peru, and Vietnam, talking to them and their households every 3 years for 15 years as they grow up, with quantitative and qualitative components. RCTs aside, this kind of longitudinal survey is pretty rare in developing countries to begin with, but in addition to all the standard demographic and socio-economic information, this survey also measures malnutrition, learning outcomes (language and maths), and asks the children directly about their experiences, hopes, and aspirations. (I just tried to see if I could download the actual questionnaire to see what the wording of the questions was, but apparently you need to be part of an academic institution to use the UK data archive where they are kept, which is a bit annoying).

The conference seems to be all economists, which I'm obviously not really going to complain about but it might have been interesting to see some of the fruits of the qualitative work. There are some really big names in the economics of education; Paul Glewwe, Lant Pritchett, Karthik Muralidharan, Jere Behrman, Stefan Dercon, and also some guy called Abhijeet Singh.

Now a bit of a quibble: the conference is organised around the theme of inequalities in children's outcomes. I remain unconvinced about the whole inequality narrative which is being pushed at the moment. You don't have to agree with Mankiw (I don't) to recognise that talking about inequality turns off a lot of people on the right. I still think the last UK Labour government made a great tactical move by tackling inequality and doing redistribution by defining child poverty by a measure of inequality. You can argue with caring about inequality, it's much harder to argue with caring about child poverty. But the focus here is not income inequality but inequality in outcomes. But in a sense isn't it still true that "we are all poor here?" Should we focus on the differences in health outcomes between the children who are only a bit malnourished and the ones who are totally stunted for life, or just despairing about how badly off they all are?

Finally, on my last count there were close to 60 papers being presented here, so we're going to struggle to even scrape the surface here, so do take a look yourself. I think the plan is also to stream the plenaries online. And just as a warning, if Matt is off his best blogging game today, you should cut him some slack, it might have something to do with all the rigorous qualitative research he was doing yesterday into the positive impact of post-war caribbean migration on contemporary British culture (the Cowley Road carnival in the baking heat yesterday might have been the best thing that has happened in Oxford in 1000 years, well worth missing Wimbledon for. So no pressure Young Lives but this conference better be good).

After Google Reader

I've left it to the very last minute to switch. For what it's worth - I'm now using feedbin, mostly because it lets me keep using the excellent Reeder app on my iphone. And until the new Reeder app comes out for ipad (with feedbin support), I'll be using Mr. Reader on the ipad (which is pretty decent but not quite as good as reeder). Something about feedly didn't quite do it for me.

Social media and the End of NGOs?

Then of course there is the issue of whether in the future we will even need international NGOs any more. Will people just be able to use social media to directly sponsor a child in the community? 

In development there is a tendency to think about trends and gradual change but we need to be aware of the possibility of disruptive change. Development may face a complete sea change at some point - similar to what happened to the music industry where suddenly downloads decimated the CD market. However, preparing for that is more difficult: usually the change that comes is not what you put on your risk register.

Caroline Harper, chief executive of Sightsavers interviewed in the Guardian

"Nature is not on our side"

I enjoyed this riposte from Rob Rhinehart to critics of his new chemical liquid food which fall prey to the "appeal to nature" fallacy;

Nature is not on our side. Most of it is trying to kill us. Nature abounds with neurotoxins, carcinogens, starvation, violence, and death. It is technology that makes our lives so comfortable. We have a responsibility to protect the environment, but it feels no such responsibility for us. Technological innovations should be thoroughly tested and verified to be safe, and they are. Besides being an arbitrary distinction, being "natural" is absolutely no guarantee of safety, usefulness, or practicality. Today it is often the opposite. I think it's a little weird to eat food that comes from a tree. Do we still use leaves for clothing? Like diet, balance is key. I am glad to drink fluoridated water for the same reason I prefer the natural sky. It's healthier.

I'm looking forward to trying some.

"God must be a white man"

An excellent, heart-felt lament in the Sudan Tribune from a man with what must be an incredibly difficult job, the Auditor General of South Sudan, Steven Wöndu.

Donors beware! The African big man eats without limit. He accumulates without consideration for the needs of others. ‘Enough’ does not exist in the dictionary. Africans in power do not want to consider the fundamental zero-sum concept that the more you scoop from the common pot, the less everyone else gets. They refuse to ponder the life or death consequences of misappropriating resources intended for interventions in maternal care, solid waste management, infant vaccination, and clean drinking water. They do not feel the danger of living in a massive mansion surrounded by tragic slums. In the rare event that an African big man visits a village, he would ‘donate’ a class room, or a clinic or a road that would never exist except in the government’s financial expenditure schedule of that year. The announcement of the fake donation is proof that the big men know the needs of their people. Why they choose not to do the right thing beats imagination. But then, they are modern African big men! 

We are very efficient in taxing our poorest people mercilessly. In Africa, a woman with three chicken eggs to sell because the baby has fever is taxed in the village market. Nobody knows the destination of Africa’s tax proceeds. We only know they do not go to municipal services. Municipal services my foot! In Africa, every upper class household is a municipality with a mayor, a generator, a bore well and a septic tank. In Africa top government officials proudly import 4-wheel drive SUVs duty free. 

In other parts of the world, corrupt officials risk prosecution, fines, imprisonment or even hanging. There is a judicial deterrent to graft. In Africa, impunity is the norm. In the best case scenarios, selective justice is applied. Only the ‘small fish’ see the jail house. Reports about major embezzlement in high places only warrant inconclusive investigations. That is why African corruption is not practiced discretely. There is no need to disguise sleaze. Conversely, scruples are equated to stupidity. That is why African auditors have no difficulty reaching adverse findings.

via Abhijeet Singh 

How to invest in Africa

Todd Moss and Ross Thuotte at CGD made the case a few weeks ago for investing in Africa (literally). 

The question for how best to do this popped into my head a few weeks ago at a conference on business in Africa. So what can you do? I have some very meagre savings, and interest rates are so low right now in the UK they may as well be stuffed under my mattress. Surely there is a way of tapping into the high growth in developing countries?

I asked the question on twitter and got these five responses:

‏@davidcshipley: There's plenty of options which will get you way more than 2% - EMD, AR strasts etc. (Not investment advice). for Africa specifically there are some frontier funds which directly focus there, plus EM Eqs which have some allocation 

@Scott_Gilmore:  Try http://myc4.com  

@StyledByAfrica: Check out @Homestrings about investing in Africa 

‏@tylercowen: Some contemporary African art is fairly liquid, it has a high bid-ask spread but one way to start... 

‏@RealClearAfrica:  http://investinginafrica.net/#cat-1

Anyone have any experience with any of this or any other tips?

Chart of the Day: What do Africans think their governments should be doing?

Afrobarometer asked over 33,000 Africans between 2010 and 2012 what the most important problem facing their country that government should address is. Here are their answers. With apologies for the tiny font, but it's worth reading down the full list (I left off a few of the country-specific responses at the bottom).

Data from: Benin 2012, Botswana 2012, Burkina Faso 2012, Burundi 2012, Cape Verde 2011, Ghana 2012, Kenya 2011, Lesotho 2012, Liberia 2012, Malawi 2012, Mali 2012, Mauritius 2012, Namibia 2012, Nigeria 2012, Sierra Leone 2012, South-Africa 2011, Tanzania 2012, Togo 2012, Uganda 2012, Zimbabwe 2012 (Base=33598; Weighted results)


I'm quite surprised by how high up water supply is, but less surprised by the top 3 of unemployment, the economy, and poverty. The public policy challenge is still, first and foremost, about broad-based inclusive economic growth. Interesting to compare this with Justin Sandefur's analysis of what African researchers care about (jobs).

The tragedy is that we don't really have a clue what policy instruments can create jobs. For most of sub-Saharan Africa the challenge is a lack of demand for labour. What is needed is a way of linking African workers with consumers who have money - who are mostly in rich countries. This link could come in 3 ways:

1: Trade. Africans stay where they are and export things to rich countries. This one looks difficult in most countries, which are uncompetitive with poor Asian countries in manufacturing, and don't yet have the skills or infrastructure for high-tech service exports. Gains to agricultural productivity holds some promise, but faces serious barriers to getting going.
2: Migration. The Africans come to rich countries. An economic no-brainer, and a political non-starter.
3: Tourism. The rich people go to Africa. Tourism? Really?

There will probably be marginal improvements in all 3 areas, but its hard to see where the really big shift that could get millions of Africans up to rich country poverty lines of around $12.50 per day over the next generation is going to come from.

The very easy to use online Afrobarometer data analysis tool is here.

[and before anyone says it, of course Africa is not a country, but actually the patterns look pretty similar when you look at the country-level data, I just couldn't figure out a good way of showing that data visually - very open to suggestion]

"Aren't you going to recycle that?"

I'm terrible about remembering to recycle. It's partly moving about so much and not keeping up with the different rules in each city. It's partly just laziness.

Andrea reprimanded me for forgetting last week, and after some debate I realised that though she was right about the recycling and my excuses were lame, I was annoyed about the reprimanding. Or rather interested - how and when did it became socially acceptable to reprimand people for forgetting to recycle, or for wasting food as the pope just did (by the way, it's great that this new pope is such a do-gooder, but wow is he a naive do-gooder for someone who has presumably been in the do-gooding business for... how many years?).

And why is it not socially acceptable to reprimand people for wasting money on crap they don't need when they could be giving it to charity and saving lives? Sure it's annoying, but so is being reprimanded about recycling.

In the UK at least I think a lot of this is our squeamishness about talking about money in general. As Kate Fox writes:

Our distaste for money-talk in everyday social life is well established: you never ask what someone earns, or disclose your own income; you never ask what price someone paid for anything, nor do you announce the cost of any of your own possessions. In social contexts, there is a sort of ‘internal logic’ to the money-talk taboo, in that it can be explained, to some extent, with reference to other basic ‘rules of Englishness’ to do with modesty, privacy, polite egalitarianism and other forms of hypocrisy.

But at the same time it is kind of nuts that there are such good giving opportunities out there to make the world a better place and we're not allowed to talk about them. In the UK only 39 percent of people give more than fifty pounds a year to charity (NPC 2013). Of those, the average amount for "mainstream" donors is £303 a year, for high-income donors £1,282 a year. Meanwhile, you could be saving a child's life for as little as £1500.

I have a bit of an advantage here because I'm from Yorkshire. As Kate explains:

There are pockets of stronger resistance to the money-talk taboo, particularly in Yorkshire, a county that prides itself on being forthright, blunt and plain-spoken, especially on matters that mincing, hesitant southerners find embarrassing, such as money.

Also it's much easier to type things into the internet ether than to actually say them to people in person. So consider this an annoying reprimand. We have the tools. We have efficient, low overhead, transparent, charities that have proven impact on poverty and child health. Excuses about waste and corruption just don't cut it any more. Aren't you going to recycle that?

Addendum:

I waste money on crap I don't need all the time, and it's annoying to be reminded of that. But is it bad to be reminded? Shouldn't I feel guilty? Andrea owes me some money, and so I'm making her give it to poor Kenyans via givedirectly.org instead of back to me, mostly just in order to annoy her or something. Which had instant positive results - she writes "excellent punishment actually. I started searching "bobbi brown cream eye shadow" and had to close it because the givedirectly page was open right next to it". How about that for a nudge - keeping a givedirectly tab open in your browser all the time?