20 July 2016

Answering THE big question in global education: Why is Vietnam such an outlier?

This post first appeared on the RISE website

Why do Vietnam school children score over 100 points better on comparable tests than the average for low-income countries?

Vietnam is basically the only low-income country in any of the internationally comparable tests that performs at the same level as rich countries. Vietnam is a massive outlier, performing substantially better than should be expected for a country at that level of income. Rich OECD countries such as the UK and US flock to see the top performing places in the world on the PISA test to try and understand what is so special about education systems in Shanghai and Finland that enables them to perform 100 points better than the OECD average. Vietnam scores over 100 points better than the average for low-income countries.

And this isn’t just on one test – other research by Abhijeet Singh has linked the Oxford Young Lives survey with the international TIMSS test, and again Vietnam massively outperforms the other low-income countries (see chart). Singh’s study shows that the advantage starts early, with Vietnamese children slightly outperforming those in other developing countries before they even start school at age 5, but this gap then grows each year. A year of primary school in Vietnam is considerably more ‘productive’ in terms of skill acquisition than a year of schooling in Peru or India, the paper finds. The question this research raises – and the Vietnam experience suggests - is: “Why is learning-productivity-per-year so much greater in some countries than others?” Or to put it more simply, why are schools so much better in some countries?

new paper by World Bank researchers Suhas D. Parandekar and Elisabeth K. Sedmik shows just how difficult the “Vietnam effect” is to unscramble. With a statistical decomposition using available measured factors, the research suggests that a combination of targeted investments and “cultural factors” explain roughly half of the huge ~100 point gap between Vietnam and the other low-income countries on the PISA test.

The main factors are:


  1. Higher level of access to pre-school.
  2. Investment in school infrastructure, especially in cities and small towns.

And Cultural factors

  1. Students work harder – skip fewer classes, spend the same or more time in school, plus substantial extra time studying after school. Students are more disciplined and focused on their studies.
  2. Teachers appear to benefit from closer supervision of their work by the school principal and others.
  3. Parents may have an important role to play, by taking an active part in combining high expectations of their children, following up with their children’s teachers and contributing at school.

All this only gets us part of the way toward explaining the Vietnam phenomenon. The other half of the answer remains a Vietnam enigma, which we’re hoping the RISE Vietnam research team can help to unravel, ultimately possibly even providing some useful lessons for other low-income countries.

* The chart here shows the average proportion answering a TIMSS question correctly, averaged across 6 questions focused on the number content domain, taken from the 2003 TIMSS assessment

21 June 2016

Grit: Probably not that important in developing countries

“Angela Duckworth’s new book Grit: The Power of Passion and Perseverance has been launched with great fanfare, reaching number two on the NY Times Nonfiction bestseller list. She recently gave a very polished and smooth book launch talk to a packed audience at the World Bank, and is working with World Bank colleagues on improving grit in classrooms in Macedonia.”
That’s David McKenzie in a great book review, considering what development economists can learn from this hot psychology research trend. Grit – the ability to keep going when things get tough and you aren’t successful straight away – can help explain all sorts of individual outcomes beyond tests of skill or ability. David notes amongst other things how U.S. - centric the research on grit is, and questions how large the effect of grit is even in this context.

So what do we know about the importance of grit in developing countries?

Fortunately, a separate team at the World Bank has recently been rolling out a series of surveys measuring psychological traits including grit alongside measures of skills, income, and other demographics. Data is currently available for 10 countries; Armenia, Bolivia, Colombia, Georgia, Ghana, Kenya, Laos, Sri Lanka, Vietnam, and Yunnan Province (China).

Here’s what I found from some very quick analysis.

Grit and income

I started by looking at the relationship between adults’ grit and their earnings. For seven countries there is no relationship. For the three where some relationship exists, grit explains very little of the variation between the income of individuals. (That is, in the table below, the r2 statistic is less than 0.005.)  Adding in a few basic control variables (age, parents’ education and socio-economic status) makes even that weak correlation disappear altogether.

Screen Shot 2016-05-31 at 18.23.35.png

Grit and schooling

Moving next to years of schooling, something more substantial emerges. Grit has a statistically significant relationship with schooling in every country, and throwing in a bunch of control variables doesn’t seem to make it go away. I’m not sure what to make of the magnitude though - less than half a year of extra schooling for a 1-standard-deviation increase in grit. Maybe that’s a lot, maybe it isn't. I suppose the policy-relevant question is how responsive grit might be to interventions.

Grit and learning outcomes

Finally, looking at the correlation with answers on a reading comprehension test. Grit scores pretty poorly here too; one positive correlation, one negative, and eight statistically insignificant. Other control variables by comparison do have the kind of statistically significant relationships you might expect - people tend to score better with more schooling and if they grew up in wealthier families.

I’ll spare you some of the snarkier comments from the office, needless to say that unsurprisingly to some, from a quick look the data does not seem to suggest that grit is all that important in explaining important outcomes in developing countries. Unsurprising, because all the grit and resilience and perseverance in the world is unlikely to help a child succeed at school if they haven’t eaten that day and their teacher hasn’t turned up due to a dysfunctional school system. Similarly in the labour market, individual motivation and grit by itself isn’t going to create any well paying jobs in places where the demand for labour is low because of systemic factors such as bad infrastructure and bad governance.

I will offer one caveat – this measure of grit is based on only three questions rather than Duckworth’s preferred ten, so it is possible that a better measure of grit would matter more. But I doubt it.

And finally – this is not meant as a counsel of despair. For individuals living in low-income countries, of course they should try and persevere as hard as they can to try and achieve their goals. But when it comes to making policy – we should focus on the systemic constraints that are critical to shaping people’s opportunities, rather than just telling them to try harder. Bad schools, infrastructure, and governance, are all fixable public policy problems.

17 May 2016

Markets not in everything [if you’re Pakistani]

From a Facebook friend:

So, it turns out that Pakistanis are not eligible to purchase travel insurance online with any insurance company. We are simply not on anyone's list of "eligible countries''. Someone has decided that in addition to not being allowed to enter most of the world, we also do not deserve access to basic financial services. Pakistanis wanting to travel can just f@‪#‎k‬ themselves.

Struggling for the right words… contender for absolute worst ‘aid’ project in the world

Der Spiegel reports… the EU is planning to provide training, equipment, and DETENTION CAMPS to the government of Sudan, which is led by a wanted war criminal, in order that they can prevent human beings from crossing the border out of Sudan in the direction of Europe. The project is to be coordinated by the German development agency GIZ. Wow I feel sorry for the idealists at GIZ who signed up in order to help people, and now they’re building detention facilities for war criminals.

"The ambassadors of the 28 European Union member states had agreed to secrecy. "Under no circumstances" should the public learn what was said at the talks that took place on March 23rd … Europe wants to send cameras, scanners and servers for registering refugees to the Sudanese regime in addition to training their border police and assisting with the construction of two camps with detention rooms for migrants."

Let’s recap that arrest warrant for the wanted, at large, alleged war criminal who is the President of Sudan, the man we are apparently hoping to pay to do our dirty work:

"Charged, as an indirect (co) perpetrator, with ten counts of crimes: five counts of crimes against humanity: murder, extermination, forcible transfer, torture, and rape: two counts of war crimes: intentionally directing attacks against a civilian population as such or against individual civilians not taking part in hostilities, and pillaging; three counts of genocide: by killing, by causing serious bodily or mental harm, and by deliberately inflicting on each target group conditions of life calculated to bring about the groups’s physical destruction, allegedly committed at least between 2003 and 2008 in Darfur, Sudan.” (my emphasis)

I mean, certainly you can criticise things like sanctions and aid conditionality, but is there any point at which we might consider it to be a bad idea to do business with this government? Is there any moral threshold here? Does Sudan need to get to be charged with 4 counts of genocide before we stop giving them stuff? Is only 5 counts of crimes against humanity not quite enough??

Note that the arrest warrant refers only to Darfur, ignoring all the other confirmed bombings of civilian and humanitarian targets by military forces in South Sudan and the Nuba Mountains, ignoring deliberately destroying crops during a famine, and ignoring the reports of slavery and child soldiers in Northern/Central Sudan.

I despair.

ht: John Ashworth

12 May 2016

Content knowledge vs pedagogy in Teaching

A fascinating abstract from Roland Fryer. In many school systems you have a single teacher for most subjects in primary school, but increasingly specialised different teachers for each subject in secondary school. This RCT tried out having subject specialist teachers in primary, finding that this worsened outcomes, in theory because at least at primary level it is more important that the teacher knows the kids’ and their ability and is therefore better able to target their teaching, than that they have more specialised knowledge of the subject matter.

Starting in the 2013-2014 school year, I conducted a randomized field experiment in fifty traditional public elementary schools in Houston, Texas designed to test the potential productivity benefits of teacher specialization in schools. Treatment schools altered their schedules to have teachers specialize in a subset of subjects in which they have demonstrated relative strength (based on value-add measures and principal observations). The average impact of teacher specialization on student achievement is -0.042 standard deviations in math and -0.034 standard deviations in reading, per year. Students enrolled in special education and those with younger teachers demonstrated marked negative results. I argue that the results are consistent with a model in which the benefits of specialization driven by sorting teachers into a subset of subjects based on comparative advantage is outweighed by inefficient pedagogy due to having fewer interactions with each student. Consistent with this, specialized teachers report providing less attention to individual students (relative to non-specialized teachers), though other mechanisms are possible.

The 'Pupil' Factory: Specialization and the Production of Human Capital in Schools, Roland G. Fryer, Jr

05 May 2016

Does “the Economist” know what a “market failure” is?

Apparently not. "Do British housing markets suffer from market failure”? The answer is no.

Here’s a quick refresher - a market failure is when the market - left alone - doesn’t produce an efficient or pareto optimal solution. Good examples are externalities such as pollution or congestion, or public goods such as new vaccines. The London housing market is not a good example of a market failure, where poor outcomes are the result of clumsy government regulation restricting supply. The fact that the market is responding to insane restrictions on new supply by focusing what little building they are allowed to do on expensive rather than affordable houses is not a market failure. Yes it is a market, and yes it is producing poor outcomes, but it is failing because of over-regulation. That is a government failure not a failure of the market mechanism.

It’s one thing when ordinary people use economic jargon in a colloquial sense with a totally different meaning to the economic term, but you expect better from a magazine called the Economist.

02 May 2016

A rare non-ideological argument against for-profit schools

To [Samuel] Abrams, the problem with for-profit operation of schools is not that businessmen are making money off the provision of a public service such as education. Textbook publishers, software developers, and bus operators all make money from schools and should, he said, but they are all providing a discrete good or service that can be easily evaluated. “School management, on the other hand, is a complex service that does not afford the transparency necessary for proper contract enforcement,” he said. “Without such transparency, there’s client distrust: parents, taxpayers, and legislators can never be sure the provider is doing what was promised; and the child as the immediate consumer cannot be in a position to judge the quality of service. Regular testing has been promoted as a check on quality. But teachers can teach to the test. And worse, as we know from cheating scandals in Atlanta and many other cities, teachers can change wrong answers to right answers on bubble sheets once students are done.”

From the International Education News blog at Teacher’s College, Columbia.

The trouble with the argument is that it rests on the empirical question of whether student test-based accountability systems for schools can work, and though the question is not conclusively settled, my reading is that the evidence seems to be leaning in the direction of roughly “yes".

28 April 2016

Top 10 Must-Read Articles on Education & Development

What should you read first if you’re a new policy advisor in a Ministry of Education? Here is my response to a couple of recent emails along these lines on the CGD blog.

06 April 2016

Does it help to be African to study Africa?

For sure there is tacit local knowledge, but how you define “local” matters. Ken Opalo writes:

"As a social scientist, my knowledge of Kenya is largely informed by my experience as a Nairobian. Over the years I have had to learn a lot about the rest of Kenya, in much the same way an Australian would. In doing so I incurred a lower cost than a hypothetical Australian would, for sure, but the cost was not zero. And who is to say that I would necessarily be able to articulate a research agenda on whatever subject in Malawi better than a Southern Californian? What proportion of Kenyans can locate Bangui on a map?"

30 March 2016

The right to education is a right to learning

Kishore Singh, the "UN Special Rapporteur on the right to education” is totally freaking out that apparently the government of Liberia is planning to outsource all of its primary schooling to Bridge International Academies.

“education is an essential public service and instead of supporting business in education, governments should increase the money they spend on public educational services to make them better.”

Kishore is of course commiting Mike Munger’s unicorn fallacy. The public primary school system in Liberia was frankly pretty rubbish, even before the Ebola crisis hit, and there is no evidence that just spending more money is likely to improve things.

RTI did a baseline for their early grade reading intervention project in 2008. They found that children in Grade 3 in Liberia could read 28 words per minute. To put this in perspective, in the US children are considered to be at risk if they can read fewer than 70 words per minute at the end of Grade 2. Clearly there is nothing inherently wrong with Liberian children, the schools just aren’t working (I should also be clear that I’m not beating on Liberian teachers either, it is the system that is not functioning - don’t hate the player, hate the game). The children attending free public primary schools in Liberia might be exercising a right to schooling, but they certainly aren’t exercising a right to learning or any meaningful definition of the word education.

It should be in this context that we note that AllAfrica reports that "The deal will see the Government of Liberia paying over $65 million over a five-year period.” That is $13 million per year then, to cover a full population aged 5-14 of 920,000, or roughly $14 per student per year. That’s an absolute bargain. That’s an absolute bargain, even if Bridge are only actually covering a small fraction of that full population (Normally Liberia spends around 2.5% of GDP on education, which is around $47.5 million).

Clearly the jury is still officially out on whether independent estimates will show better learning outcomes from Bridge schools, but there is a hell of a lot of research which backs up elements of their theory of change, from how teacher qualifications and experience matter so much less than the type of contract a teacher is on, to the importance of feedback for teachers on student learning.

Meanwhile over at Kishore Singh’s website, he highlights 8 key priorities, none of which include the global learning crisis, in which there are 250 million children without basic skills, the majority of whom have already sat through 4 years of school. I’m frankly baffled by the inordinate amount of hostility shown by campaigners to social entrepreneurs who are still educating absolutely tiny proportions of the global poor, and how little attention is paid to the totally failing free public systems of schooling that millions of children are enduring.

26 February 2016

Tom Kane on Education RCTs

"If our goal is to change behaviour and drive policies towards more effective solutions, what we have done so far is a complete failure. People who are running the What Works Clearing House don’t even have a theory [of how evidence would affect policy], or to the extent that they have a theory, its been proven wrong. … We’re just deluding ourselves if we think the 5 year, $15 million studies are having any impact whatsoever."

That’s Tom Kane (somewhat echoing Lant) on the Education Next podcast. His preferred alternative to the RCT+systematic review approach though has nothing to do with crawling on any design spaces. Rather it’s doing much more quick turn-around quasi-experimental research using the multitudes of outcomes data now being collected in the US for teacher and school accountability purposes. All that’s apparently really missing is data on the actual inputs - there is amazingly rich longitudinal data on student test scores, but no record which could be matched of what textbooks are being used in different schools, or what training courses different teachers are going on. Sounds pretty sensible to me.

What’s the single biggest growth opportunity that no-one really tried?

Paul Collier and Astrid Haas just wrote an IGC blogpost “Why Kampala holds the single biggest growth opportunity for Uganda.” Single biggest? Well indeed, the first rule of blogging is HYPERBOLE, but then the first rule of reading blogs should be a heavy dose of scepticism. Second, I’m reminded of Michael Clemens’ presentation on The Biggest Idea in Development That No One Really Tried. Might migration (the kind that Paul apparently thinks is harmful for poor countries) hold a bigger growth opportunity for Uganda than better urban planning in Kampala?

At present, around 1% of Ugandans live and work overseas (roughly 400,000 of a population of 37.6 million). This 1% of the population send home 4% of Uganda’s GDP in remittances. According to a Gallup poll, around 35% of Ugandans would permanently move to another country if they were allowed to. If they all could, and sent back as much as current Ugandans abroad do, that would be a one-off 140% increase in Uganda’s economy. Or if Uganda had the same level of emigration as the UK does (8%, 5.2 million of a population of 64.1 million Brits live and work overseas), that would be a 28% increase in Uganda’s economy. And that’s totally ignoring the increase in income for the actual migrants themselves. Median income in Uganda is $2.5 per day (in PPP), so even a job on UK (“relative-") poverty pay levels of around $23 per day would be a NINE-FOLD increase for them. Needless to say, the main reason that more Ugandans don’t work in high-wage economies, is that the governments of high-wage countries impose restrictions on the entry of people, particularly those from poorer countries.

Brain drain I hear you say? Michael Clemens killed that one too (ethically it is pretty unreasonable to restrict individual's freedom to such an extreme extent, even if there were any real evidence that emigration of highly-skilled people hurt an economy, which there isn't anyway).

So yes, clearly planning Kampala’s urban development is important for growth. But does it really have more “potential” than something that could double the country’s economy overnight?

26 January 2016

The reductive seduction of OPP (Other people’s problems)

Courtney Martin has an interesting post at the Development Set about the “reductive seduction” of other people’s problems. Problems we know something about (gun control in America as her example for the Americans) seem complex, political, and intractable, whereas problems we know less about (rural hunger or girl’s secondary education or homophobia in Uganda) seem more straightforward.

Which reminds me a little of the difference between statements from leaders on education in their own country and in other people’s countries.

Here’s Julia Gillard on education in Australia:

"We need a commitment to transparency and accountability. It's my strong view that lack of transparency both hides failure and helps us ignore it. It feeds a culture where all the adults involved – the teachers, the principals, the community leaders and the members of parliament – avoid accountability. And lack of transparency prevents us from identifying where greater effort and investment are needed. Importantly, transparency and accountability are overwhelmingly supported by parents."

And Julia Gillard on education in developing countries:

"More money.

Constantly improving ways of working.

An even greater embrace of the power of partnership."

Gordon Brown on education in the UK

"we will focus on classroom standards, ensure that we monitor exam standards rigorously, reform the qualifications system … World class education, we know, achieves high standards for 100% of the children when there are systems of accountability, funding and pupil tracking that leave no child behind and personalised learning is tailored to the unique potential of every child with one-to-one tutoring and support. That world class education depends on a systematic intolerance of failure and a preparedness of public authorities to intervene and to innovate to eradicate failure"

And Gordon Brown on education in developing countries

"The biggest obstacle to what would be a spectacular achievement – as is so often the case – is a shortage of money."

Applying behavioural insights to impair life chances

David Halpern, Chief Exec at the Behavioural Insights Team (aka “Nudge Unit”) has a new blog up about how behavioural science can be used to improve people’s life chances.

"Why is it that a Kenyan market seller spends half her profit on money lenders rather than saving a tiny sum each day to escape such debt? Or why does a low income family in the UK or USA spend twice as much on a stove (cooker), bought on expensive hire purchase, than a middle class family?"

The thing is, as we know from Branko Milanovic, the country that you are born in matters more for your life chances than everything else combined. "A proper analysis of global inequality today requires an empirical and mental shift from concerns with class to concerns with location,"

So what does the BIT have to say about the movement of people? As Matt points out, the Home Office is currently paying the BIT to find ways to convince illegal migrants to voluntarily leave the UK. That is, to support the Home Office in its agenda of shutting down the single best way that exists of improving someone’s life chances. Perhaps someone could instead work on a clever “nudge” to make people less scared of foreigners?