Coaching is better than training, but there is still a questionmark on scalability

"So should governments switch to frequent coaching sessions? Possibly, but the next step should first be to try this type of intervention at scale. 

Finding three highly skilled coaches is one thing, but you might need hundreds or thousands of them if you were to run a similar programme across an entire country. 

One potential route to scale is through new uses of technology. A study in Brazil found positive impacts of a virtual-coaching programme run via Skype, for example. 

But perhaps the most straightforward type of technology to go for is scripts, which this paper suggests have positive effects on learning both when presented through centralised training and intensive coaching."

My latest at TES.com

How to improve capacity building

Shanta Devarajan at the World Bank thinks we should be focusing on demand-side interventions (demand for improved performance, rather than "supply-side" interventions focused on training etc to improve the supply of improved performance), and then doing rigorous experiments to test these interventions (yes, RCTs).

SD: It is definitely not just about technical solutions. At the first level it is a question of incentives. And it is even deeper than that. At a fundamental level, it is a problem of politics.

Well worth reading in full. (HT: TH)

Do Urban Livelihoods Programmes Work?

Apparently not in Sri Lanka.

The authors conduct a randomized experiment among women in urban Sri Lanka to measure the impact of the most commonly used business training course in developing countries, the Start-and-Improve Your Business program. They work with two representative groups of women: a random sample of women operating subsistence enterprises and a random sample of women who are out of the labor force but interested in starting a business. They track the impacts of two treatments -- training only and training plus a cash grant -- over two years with four follow-up surveys and find that the short and medium-term impacts differ. For women already in business, training alone leads to some changes in business practices but has no impact on business profits, sales or capital stock. In contrast, the combination of training and a grant leads to large and significant improvements in business profitability in the first eight months, but this impact dissipates in the second year. For women interested in starting enterprises, business training speeds up entry but leads to no increase in net business ownership by the final survey round.

Suresh de Mel, David  McKenzie, and Christopher Woodruff , "Business training and female enterprise start-up, growth, and dynamics: experimental evidence from Sri Lanka" (HT: @timothyogden)

Targeting the Hard-Core Poor

As briefly flagged in the IPA Annual Report, there are some exciting positive results coming out of the BRAC graduation model. IPA is coordinating evaluations all over the world, and some of the first results are coming out of the Bandhan implementation in India, the evaluation led by Abhijit Banerjee, Esther Duflo, Raghabendra Chattopadhyay, and Jeremy Shapiro (paper here).
As The Economist reports;

Well after the financial help and hand-holding had stopped, the families of those who had been randomly chosen for the Bandhan programme were eating 15% more, earning 20% more each month and skipping fewer meals than people in a comparison group. They were also saving a lot. The effects were so large and persistent that they could not be attributed to the direct effects of the grants.

What worries me is how scalable this programme is. That word "hand-holding" worries me. How many developing country governments have the resources and management capacity to arrange for a detailed skilled-labour-intensive personalised package of intensive support for every poor person? How many developed country governments have the capacity for that?

It worries me especially when there are so many easily scalable cost effective programmes out there that are not being funded. Why not focus first on the simple things that we know to work, like universal child grants or universal school meals, that can easily have a big impact on millions?

On capacity building

Hypothesis: Effective organisations are built by insiders who have learnt how effective organisations work whilst at a different, established, effective organisation, rather than by outsiders coming in and making bad organisations into good ones.

Application to development: it might be better for large international NGOs to directly hire more local staff to deal with ground-level implementation than to “partner” with local organisations and try to delegate tasks to them, and struggle with that whole capacity building thing, which is inherently incredibly difficult, especially when you are an outsider, and which very few people seem to really know how to do very well. If someone works for you directly, you have far greater control over their work, and they get to experience working in an established effective organisation, which may well do more for capacity building in the long run.

Any thoughts? In particular any data or evidence, even anecdotal?

---

A better stated hypothesis by email from Monica: a lot of the soft skills stuff that comes up with capacity building  (i.e. leadership development, management training) is very important but very difficult to teach. It is better learned through "modeling" than workshops. Think students in the West with expensive long degrees who still need to do unpaid internships to learn how to actually do a job.

We need to talk about capacity-building

My new employers Oxford Policy Management have just published an interesting new opinion piece by Alex Matheson looking at the failure of capacity-building.

A 2006 Australian report ‘Capacity Building Evaluation’ states that the development community spends $15 billion annually on capacity development but is “unsure” of the return on its investment. In 2008, the World Bank concluded that no more than half of the $720m it spends on training each year actually resulted in enhanced capacity.

That bears repeating - less than half of World Bank spending on training actually does anything.

So why does this continue? Matheson explains how both donors and recipient governments have every incentive to carry on with the status quo and not rock the boat. For the recipients, well who doesn’t like an excuse to get out of the office for a few days of training? And for donors, it’s a nice clean easy way to spend some money, which may well have at least some short-term superficial outputs.

What is typically though needed in dysfunctional organizations is not more inputs into a bad organizational system, but the much more challenging job of managerial and organizational reform.

The first quick win is to be clear on terminology.

Training ≠ capacity development. 

The two are not synonymous (technical training for individuals can of course be very useful, we should just be clear that it is not going to create effective organizations, meaning the working relationships between individuals).

Finally Matheson offers up some more substantive solutions;

• Making a prior political economy assessment of the barriers to capacity development and the prospects for building conditions for successful partnership. 

• Not engaging unless the leadership group in the organisation is committed to the goals and is willing to allow space for change. 

• Designing interventions that take account of the political and economic environment. For instance, organisational change strategies in civil services in South Asia must take into account the power and influence of cross-departmental cadres, and the strong bonds between batch-mates. 

• Designing support programmes that allow enough time for new capacities to be institutionalised and are sufficiently flexible to survive political inattention, senior staff turnover, and periodic distractions. Important elements of this are establishing inclusive networks and engaging on a wide front to allow for changing partners and temporary reversals. 

• Promoting a process of change that allows time and opportunity for confidence building through small successes amongst the leadership team, before addressing major challenges. 

• Adopting a patient but persistent approach that keeps long-term behavioural goals in mind, exploits windows of opportunity, avoids unnecessary confrontations and focuses on results rather than publicising a badged reform.