What should NGOs say & do about private schools?

Susannah Hares from Ark and I attended a meeting with the senior staff of an INGO last week, along with a couple of other external invitees, to help them think through their position towards private schools and fees in education. We wrote up our recommendations & takeaways on the Huffington Post.

First and foremost, it should reaffirm its principles: that education should be free; that government should be the guarantor but not necessarily the sole provider of education.

Second, it should focus its education reform efforts on setting high expectations for all children and improving accountability for all schools, whether public or private. School inspections, good data, robust assessments: these are all critical components of a good system accountability and should apply to public and private schools alike.

Third, in contexts in which substantial numbers of children already do attend private schools, the INGO could see if there are any clear market failures in the private sector that could be improved by government or NGO intervention. For example, training of teachers, or addressing information asymmetries through better community engagement.

Fourth, where public-private partnerships are planned or in place, the INGO should challenge government and private sector partners to place equity and quality at the heart of any programme. The UK academy programme started out under the Labour government with a very clear objective: to turn around the 200 worst performing secondary schools in the most deprived areas. This laser sharp focus on closing the achievement gap between rich and poor has changed the conversation about education in the UK: a school cannot be rated as “outstanding” anymore unless it can show it is delivering learning gains for children on free school meals.

Finally, and perhaps most radical of all, the INGO could set up its own network of non-profit PPP schools in a developing country. Demonstrating what high quality education for disadvantaged communities looks like would be a significant value add to any system, particularly if delivered through a PPP with the sustainability and public accountability that should bring. The experience of PEAS schools in Uganda shows that it is possible for an NGO to deliver better management practices and improve performance, at the same cost as local operators and with the inbuilt sustainability that comes from domestic government financing. And Ark’s own experience of running a high performing network of non-profit PPP schools in the UK shows how much an organisation can learn from actually running a set of schools, and how much more effective advisors to governments this can make them. Finally, establishing more non-profit providers in this space might calm some of the heated rhetoric around some of the international for-profit providers.

Effective Altruism, RCTs, NGOs, & the Government End-Game

Good Ventures just gave a $25 million unrestricted grant to Give Directly on the advice of Givewell. That’s a lot of good news in one sentence, but it’s not even the best part. Givewell buried the lede when they mention around paragraph 20 that;

"GiveDirectly plans to discuss partnerships with the following types of institutions:

- Donor aid agencies
- Developing country governments (national and local). (For example, several governors in Kenya have already approached GiveDirectly about running cash transfer programs in their counties.)"

That’s what it’s all about. To really get sustainability and scale in social policy you need government involvement - that’s why the best NGOs combine a mixture of immediate direct service delivery in places where government just doesn’t have the capacity to deliver, with support to interested governments to build that capacity for the longer-term, often at the local level where administrators struggle to actually implement well-designed central policy documents, and with innovation in new models of service delivery, that governments might later adopt, of which GiveDirectly is clearly a strong example. Similarly whilst Innovations for Poverty Action and J-PAL may have started off following that recently infamous Kremer-Miguel deworming study by working on service delivery through small NGOs, their focus is on things that can work at scale, and having built a reputation through working with NGOs have been able to transition to working with governments (for example in Ghana and Peru).

As Jessica Brass writes,

"Government and NGOs learn from each other to improve what they do. In particular, many government agencies notice the successes achieved by NGOs and, whether intentionally or not, mimic their actions"

So yes, maybe some of the effective altruists can be accused of being philosophers not development wonks, and potentially even naive about politics, but for every anecdote-backed theoretical case for how aid might undermine the process of building citizen-state accountability, I can come up with an anecdote-backed theoretical case for how aid can support improved governance through innovation in service delivery models, and until we get some quantitative evidence on the issue, I don’t see how else we’re going to resolve the debate.

Did I miss anything?

Social media and the End of NGOs?

Then of course there is the issue of whether in the future we will even need international NGOs any more. Will people just be able to use social media to directly sponsor a child in the community? 

In development there is a tendency to think about trends and gradual change but we need to be aware of the possibility of disruptive change. Development may face a complete sea change at some point - similar to what happened to the music industry where suddenly downloads decimated the CD market. However, preparing for that is more difficult: usually the change that comes is not what you put on your risk register.

Caroline Harper, chief executive of Sightsavers interviewed in the Guardian

Livelihoods 2.0

Duncan Green outlines an emerging new approach at Oxfam;

how do these projects differ from traditional income generation? For decades, NGOs have been showing up in communities and persuading people to raise chickens or rabbits, open tailors, or plant the latest new wonder crop. The record is decidedly mixed. What’s different about this latest round? 

- Involve local government and private sector from the outset - they are the only long term guarantors of ‘sustainability’. 

- Scale - it’s no use just running a pilot and then crossing your fingers. From the outset, you have to think how your intervention needs to be designed to benefit 100,000s of people, rather than 100s 

- It’s about value chains, not just production. Often the real barrier is not growing or making stuff, but finding the credit you need to keep you going between planting and harvest, getting the product to market (the roads here are terrible, gulleyed by rain and gouged by illegal logging trucks), or finding a reliable buyer who pays decent prices. Multiple actors need to be involved - it’s no use just funding a local NGO to hand out seedlings. Systems analysis is essential 

- Advocacy: a systems approach resembles a micro version of Dani Rodrik’s bottlenecks to growth. Resolving one bottleneck (eg supplies of decent seeds), allows the effort to move on until it hits the next one (roads, access to finance, quantity and quality). Some of these can be incorporated into the programme, but many require local level advocacy (eg lobbying the public works department to do something about the roads, or the state bank to start lending to long gestating crops like rubber).

How to find an NGO to support

It is likely that bigger and more well-established agencies will be better able to answer these questions than smaller ones, though these do not have to be international agencies. This is an important observation as it suggests that bigger, more experienced, and more independent organizations with a greater range and depth of skills and deeper knowledge of the countries in which they are working are more likely to make wiser choices about how to deploy their funds than are smaller and newer agencies, which are often run and staffed by people with little development and country experience. It should be added that there are not only many competent nationally based poverty-focused organizations but that many of these have a better understanding of poverty and especially how it might be eradicated faster than do some international agencies. Also, it must not be thought that it is only the bigger agencies that do good development work: many smaller, especially locally based, agencies perform very valuable work. Additionally, a number of smaller agencies set up by people now living in the industrialized world but based on a deep understanding of the local communities in need—such as Send a Cow—continue to have a significant impact. The challenge is finding out about them.

Thoughts from Roger C. Riddell, a Non-Executive Director at Oxford Policy Management, writing in a special issue of  "Ethics and International Affairs" (HT: themonkeycage)

Whither participation?

Esther Duflo's recent Tanner Lecture (download it here, really, go and read it now) offers a provocative attack on the dogma of community participation in NGO development projects. The lecture builds on ideas expressed here and here and drawing from the recent behavioural economics literature on limited attention. Sometimes a bit of paternalism is good for us. Good defaults are good. You don't need to worry about most services in England because everything just works.

I asked people in Masisi a few weeks ago whether they had been consulted enough by the big international NGO running the programme in their town. Some were happy, but some were a little nonplussed - "it's their [BINGO] programme - let them decide how to do it". One guy who had not been selected for assistance (through a very participatory community ranking exercise) suggested that if big international NGO wasn't going to pick him, then maybe they should be doing the work of selection and not asking him to spend half a day doing it (note this is in no way a criticism of BINGO, they exemplars of best practice. It is a tentative criticism of that best practice, which is widely supported). 

Many social programs insist on beneficiary participation in management, claiming that it is valuable and instrumental for program success. 

Perhaps, for rich parents who have the luxury of being able to spend time worrying about their children’s educations, participating in the Village Education Committee and being given a voice to obtain more resources for their schools is indeed empowering. Poor parents may care just as much about education, but may have no energy left to figure out exactly how to work the system or to figure out what they might be able to accomplish when they are given vaguely defined powers ... perhaps, finding ways to make schools actually work without the community having to worry about it at all would be even more empowering.

Back in England, I can't imagine anything worse than having to meet all of my neighbours after work to figure out how we are going to run the rubbish collection or fix the potholes in the road. That stuff just gets done. Services get delivered without me having to think about it at all. All I need is a mechanism to complain if things don't work, but don't ask me to help you plan how to fix it. 

There is a great quote from the earlier paper "Mandated Empowerment" (HT: @thrh)

Both examples raise concerns about committing ourselves entirely to antipoverty strategies that rely on the poor doing a lot of the work.

When you put it like that, it sounds pretty sensible. The implication of which is not I think "don't consult people," quite the opposite - provide an open platform for suggestions, comments, and complaints. Just consider how much work you are asking from your "beneficiaries."

Your thoughts?

On capacity building

Hypothesis: Effective organisations are built by insiders who have learnt how effective organisations work whilst at a different, established, effective organisation, rather than by outsiders coming in and making bad organisations into good ones.

Application to development: it might be better for large international NGOs to directly hire more local staff to deal with ground-level implementation than to “partner” with local organisations and try to delegate tasks to them, and struggle with that whole capacity building thing, which is inherently incredibly difficult, especially when you are an outsider, and which very few people seem to really know how to do very well. If someone works for you directly, you have far greater control over their work, and they get to experience working in an established effective organisation, which may well do more for capacity building in the long run.

Any thoughts? In particular any data or evidence, even anecdotal?

---

A better stated hypothesis by email from Monica: a lot of the soft skills stuff that comes up with capacity building  (i.e. leadership development, management training) is very important but very difficult to teach. It is better learned through "modeling" than workshops. Think students in the West with expensive long degrees who still need to do unpaid internships to learn how to actually do a job.

Does aid improve governance?

Jennifer Brass makes the case in the "Governance" journal [gated] that international NGOs operating in Kenya have contributed to improved democratic governance and accountability. This contrasts with the Bauer/Easterly/Moyo-ish lines that external aid undermines governance, but resonates with my recent limited experience looking at NGO programmes in Kenya which comes across as pretty positive interaction with government, and also with something Paul Collier has hypothesised, that the difference between aid and oil (and thus the explanation for better outcomes on average from aid than oil) is the added value of the technical assistance.

Brass writes

"Governance is no longer the purview of only public government actors; it is increasingly seen as a shared or networked process among several types of organizations. 

Government and NGOs learn from each other to improve what they do. In particular, many government agencies notice the successes achieved by NGOs and, whether intentionally or not, mimic their actions, recalling DiMaggio and Powell’s (1983) mimetic isomorphism. This is most obvious in their attempts at participatory approaches, in which opinions from the village to the city are solicited (if not always actually listened to). As a result, governance in Kenya has slowly begun to more democratic, moving away from its hierarchical, authoritarian past."

She also reports broad individual support for NGOs in Uganda and Kenya (perhaps not a surprising result that people report to surveyors that they like people who give out free stuff);

"in a survey of NGOs in Uganda, 90% of organizations reported involving host communities in the delivery of services, and nearly 60% of beneficiaries of these NGOs agreed that the NGOs seek community participation (Barr, Fafchamps, and Owens 2005). While NGOs claimed more participatory involvement than the respective communities saw, 60% participation rates are significant. Relative to the Kenyan government and its public administration over the past 40 years, NGOs unquestionably try to be more participatory and accountable. 

Kenyan citizens agree, viewing NGOs as looking after the interests of the common man. When asked, “To what extent do you think NGOs have the interests of the people in mind?” in a survey, the author conducted on service provision and service providers with 501 Kenyans, 70% of respondents answered positively, and only 20% responded negatively."

Finally, a separate paper finds that NGOs on average choose to locate themselves where need is great, but also in convenient locations (close to roads and towns - which isn't necessarily the worst thing for cost effectiveness), and best of all not due to political factors.

BRASS, J. N. (2012), Blurring Boundaries: The Integration of NGOs into Governance in Kenya. Governance, 25: 209-235. doi: 10.1111/j.1468-0491.2011.01553.x

Scaling up Proven Interventions

If you are in the business of piloting development policies with NGOs, this chart should be keeping you awake at night. If you like to think about "sustainability" and "scale", about handing over your activities to the government, you need to be really really worried. 

Researchers persuaded World Vision and local government in Kenya to both implement that exact same intervention at the same time. The program as implemented by World Vision found a large impact on test scores. The exact same program, as implemented by the local government, found zero impact. 

For more see Gabriel Demombynes.

His conclusion:

Evaluation skeptics may try to cite this as evidence that RCTs are a waste of time, since it suggests that successful interventions implemented by NGOs, as they often are in experiments, may not be replicated at scale by governments. Others might take the paper to indicate that NGOs should be the preferred vehicle for interventions. I think these readings would be mistaken, and I take two reflections from the paper. First, we should do many more rigorous studies working with governments where we vary forms of service delivery to better understand what can work in practice. Second, the World Bank’s approach to public services—the long, difficult slog of working to improve government systems—is the right one, because it’s the only way to ultimately make services work for the poor at large scale.

 I agree. Clearly working through government systems is essential. Innovations for Poverty Action are doing just this - scaling up the exact same contract teacher program tested with NGOs in Kenya and India, but doing it with the government in Ghana, and doing an RCT as they go.

Addendum: Here is the link to the full paper: http://www.cgdev.org/doc/kenya_rct_webdraft.pdf

The Impact of IPA

Measuring policy influence is hard, but this looks like a slam dunk for IPA: Concern Worldwide are launching a new Gates Foundation funded initiative focused on Innovation, Evidence, Action, and What Works.

Compare and contrast the Concern website;

and the IPA website;

Congrats all round. 

NGOs, RCTs, and Institutions

There is a malicious and perverse relationship between the force of NGOs and the weakness of the Haitian state.

That is Ricardo Seitenfus, the Brazilian head of the Organization of American States mission to Haiti during the 2010 earthquake - quoted by Acemoglu and Robinson.

This is the real challenge to aid project impact evaluations, whether randomised or not, and is really the key argument made by the critics from Bauer to Easterly to Moyo. 

What isn't clear to me is how we should expect the impact of NGO activity on state strength to vary by the effectiveness of the project. Does the measured impact of the project on individuals have any relationship with the impact of the project on broader governance? It's at least plausible that a very effective health intervention (measured by outcomes for individuals) could also be effective in building state capacity for service delivery. Or on the contrary, the same measured improvement in outcomes for individuals could be consistent with undermining state capacity, by encouraging the state to spend less on health and more on other goodies, and also by poaching the best staff from the government into the NGO sector. Tricky.

Answers / more data sources on a postcard. 

Evaluating TOMS shoes, child sponsorship, cow donations, and fair trade coffee

And the prize for most diverse recent set of popular NGO development program intervention evaluations goes to (drum roll please) **~!Bruce Wydick!~**

James Choi quotes him writing in Christianity Today on Fair Trade Coffee:

Fair-trade coffee isn't a scam, but it is hard to find a development program that has attracted so much attention while having so little real impact. The most recent rigorous academic study, carried out by a group of researchers at the University of California, finds zero average impact on coffee grower incomes over 13 years of participation in a fair-trade coffee network.

 So I looked up his page and found this on child sponsorship:

Although international child sponsorship may be the most widespread form of personal contact between households in wealthy countries with the poor in developing countries, to date there are no published studies that have analyzed whether the beneficiaries of these programs have experienced changes in their life outcomes.  

We find large and statistically significant impacts of the [compassion international] child sponsorship program on most of our outcome variables.

And then there are rigorous impact evaluations to come on Heifer International and TOMS shoes - exciting stuff!