Bias

Brooks discusses the Well-Plannned Life vs. the Summoned Life.

Could this be a good excuse for those of us who never plan more than 5 minutes ahead?

Is it possible I like Searchers because of my personality type?

That is Easterly on why he isn’t a planner. That’s ok Bill. Half of the reason I like randomised trials so much is because it’s a bit like gambling…

Learning to Randomise

So yesterday was the last day of this year’s training week for JPAL and IPA staff based in Africa. I’m going to be starting in September as the IPA Communications Coordinator (professional blogger?!) based in New Haven, so this was a great opportunity for me to meet a bunch of the field and head office staff, and learn a bit more about how JPAL and IPA actually conduct an evaluation.

Things I have learned:

1. The phrase for “gamble” in Liberian English is to “put it in the hole.” Which apparently makes for some entertaining interviews with youths about their appetite for risk.
2. Limuru, Kenya, is COLD!
3. People can go to great lengths to get a drink. (but this is OK).
4. IPA staff are awesome. Everyone is fun, clever, and motivated, and are all total data-geeks. My kind of people.
5. Oh yeah, and how to conduct a randomised evaluation.

Here is my ultra-condensed summary.

Day 1: Why randomise anyway? (Because if you care about measuring the impact of your project - this is by far the best way)

Day 2: How to use STATA. (just spend a few hundred hours learning to code)

Day 3: How to design an evaluation, and how to manage data collection in the field. (what are you randomising and why? how do you manage the logistics in the field)

Day 4: Data entry and project management.

Day 5: Ethical and privacy issues, dealing with research problems, and budgeting (I totally rocked the budgeting session)

Day 6: Bringing it all together: presenting a complete project design from start to beginning.

If you are interested, all of the training materials are available on the MIT website, including videos, lecture notes, case studies and exercises.

The only thing not included is the trip to Lake Naivasha to walk amongst the zebra and giraffe. For that, you might just have to go and sign up…

Good research ideas

We’ve set up each of our randomized control trials with a team of one to two full time Liberian qualitative researchers. Over two years we’ve discovered people with an incredible talent for observing, talking, confiding, and (increasingly) analysis. They get reams of information and insight unavailable to the foreigner. The data are voluminous. If the interventions work, we have a solid idea why. Another benefit: new questions and hypotheses to test quantitatively—some more interesting than the originals. And it’s a poor but respectable substitute for the dwindling time I have to spend in the field. Every research project should build this in.

Chris Blattman on mixing qual and quant methods.

Randomisation done right

Probably the primary criticism of randomised control trials (RCTs) is that although they prove something very well in one context, the findings aren’t necessarily transferable to another context.

Another major criticism is that they tell you what happened but not necessarily how it happened.

To solve both of these problems what you need is theory. There is of course nothing intrinsic to doing an RCT which precludes also using this evidence to test or develop a theory.

Two new papers by leading researchers in this field do just this:

A BREAD Working Paper by Karlan, McConnell, Mullainathan, and Zinman explains why reminders to save are so effective, and a Working Paper by Duflo, Hanna and Ryan explains why financial incentives are effective at getting teachers to turn up.

 

Getting to the Top of Mind: How Reminders Increase Saving

We develop and test a simple model of limited attention in intertemporal choice … We find support for [the model’s] predictions in three field experiments that randomly assign reminders to new savings account holders.

 

Incentives Work- Getting Teachers to Come to School

We use a randomized experiment and a structural model to test whether monitoring and financial incentives can reduce teacher absence and increase learning in rural India.

What is better than deworming for increasing school student attendance?

For me this was the stand-out chart from Esther Duflo’s recent TED talk. Apparently providing information on the returns to education does even better than deworming for increasing school attendance.

I am somewhat surprised that there haven’t been more efforts to replicate the Kenya deworming finding, which is quite old now (noting that there is one study on India here - but what about other African countries?). A key defence of the randomistas position is that external validity can be improved by replication of findings in different settings. The incentives probably aren’t there for academics to do boring replication studies, but surely there is an incentive for donors to want to fund such studies?

How do poor people define poverty?

A fascinating new paper by Ben Olken and Abhijit Banerjee uses a field experiment to look into the difference between “objective” consumption-based measures of poverty and what Indonesian villagers define as being poor. They find that a community ranking exercise basically does at least as well as an objective alternative in estimating consumption (until people get tired of the exercise), but that villagers value other non-consumption factors in determining who is poor.

Independent of consumption, poorer households are deemed to:

• Be smaller (reflecting a view that there are household economies of scale)
• Have more children
• Not be elite-connected
• Not be connected to the financial system
• Not have family outside the village (who might be able to send remittances)

the community seems to have a widely shared objective function that the government does not necessarily share, and implementing this objective is a source of widespread satisfaction in the community. Moreover, what makes this objective function different is neither nepotism (elite capture) nor majoritarian prejudices. Rather, these preferences appear to be informed by a better understanding of factors that affect the earning potential or vulnerability of the household, such as the returns to scale within the family, incentives, and insurance, as compared to relying purely on consumption as the government does.

Targeting the Poor: Evidence from a Field Experiment in Indonesia, Vivi Alatas, Abhijit Banerjee, Rema Hanna, Ben Olken, and Julia Tobias. May 2010.