21 January 2013

When rigorous impact evaluation *does* make quite a big difference

If you care at all about unemployment and labour market policy, or really about much of social policy, this new paper from Esther Duflo and co-authors should have you quite worried.

The policy - pay a private provider for each unemployed person that they get into a job.

The result (part 1) - the policy was successful at getting unemployed participants into jobs.

The result (part 2) - almost all of these jobs were just taken from other people who would otherwise have got them. Pure displacement. No net change in unemployment.

Most impact evaluations don't measure such "spillover" effects or "externalities", because they are really hard to measure (neither do most non-randomised evaluations.., this is not a criticism of RCTs).
Ignoring externalities, we would have thus concluded, for example, that 100,000 euros invested in the program would lead 9.7 extra people to find a job within eight months. Since the eff ect disappears by 12 months, this already appears to be quite expensive, at about 10,000 euros for a job found on average four months earlier. But at least, it is not counterproductive. With externalities, investing 100,000 euros leads to no improvement at all.
Bruno Crepon, Esther Duflo, Marc Gurgand, Roland Rathelot, and Philippe Zamoray (2012), Do labor market policies have displacement effects? Evidence from a clustered randomized experiment

1 comment:

Post a Comment