08 May 2012

OMG Millennium Villages Increase Poverty ROFL!!

The Millennium Village PR Department Guardian newspaper reports "Child mortality down by a third in Jeffrey Sachs's Millennium Villages." Which is possibly true (I'm not going to even go into the validity of the non-random controls). But if you take a casual glance at the paper's results table, you'll also find no statistically significant impact of the project on poverty, nutrition, education, or child health.

Of all 18 indicators, 10 are totally statistically insignificant (no difference between intervention and comparison) and only 1 of the 18 indicators is significant at the 1% level.

The text of the Lancet paper mentions 3 times that poverty has fallen in the village sites. And just once that this reduction is actually no different to that in comparison villages.

And check out this sentence;
For 14 of 18 outcomes, changes occurred in the predicted direction. No significant differences were recorded when comparing poverty ...
So, mention the direction of the effect when it is the direction you want (but statistically insignificant from zero), and neglect to mention the direction of the effect when it is the direct opposite of what you want (but also insignificant).

Now THAT, folks, is science. (Here's the Lancet link, HT: Maham). 


Post a Comment