Is deworming really effective? A response to recent deworming studies

10 min read

23 Jul 2015

Are mass deworming programs valuable? Giving What We Can’s position on the recent Cochrane Review.

Updated 26/07/2015: We made some changes to this blog post based on feedback

Givewell has published an excellent and detailed analysis of on the new deworming re-analyses and Cochrane review and concluded “Our current view is that these new papers do not change our overall assessment of the evidence on deworming, and we continue to recommend the Schistosomiasis Control Initiative and the Deworm the World Initiative”.

A recent Cochrane review paper by Taylor-Robinson and colleagues ^[1] has called into question the effectiveness of mass deworming programmes. Specifically, it suggests that there is not good evidence that mass deworming contributes significantly to overall childhood health and nutrition, educational attainment, or school attendance.

This has implications for Giving What We Can, because two of our charities focus on delivering mass deworming programmes. We wanted to update our members on the current state of play.

Health benefits of treating populations with schistosomiasis are not questioned by the review

An important takeaway from this is that the conclusions of the review specifically focuses on soil-transmitted helminths (STH) — these include whipworm (Trichuris), roundworm (Ascaris), and hookworm. The conclusions of the review on health outcomes do not relate to schistosomiasis.

Schistosomiasis is associated with more severe ill-health than STH ^[2] , ^[3] , ^[4] . Consequently, the disease burden is higher, and treating it might be more cost-effective than treating STH ^[5] . Even the Cochrane review agrees that there is benefit of treating populations with schistosomiasis:

“The evidence for the benefit of treating populations with schistosomiasis is fairly clear, as the infection has a very substantive effect on health. (Danso-Appiah 2008 as cited in the Cochrane Review by Taylor-Robinson et al. 2015 1)” ^[6]

To this end, this review does not substantially affect our overall conclusion on the health benefits of population deworming for schistosomiasis as done by the Schistosomiasis Control Initiative, even though SCI also treats both Schistosomiasis and STH (‘combination deworming’).

Deworm the World

The results may have more implications our recommendation of the Deworm the World Initiative, but at this stage we have not significantly changed our position on them.

Our recent update on Deworm the World took into account an earlier version of this Cochrane Review, which reached similar conclusions about the small effects observed on overall health and educational outcomes. However we chose to continue to recommend Deworm the World, for a number of reasons.

Firstly, a paper that suggested more positive outcomes from treating STH infections was not considered by the Cochrane Review based on their predefined exclusion criteria (even though the paper makes a good case for showing a causal relationship, it was excluded, because it was ‘Not a comparison of deworming with placebo or no treatment.’). We considered this paper in our recent report, and a subsequent analysis of its data by GiveWell:

Another recent paper looked at deworming spillovers during early childhood and suggests that there are improvements in cognitive performance equivalent to between 0.5 and 0.8 years of schooling. However, this paper looked at combination deworming. Givewell conducted a reanalysis of this data looking at the effects of STH by excluding all the schools that received praziquantel against schistosomiasis. They find that not all but many of the results still hold, which means that DtWI’s case of only deworming for STHs in India is still strong with regards to improvement of school performance.

Secondly, even if the health effects of deworming are small, the low cost of the intervention means that it may extremely cost-effective. Not only is mass drug administration cheap to deliver, DtWI provides technical assistance to the Indian government rather than distributing drugs themselves, which enables them to cause more children to be dewormed for a certain donation. Given the low cost of treatment, it is plausible that DtWI is still extremely cost-effective.

Thirdly, while the Cochrane Review does not support mass deworming programs (such as those that DtWI is helping to coordinate in India), it also makes the point that “children with worms should be dewormed”. However, as it is time-consuming and expensive to individually screen children for worms, and the deworming medications do not have harmful side-effects, the most cost-effective ways to treat infected individuals is through mass drug administration. As we noted in our recent update on Deworm the World :

In the light of this cost-benefit analysis for screening versus mass treatment, the case for mass deworming is still strong, but one might suggest that it is better to focus more on more heavily infected populations and improve trials. Because DtWI provides technical assistance and evaluation of whether an area should be dewormed based among other whether the local worm burden is sufficiently high, they have probably improved the effectiveness of mass deworming programmes in the more recent past.

If it is the case that we should deworm children, then it follows that we should do so using the most effective process — namely, mass drug administration. A recent paper suggests that treating all children at risk for worm infection would cost approximately 300 million dollars a year, but the cost of treating them via screened programs might be 2 billion dollars or more per year ^[7] .

Deworm the World has a number of programs. Its program in Kenya and Ethiopia treats both STH infections and schistosomiasis and they have plans to move to other African countries where schistosomiasis is present. Given the higher disease burden of schistosomiasis, this particular program is likely to be preventing a significant amount of ill-health irrespective of the value of concurrently treating STH infections.

It’s important to note that one of the Cochrane Review contributors caveats the conclusions of the Review by saying that they should be taken in the context of a world that is different to that of 20 years ago — specifically, with a lower overall worm burden:

“So deworming may have helped in these exceptional, heavily infected, untreated populations from another decade, but this is scarcely a solid base for contemporary policy: public health nutrition has changed, worm burden has declined and this probably accounts for the lack of effect on biomedical outcomes in contemporary studies. 10 ”

This suggests that there is still value in treating STH infections in areas of high worm burden. Recent studies in Bihar state in India indicate that worm burdens are still very high ^[8] and therefore that mass deworming of children — at least in this particular case — might be justified.

Finally, DtWI is using their donations to conduct research into deworming and its effectiveness — something that in the light of the recent papers seems to be very much needed.

Deworming and School Attendance

One of the most controversial components of the Cochrane Review are two papers that revisit a well-publicised study that links deworming to improved school attendance.

Some media reports have sensationalised some of the findings of the recent papers. We think this unfairly represents their implications. What’s actually going on?

The recent re-analysis of the original Miguel and Kremer study has recently been published by researchers working at the London School of Hygiene and Tropical Medicine (Davey et. al.). This re-analysis was made possible because Miguel and Kremer released their original data, along with the statistical modelling software that they used to produce their results. This is admirable — very few authors provide this level of transparency and ability for proper peer review, and Giving What We Can commends them for it.

There were two components to the re-analysis. The first set out to simply replicate the original study’s results. The authors corrected some minor errors in the data (rounding errors etc.) and an error in the original computer code. The replication study reproduced the majority of the results of the original, but called into question the improvements in school attendance.

The new analysis set out to compare the results obtained from the original methodology (drawing on conventions in development economics) with those obtained from an epidemiological methodology. As the authors of the re-analysis state:

Our interest in replicating M&K came not from an interest in school-based deworming but in evaluation methodology ... We replicated M&K to learn more about the techniques used in economist-led trials, by re-analysing and presenting this famous trial in a format familiar to epidemiologists.

— Davey et. al ( source )

The re-analysis found that the evidence for improved attendance at schools where treatment was given was weaker than the original claimed, and the authors highlighted concerns that, because of limitations in the data, that the results could be susceptible to bias.

The conclusion drawn from the re-analysis of the data and the updated Cochrane Review are controversial. The authors of the original study argue that their main conclusions regarding school participation gains are robust ^[9] , and that the results fit in with other studies on deworming that suggest positive impact on labour productivity and education ^[10] . A review from the World Bank also finds the re-analysis unconvincing and concludes that the results from the original study hold ^[11] . Givewell has also recently responded to these papers and argues that the evidence still seems to suggest that deworming increases labour productivity. Givewell estimates that:

“Deworming generates a net present value of $55.26, against an average cost of $1.07, i.e. that deworming is ~50 times more effective than cash transfers” ^[12] .

Obviously the re-analysis suggests that deworming may provide less of an overall benefit in the form of improved educational outcomes, but at this stage, the robustness of these results with regard to school attendance is unclear. Both sets of authors have made decisions regarding methodology that reflects their particular academic discipline (i.e. development economics vs. epidemiology).

However, from Giving What We Can’s perspective, these results are only a small fraction of the whole picture. Our recommendation of Deworm the World is not substantially based on the school attendance results of the Miguel and Kremer paper, and so we do not feel that the re-analysis — even if it does prove to be the correct interpretation of the data — significantly updates our position.

Next steps

Part of Giving What We Can's purpose is to use the best available evidence to inform our charity recommendations. We will be taking some time to incorporate these findings into our recommendation of Deworm the World, and attempt to get in touch with relevant parties to clarify a number of our concerns. However, it’s important to note that the evidence that we based our previous recommendation on has not changed significantly since we made it. At this stage we will continue to list the Deworm the World Initiative as one of our ‘Promising Charities’, while investigating the implications of this Review further.

Is deworming really effective? A response to recent deworming studies

Are mass deworming programs valuable? Giving What We Can’s position on the recent Cochrane Review.

Health benefits of treating populations with schistosomiasis are not questioned by the review

Deworm the World

Deworming and School Attendance

Next steps

Further reading: