As we enthusiastically announced in April, Giving What We Can has partnered with AidGrade, “a new organization which does meta-analyses of impact evaluations on charitable interventions.” (1) But what exactly are these meta-analyses and impact evaluations? What role can AidGrade play in improving Giving What We Can and, conversely, what can they gain from us? Miguel Almunia, one of the founding researchers of the organisation, offers some insight into the formation, work and goals of AidGrade and how their aims coincide with ours.
Founder Eva Vivalt formed AidGrade when she graduated from the University of California, Berkeley with an Economics Ph.D. in 2011. (2) Working at the World Bank, Vivalt saw the utility of an organisation that would quantitatively evaluate the effectiveness of charitable initiatives, an idea she had harbored since 2004. Finding interested parties at both her university and her work, Vivalt involved classmates and professionals alike in the project, adding, among others, Willa Friedman and Miguel Almunia to her team. AidGrade has since added members from other institutions, such as three new staff members from the Washington, D.C area. (3)
AidGrade’s approach is to take the data of numerous studies and draw comprehensive conclusions by combining their results. Almunia describes their methodology as an “agnostic approach,” meaning that the researchers are willing to look at any study that might demonstrate a correlation between programme and a particular outcome. Currently, AidGrade is putting its efforts into the first half of the 40 programmes it intends to evaluate, and has the first ten - bed nets, microfinance, etc. – already accessible on the site.
In order to give potential donors information about what is most likely to have an impact, AidGrade works to obtain access to all existing studies related to the given areas. To improve the validity of the results, the researchers have a preference for quantitative studies, although some qualitative ones can be incorporated if they were rigorously executed. They then perform a meta-analysis to combine the studies’ results. (3)
The numerous studies conducted are, of course, not identical; each has its own set of variables such as trial length or programme size. The researchers collect these kinds of variables so the papers can be divided into subsets according to what one wants to look at. But how, then, do they make sense of the data without simply totaling the number of studies that found a positive effect, or “vote counting”? Almunia offers Nate Silver, publisher of the New York Times’ FiveThirtyEight election prediction blog, as a model example. (4) Before the election, Silver did not just compile the numerous polls by tallying the number that put the Democratic or Republican Party in the lead, but instead gave each poll a weight based on its previous strength and current sample size. By doing a statistical evaluation of each poll relative to the entire body of polls, Silver – or in AidGrade’s case, the researchers - can then calculate a more accurate representation of the result and their margin of error. Although it is difficult to guide people towards the “right” target issues, AidGrade hopes to give informed advice on which interventions work within each area.
Based on this description, how is AidGrade any different from GiveWell? If their goals are both to evaluate for effectiveness, why not merge? Almunia sees the purposes of the two as distinct. Part of the motivation to set up AidGrade, he says, was that they saw the limitations in the way GiveWell operates with a seeming emphasis on inciting interest. Instead of recommending certain organisations like GiveWell does, AidGrade focuses more on general intervention types and producing rigorous data to back their results. While some may see them as competitors, Almunia believes donors view them as complementary.
Almunia recognises that a problem in academic research is that researchers do not often explain their findings well enough to the public, when he feels they should be generally accessible. He hopes AidGrade can offer an improvement on this through their website, which has a tool that allows people to analyse the data. AidGrade is also in touch with other academics, large international organisations, and non-profits that they see as complementary to their work. While Almunia “can’t say whether people are paying attention or not” to their findings yet, he notes that the “development community has demand for this kind of research” and will continue to work toward that end.
Giving What We Can and AidGrade are pleased to have formed this partnership, sharing mutual interests in helping people choose how to be most effective through investments in personal education and efforts in developing regions. Almunia is excited by the opportunity to disseminate results and influence those who want to donate. “When I told people who are interested in development, both academically and as practitioners, everyone was excited; they saw it as something that was missing and they were glad we were doing it.” (3)
References
- Hutchinson, Michelle. "Giving What We Can: April 2013." Message to the author. 1 Apr. 2013. E-mail.
- "Who We Are." AidGrade. N.p., n.d. Web. 24 Apr. 2013.
- Almunia, Miguel. Personal interview. 15 Apr. 2013.
- Silver, Nate. "FiveThirtyEight to Partner with New York Times." FiveThirtyEight: Politics Done Right. N.p., 3 June 2010. Web. 10 June 2013.
Image credit: AidGrade