Why and how Giving What We Can evaluates evaluators

A major focus of Giving What We Can’s research team is evaluating impact-focused charity evaluators and grantmakers — this page explains what this work comprises, why this is our focus, and how we do it.

We want to make the best possible recommendations we’re aware of to donors looking to maximise their impact. To be able to serve donors with a variety of values and starting assumptions, and to cover as many causes and charities as possible, we rely on third-party expert evaluators for our recommendations and grantmaking, rather than evaluating individual charities ourselves. We evaluate evaluators and their methodologies so that we can always rely on the highest-quality and most up-to-date recommendations available across a range of causes.

On this page, you can find the evaluators we currently rely on, the evaluators we’ve looked into so far, key results and context from each iteration of our evaluators research, and background and context about the evaluators project as a whole.

Note that we do not recommend using sites like Charity Navigator or GuideStar to determine where your donations will have the greatest impact, since comparing impact is not the primary focus of these organisations. (More here.)

Evaluators we’ve evaluated whose research we currently rely on

Based on the results of our 2023 and 2024 evaluators research, we currently rely on the following evaluators/grantmaking programs (in alphabetical order) for our charity recommendations and to advise our cause area funds:1

All evaluator reports

We’ve so far lookoed into eight impact-focused evaluator programs. You can read each report below:

Evaluator research by year

This section links to more specifics about each iteration of our evaluate the evaluators research, including the headline findings of our research, the changes that were made to our charity recommendations and funds as a result, and background/context on the project – including how we chose the specific evaluators we looked into that year.

An overview of our evaluators research

This section explains the what and why behind the evaluate the evaluators project more generally and applies to all iterations of the work.

What does it mean to evaluate evaluators?

We evaluate evaluators to decide which evaluators:

At a high level, we do this by:

  • Forming an understanding of the strengths and weaknesses of a respective evaluator and its approach to charity evaluation and grantmaking.
  • Deciding whether and how we defer to the evaluator (ideally in direct comparison with at least one other impact-focused evaluator working in the same cause area).
  • Transparently explaining and justifying that decision to donors.

Why evaluate evaluators?

We think it’s the best way we can help donors maximise their impact.

We don’t have the capacity to evaluate individual charities ourselves — there are far too many for just one research team to cover, more than a million in the US alone! — so we need to turn to other expert evaluators and grantmakers focused on impact. By our count, there are now over a dozen impact-focused grantmaking and charity evaluation organisations, some of which provide different charity recommendations in the same cause area. This leaves us, other effective giving organisations, and donors with an important choice on whose recommendations to follow.

Before 2023, we made this choice based on factors like the public reputation of an evaluator in the effective giving ecosystem, and whether its stated approach seemed to broadly align with our donors’ goals. But we wanted to do better, and thought it would be valuable to provide donors with more information about evaluators.

Beyond making our recommendations to donors, we think there are several extra benefits to evaluating evaluators:

  • As above, we’re not the only effective giving organisation that relies on evaluators for their recommendations. We think our work can also help others make more informed decisions about which evaluators to defer to.
  • We think we can provide value to the evaluators themselves, by providing them with independent feedback on their work.
  • These kinds of evaluations can improve the incentives for existing and new evaluators to deliver the best work they can.

There are substantial limitations to our first iterations of this project, which we did in 2023 and 2024, but we nevertheless think that this is a significant improvement on the status quo, in which there were no independent evaluations of evaluators’ work. We discussed some of our concerns with this status quo when we first announced our research direction at the end of 2022.

How we evaluate evaluators

In this section, we highlight:

  • Some of the principles we use to guide our evaluations
  • How we choose which evaluators to evaluate
  • The limitations of our approach

Our principles

As we do with our impact evaluation, we aim for usefulness, transparency, and justifiability, rather than comprehensiveness and procedural uniformity. Put another way, we aim to transparently communicate how we use our judgement to find the areas we think are most useful to investigate, to come to a justifiable decision on whether and how to defer to an evaluator. Some implications of this approach include that we are flexible in what we choose to investigate (making each evaluation different) and open to stopping an evaluation once we feel able to make a justifiable decision.

We also aim to avoid surprises for evaluators by alerting them of our thinking throughout the process. This is in part because we want to work with evaluators to understand and improve their approach, rather than just judging them, and also because we value their expertise.

How we choose which evaluators to evaluate

Please see our yearly reports for the specifics of how we chose the evaluators we looked into for each iteration of our project.

More generally, because of our worldview-diverse approach, we’ve based our research on the goal of maintaining impact-focused charity and fund recommendations in the three cause areas2 we think contain some of the most cost-effective funding opportunities across a broad range of plausible worldviews (rather than taking a view on how impact varies across these cause areas). As a research team, we think we can add most value within a cause area, whereas donors can decide for themselves which cause areas best align with their worldview.

Our choice of these three cause areas (global health and wellbeing, animal welfare, and reducing global catastrophic risk) has been informed by global priorities research from organisations like Open Philanthropy, the Global Priorities Institute, and (in the past) the Centre for Effective Altruism.

There are some promising philanthropic cause areas that we did not (yet) include (such as climate change). We intend to keep evaluating new cause areas and evaluators to add further recommendations, provided we find a strong enough case exists that, from a sufficiently plausible worldview, a donor would choose to support those cause areas over other options.

Additionally, prior to beginning our evaluator research, we:

  • Conducted a survey among 16 effective giving organisations (made up primarily of the larger national fundraising organisations listed here), on which evaluations would be most useful for them.
  • Investigated where our donors give — we wanted to prioritise evaluators whose funds and charity recommendations our donors were currently supporting the most, so that the results would be most useful to them.
  • Considered our previous selection of “trusted evaluators” — we wanted to prioritise evaluators whose research had informed our previous recommendations.
  • Wanted to be able to compare evaluators in the same cause area, which meant we began with two evaluators in each cause area.

These considerations heavily influenced our choice of evaluators in 2023, which in turn has influenced future iterations of the project.

Limitations of our approach

A key limitation of our approach to evaluator research is that our charity and fund recommendations may not (yet) include some donation opportunities that are, in fact, competitive with the ones we recommend. This is because we haven’t (yet) investigated all promising evaluator programs in the impact-evaluation space, and only recommend opportunities from evaluators we’ve vetted. Relatedly, we haven’t (yet) investigated several promising cause areas, such as climate change or effective giving meta organisations.

Notably, this means we’re highly confident that our charity and fund recommendations are based on the best research we’ve evaluated within each cause area. However, it also means we’re not highly confident that opportunities we’ve excluded are necessarily less impactful. We’ve tried to account for this by explaining how we came up with our recommendations on our best charities page, adding a recommendations FAQ that goes into further detail, highlighting that there are a wider range of promising programs available to donors via our donation platform, and providing resources for donors to investigate these further.

On a similar note, even after several iterations, this project will not allow us to cover the entire charity space: there aren’t evaluators in every promising cause area, and evaluators are generally far from able to cover all promising charities within the cause areas they focus on – they use prioritisation frameworks and reasoning to help ensure they’re evaluating the most promising ones.

Nevertheless, we think this project is the most efficient way for us to cover as large a part of the high-impact charity space as possible in a high-quality manner, and – as stated above – we provide donors with various promising alternative options if they want to have more choice and delve deeper themselves.

Finally, the quality of our recommendations is highly dependent on the quality of the charity evaluation field in a cause area, and hence inconsistent across cause areas. For example, the state of charity evaluation in animal welfare is less advanced than that in global health and wellbeing, so our evaluations and the resulting recommendations in animal welfare are necessarily lower-confidence than those in global health and wellbeing.

A few other potential limitations (most of these are also noted in the relevant reports):

  • We generally did not have extensive in-house domain expertise in the evaluation areas we looked into. We tried to partially account for this by asking external reviewers for input, and by keeping most of our evaluation focused on evaluation and grantmaking aspects where only limited subject-specific expertise was needed.
  • In many cases, we had conflicts of interest that we had to navigate. We don’t think these fundamentally challenged our position to do the evaluations — and generally think some conflicts of interest are nearly unavoidable in the still very small charity evaluation space — but we do think they matter and that it is important to be transparent about them.
  • Many evaluators were not set up to be externally evaluated, and we didn’t always have full access to all the relevant information.
  • We were highly time-constrained.

Given these limitations, we aimed to:

  • Focus our efforts where they would be most useful, by prioritising considerations that were relevant to our recommendation decisions.
  • Only make decisions we thought we could justify, and highlight our uncertainties where we didn’t feel confident enough to make a decision.
  • Be as transparent as possible in our reasoning in each report, without breaching any confidentiality agreements we made and limited by our time constraints.

Even with our efforts to take an approach that prioritises transparency, justifiability, and usefulness, we appreciate there still are significant limitations to our evaluations, and see the first few iterations of this project as a minimum-viable-product version which we look forward to improving on in future iterations. However, as mentioned above, we think doing these evaluations represents a significant improvement to the previous situation, in which there were no independent evaluations of evaluators’ work we (or donors and other effective giving organisations) could rely on.

Bottom line

Rather than evaluating individual charities, since 2023, Giving What We Can evaluates which third-party expert evaluators donors can best rely on to maximise their impact. This allows us to make even higher-quality fund and charity recommendations to donors with a wide variety of values and starting assumptions. We think this represents a big improvement over how we previously chose which evaluators to work with — based on rough heuristics— even though it still has limitations. It has also facilitated the launch of our cause area funds, which present a reliable default option for donors who want their money to be allocated according to our latest research.

Over time, we want to expand to more cause areas and evaluators, go more in-depth where it's useful, and keep refining our process based on feedback. Most importantly, we'll keep focusing on empowering donors and collaborating with evaluators to help donors have the biggest impact. We're grateful to all the evaluators who worked with us on this project so far, and look forward to continuing to improve together.

For those who would like to see our current giving recommendations, check out our best charities page. For the full selection of programs Giving What We Can supports, see our donation platform.