Key Concepts for Assessing Charities

This page contains an introduction to some of the concepts which need to be borne in mind when evaluating charities, in order to ensure that assessments are as complete and accurate as possible.

Future events and probability

When assessing a charity, what matters is the overall impact that further donations will make. Once you know what a program is meant to achieve, there are also risks which need to be taken into account: Corruption, misappropriation of resources and simple administrative error, to name a few. We need a way to reflect the uncertainty created by these risks in our assessment.

We can use an expected value calculation to do this. Consider the example of charities A and B, both of which are running new campaigns:

• If charity A’s campaign is successful it will do 20 Quality adjusted Life years (QALYs) of good for every £1,000 donated. Although the campaign is new, it is based on past successes and is well researched. We assess the chances of its success at 90%.
• If charity B’s campaign is successful it will do 30 QALYs of good for every $1,000 donated. The campaign is ambitious, but there are a number of risks which could cause it to fail. We assess the chance of success at 50%.

To compare these campaigns, we multiply the potential good done by the probability of it happening to get our final figures:

• Charity A: [20 × 90% =] 18 QALYs per $1,000 donated.
• Charity B: [30 × 50% =] 15 QALYs per $1,000 donated.

As a result, we can see that at this point charity A is the better prospect, even though it’s campaign is less ambitious. Of course, if charity B is successful with its campaign, then the chances of it continuing to be successful will rise, and the calculation will change.

Regression to the mean

Since we are looking for the most effective charities, we are naturally drawn to the ones which have had a high degree of recent success. However, we have to be cautious about this, especially if the success has only been for a relatively short period of time.

The reason to be careful is that there will always be a random element to success. In any given time period, different charities will have better or worse luck. If we simply donated to the charity with the best recent figures, it could well be that they have just had a lucky few years. If so, over the next few years they are likely to be less successful and “regress to the mean”.

For this reason, it is important to look for long-term success and frequently re-evaluate programs to check that they are still as effective as originally believed.

Measurement and error

The difficulties with measuring effectiveness mean that there might be errors in the data used to assess charities. Again, this creates the risk that an effective-seeming charity is actually just one where the measurement errors came out in its favour. Unlike the problem discussed above, these errors may well be the sort to re-occur year after year.

In order to combat this, it is important to work out the extent of these measurement errors. This allows you in effect to find the ‘margin of error’ around your results. If the margin is particularly wide and the difference in effectiveness between different charities is particularly small, then there is a risk that the apparent differences are nothing more than measurement errors.

Fortunately, this is not the case for our recommended charities. They appear significantly more effective than the other charities we have considered, and although there is a margin of error, it is relatively small. As a result, we can be reasonably sure that they really are more effective, and we can recommend them with confidence.