“Even in what seem like “unquantifiable” areas like political change and disaster prevention, we can still think rigorously, in an evidence-based manner, about how good those activities are. We just need to assess the chances of success and how good success will be if it happens. This, of course, is very difficult to do, but we will make better decisions if we at least try to make these assessments rather than simply throwing up our hands and randomly choosing an activity to pursue, or worse, not choosing any altruistic activity at all.”
William MacAskill, Doing Good Better
During the last Effective Giving Mini Masters, Robert commented that the value their foundation could’ve created for a better world would have been much greater if they had given almost all of Jazi Foundation’s grant money from 2013-2015 to the Against Malaria Foundation (AMF).
This would’ve also saved him a substantial amount of time. He could have spent that time enjoying precious moments with his family, as opposed to visiting Ghana’s slums, or assisting social entrepreneurs and persuasive staff from charitable organisations.
Don’t get him wrong! The value created by the Jazi Foundation in those years isn’t ‘bad’.
The point Robert was trying to convey is that new information about the effectiveness of their previous grants, and a lack of knowledge about amazingly effective alternative opportunities, had resulted in suboptimal effectiveness achieved with their limited resources.
This insight – where Robert compares the value they have created against the benchmark of an independently researched Top Charity – drives the Jazi Foundation to grow its effectiveness every day, and strive to keep on doing better than its chosen benchmark.
The Power of Benchmarks
Even the most privileged and generous people amongst us have limited resources. Therefore, when deciding how to grant/invest our money, we always have to choose between different options. By using a relevant benchmark that you can compare (albeit often in an inevitably imperfect way) your options to, you can create a more disciplined approach of making choices.
You can start with comparing your current donations to your benchmark. This may already provide you with great insights and lessons. Then, you can use your benchmark to build your portfolio. In theory, your benchmark can be the option to which you’d grant all your available donations if nothing else can ‘beat it’:
You can see this as a ‘reversal test’ where you ask yourself: “If the organisation or project already had an extra $100,000, would I take it away from them and give it to, say, GiveDirectly?” and “Would I take away money from GiveDirectly on the margin (their last $100,000 for cash transfers) to give it to another opportunity?” (from Hauke Hillebrandt).
Consequently, you can attempt to find options that are able to beat the benchmark, and if they do, invest in them.
Selecting a Benchmark
In our experience, it is helpful to pick an organisation/program as a benchmark where we can be very certain how they score on these two factors.
AMF’s bednets are a great example of a helpful benchmark. We know with fairly high certainty both the chance of success of AMF and how good success will be if it happens.
The chance of success of AMF is fairly high. AMF currently has a funding gap of hundreds of millions of dollars, and they have been rigorously evaluated by the independent charity evaluator GiveWell. As a result, we can be pretty certain that every additional Euro will go to funding a bednet that AMF will successfully distribute, and will achieve the desired result.
We can base our estimate of the success AMF will achieve when it distributes bednets on independent rigorous research. This research shows that for every $100,000, AMF will provide about 15.726 children under the age of 5 with protection from malaria via a bednet, and thereby prevent the death of about 28.7 lives of these children. t’s important to note that ‘success’ here is something we define ourselves – and surely does not capture the entire value that is created (and perhaps also destroyed) by AMF. More on this thorny issue in the detailed example below.
Our benchmark may be clear, but our other options may not be
Even if our benchmark is clear, the options we want to compare against our benchmark often aren’t.
When we want to compare these options to our benchmark, we need to make a guesstimate of their respective value (this can be done with so called ‘expected value analysis’). In doing so, we often face numerous challenges:
- We don’t have rigorous information about the probability of success
- We are not that sure how good that success will be if it occurs
These guesstimates are especially difficult when no evidence on the causal link between an organisation and their impact exists, or when the organisation we are considering is relatively new, and hence does not have a long track record to demonstrate their competence.
At Effective Giving we offer an educational program, our Mini Masters, that can help you learn more about benchmarking; how to select a benchmark that is most relevant your unique resources, how to find great options to compare against your benchmark and how to make those options comparable against your benchmark.
We are working on enriching our list of benchmarks in the giving and impact investing space that are relevant for many of us.
We attempt to cover the priority cause areas that are presenting a great threat to humanity as we know it, are relatively neglected, and where solutions already exist or are likely to be created. Selecting a relevant benchmark that can help you heighten your effectiveness is something our Head of Research, Vera Schölmerich (firstname.lastname@example.org) can help you with.
As probabilities and values are difficult to estimate, some have argued that in practice we need to be cautious about taking these estimates literally (Karnofsky 2016). They’re estimations, and we’re very likely to suffer from a strong Bayesian prior.
Despite this caution, we’ve found making guesstimates of the options at hand and comparing them to a clear benchmark extremely valuable. It forces us to make all of our assumptions explicit and provides us with more clarity in our decision making. In practice, the difference in expected value between our options has often been so wide that we felt less concerned with the uncertainty of the actual estimates included: the option most likely to create most value was very clear.