Cash benchmarking

December 26, 2023

This is a background post on cash benchmarking — there’s nothing novel here. The next post contains my thoughts on long-term benchmarking. Cash transfers are intrinsically good — giving people in extreme poverty will make them richer. They also help us understand what works better or worse in development economics. The key phrase here is “cash benchmarking”: comparing an economic development project to a cash transfer program, transferring the per capita cost of the development project you’re implementing.

There’s a direct analogy to the standard of care in medical research. A common approach in clinical trials is to give the control group the standard treatment for their condition, while the treatment group receives the experimental treatment. Cash benchmarking treats cash transfers as the standard treatment for poverty alleviation, against which all other treatments should be tested.

This is rare in practice, and I think that’s understandable. People and organizations run aid projects because they think they will work; academics evaluate those projects hoping to find positive effects. The worst case is pretty demoralizing: imagine putting all this effort into your great project, and finding out that everyone would have been better off with something far simpler.

It’s rare, but it’s been done. First, I’m going to discuss the spectrum of cost-effectiveness with cash transfers as the numeraire. Then I’ll present the few projects which have used a cash benchmark: some of these projects saw cash outperform standard development interventions; some had mixed evidence. In the next post, I’ll discuss why we should expect the differential effects of cash to diminish over time.

The spectrum of cost-effectiveness

There are two uninteresting cases for benchmarking: in the first, where your project is more effective than cash, just don’t do cash, and pat yourself on the back for finding a clever intervention. In the second, where cash would be more effective, we just shouldn’t do the development project and give the recipients cash.Or, better yet, go do something from the first category.

Projects certainly more effective than cash

Consider two categories: public goods and information asymmetries. I claim projects which act in these categories will be more effective than cash.

Providing public goods is a collective action problem. Sending cost-equivalent cash to all of the individuals who would benefit from the public good means the public good will be underprovided. Take spatial frictions — how difficult it is to move goods or people from one place to another. Consider two interventions: improving an existing dirt road connecting ten villages to the city, and giving the equivalent per capita cash to the residents of the ten villages. We’d have reason to believe the first would be more effective for improving lives.

First, most people don’t use the road. So while everyone will benefit from the reduction in potholes and flooding,Through easier access to goods, falling prices. most people will not spend the money they receive on road improvements. So only people whose livelihood depends on “reducing spatial frictions” will spend money on this — and on a personal level, that looks more like buying a motorcycle than building a road.How much better the road is than cash depends on the population size and the length of the road. Also, how bad is the road — there are diminishing marginal returns to road quality.

Information asymmetry is a fancy way economists say “I know more than you.” This is the origin of many traditional development projects, right or wrong. When wrong, these projects are at best a waste of money and at worst actively harmful. When right, providing goods and information that recipients don’t know they need can lead to large health and welfare gains. This is why it’s better to give bednets or deworming pills than to give the equivalent cash.

Finally, these occasionally overlap. When most of a population sleeps under bednets, this disrupts the lifecycle of the Plasmodium parasite, making bednets their own sort of public good.

We haven’t yet seen a benchmarking study where the project being compared to cash just crushes cash. But this makes sense — those projects which are trivially better than cash (and I can think of a few) won’t be chosen for benchmarking studies.

Projects certainly less effective than cash

Projects based on flawed assumptions or executed poorly will perform poorly.OLPC is a fun example; for a flip side of malaria, see the TVA’s efforts in the 30s and 40s.

Jeremy Shapiro, a co-founder of GiveDirectly, explored an alternative: preferenced-based benchmarking. He asked recipients their valuation in cash for common aid and development programs, and compared their values to a) the per capita cost and b) the forecasted value of development professionals. After eliciting these preferences, he finds “both recipients and development professionals value public good interventions more than interventions with spillovers, and interventions with spillovers more than private goods.”

While the recipients and professionals disagree in some areas, both place lower-than-cost values on privately provided solar power, tutoring, and family planning services. This mismatch in preferences results in a bundle of interventions, selected by development professionals, which are substantially less valuable to recipients than the bundle they would choose.

Projects we might want to test against cash

These are the two uninteresting cases. But benchmarking is still useful; there are projects in between these two! USAID has some of these projects. In 2018, they invited GiveDirectly to run concurrent cash transfer programs for six of their projects; Or maybe just five? I’ve only been able to find five in the first round, but they said six. the second round of three more benchmarking projects is underway now, and USAID is organizing some cash benchmarked projects independently. What do these studies find?

Cash-benchmarking studies

In 2018, USAID and GiveDirectly partnered to evaluate some of USAID’s projects against a GiveDirectly cash benchmark. So far, we only have results from two of these studies. Of the five USAID/GiveDirectly benchmark studies:

  1. • In Rwanda, cash was benchmarked against a workforce training program
  2. • Also in Rwanda, cash was benchmarked against a child nutrition bundle‘superior information, direct transfer of productive assets, and improvements in household diet and sanitation’
  3. • In the DRC, cash was benchmarked against a project consisting of “support and training activities to help youth create Savings and Internal Lending Communities, develop entrepreneurial knowledge, and start income-generating activities”
  4. • Two projects, in Liberia and Malawi, did not run a “a head-to-head comparison with a particular USAID program,” but only “define[d] the “benchmark” of what cash can deliver, against which traditional programming can be measured”

So this is what cash is being compared to. Such programs can be generally characterized as an investment in human capital: in the case of workforce training, this is direct; in nutrition, this is less obvious.

I discuss the results from these studies in the next post.

Some other relevant studies

Not surprisingly, GiveDirectly has pioneered cash benchmarking — and the only other cost-equivalent cash benchmarking study comes from one of their founders, Jeremy Shapiro. In this paper, he found that cash transfers equal to the cost of common agricultural development projects have the same impact as those projects. This is also a short-term — six-month — result.

While those are the only results we have comparing aid projects against traditional development projects, there are others which are comparable.

Cash benchmarking - December 26, 2023 - Joseph Levine