Mountains Beyond Mountains (Tracy Kidder) and the case against cost effectiveness

Mountains Beyond Mountains is a biography of Paul Farmer, an American doctor who founded a small clinic in Haiti and ended up saving hundreds of thousands of lives, possibly millions.  But that’s at the end.  The beginning is spent with him  doing obviously suboptimal things like spending $5000/year per patient treating AIDS patients in a country where people were constantly dying of malnutrition and diarrhea (average cost to treat: <$30), while baiting me by bragging about how cost ineffective it was.  I was very angry at him for a while, until it dawned on me that getting angry at a man for distributing lifesaving drugs probably said more negative things about me than it did about him.  Plus he did maybe avert a worldwide epidemic of untreatable tuberculosis, so perhaps I should stop yelling at the CD player and figure out what his thinking was.

Let’s take tuberculosis.  At some point in the story (it’s frustratingly vague on years), Farmer’s friend brings him to Peru, which had what was widely considered the world’s best TB containment system (called DOTS).  The WHO held it up as what the rest of the world should aspire to.  DOTS did many things right, like ensuring a continuous supply of antibiotics to patients and monitoring them to ensure compliance (intermittent treatment breeds resistance).    On the other hand, the prescribed response to someone failing to get better on drugs (indicating their infection was resistant to them) was “give them all the same, plus one more.”  This is exactly the right thing to do, if what you want is to make sure the bacteria evolve resistance to the new drug without losing its existing resistance.  The protocol specifically banned testing to see which drugs a particular patient’s infection was susceptible to, because that is expensive and potentially difficult in the 3rd world.

The WHO ignored the possibility of drug resistant TB because it was considered an evolutionary dead end.  Resistance had to evolve anew in each patient, and rendered the infection noncontagious.  I don’t know what evidence they based this on, but  at best it was a case of incorrectly valuing evidence over reason.  At worst, it indicates a giant sentient TB cell has infiltrated the WHO and is writing policy.

Hello fellow humans. I sure do hate death and suffering.
Hello fellow humans. I sure do hate death and suffering.

If your evidence says a contagious disease spontaneously becomes completely noncontagious at a convenient moment, your first thought should be “who do I fire?” because obviously something went wrong in the study design or implementation. If you check everything out and it genuinely is noncontagious, your next thought should be “wow, we really lucked out having all this extra time to prepare before it inevitably becomes contagious again.”  At no point should it be “sweet, cross that off the list”, because while you are not looking it will redevelop virulence and everyone will die.

Farmer’s response to the ban on treating multiple drug resistant TB was to steal >$100,000 worth of medicine and tests from an American hospital to treat a handful of patients.  When caught, a donor bailed him out.

Or take cancer.  A young child with weird symptoms shows up at his clinic.  He drops a few thousand dollars on tests to determine the child had a rare form of cancer.  60-70% survival rate in the US, no chance in Haiti.  He convinced an American hospital to donate the care, but when the child becomes too ill to travel commercially he spends $20,000+ on medical transport.

Both of these went against standard measures of cost effectiveness, as did Farmer’s pioneering work treating AIDS patients in the 3rd world.  But let’s look at his results:

  1. The WHO’s “yeah, it’s probably fine” approach to drug resistant TB worked as well as you would think.  Farmer proves it is contagious, is treatable, and drives down the price of treatment (to the point it is $/DALY competitive with standard health interventions). He goes on to change the international standard for TB treatment and lead the effort in several countries. Book gives no numbers but I estimate 142,000 lives and counting, plus avoiding an epidemic that could cost 2 million lives/year.*
  2. Kid’s cancer is inoperable, he dies in the US.  American hospital agrees to take a few more cases each year.

In retrospect his actions in the TB case seem pretty damn effective.  But he didn’t set out to change the world.  He stole those drugs for the exact reason he flew that boy to the US: someone was dying in front of him and it made him sad.  It’s possible you couldn’t correct his answer in the cancer case without also “correcting” his answer in the TB case. And while someone more math driven could have launched the world changing anti-MDR TB campaign, they didn’t.  Farmer did, and we need to respect that.

Lots of people in the philanthropic space, including Farmer and most EAs, say that it’s unreasonable to expect perfect altruism from everyone.  People need to spend money on themselves to keep themselves happy and productive, and constant bean counting about “do the morale effects of name brand toilet paper make up for the kids I won’t be able to deworm?” is counterproductive.   You put aside money for charity, and you put aside money for you to enjoy life, and you make your choices.  What if we view Farmer’s need to save the life in front of him as a morale booster that enables our preferred work (averting world wide incurable TB pandemic), rather than the work itself?  By that measure,  $150,000 on a single kid’s cancer and 7 hours doing a house call for one patient in Haiti is a steal.  Given that I pay my cats more (in food and vet care) than what 1/6 of the world survives on, I do not have a lot of room to judge Paul Farmer’s “saving children from cancer” hobby.

Farmer himself raises this point, in a way.  It turns out that effective altruism did not invent the phrase “that’s not cost effective.”  Lots of people with a lot of power (e.g. the WHO) have been saying it for a long time.  From Farmer’s perspective, it seems to be used a lot more to justify not spending money, rather than spending it on a different thing.  He also rejects the framing of the comparison: cancer treatment may save fewer lives per dollar than diarrhea treatment but it saves way more per dollar than a doctor’s beach house, so how come it only gets compared to the former?  Those are fair points.

It’s not clear he could have redirected the money even if he wanted to.  Most of the care for the cancer patient was donated in kind; there was no cash he could redirect to a better cause (although that’s not true for the cost of the medivac).  No one gave him $100,000 to spend on TB treatment, he stole drugs and got bailed out.  It’s not clear the donor would have been as moved to rescue him if he stole $100,000 worth of cheap antibiotics.

In essence, I’m postulating that Farmer operated under the following constraints.

  • Evaluating cost effectiveness is emotionally costly even in the face of very good information.
  • Low quality information on effectiveness
  • Financial discipline was emotionally costly
  • Some money was available for treatments of less relative effectiveness but could not be moved to the most effective thing.  But the money was not clearly labeled “the best thing” and “for warm fuzzies only”, he had to guess in the face of low information.

Under those things, evaluating cost effectiveness could easily be actively harmful.  Judging by the results, I think he did better following his heart.

Doing The Most Effective Thing is great, and I think the EA movement is pushing the status quo in the right direction. But what Farmer is doing is working and I don’t want to mess with it.  At the same time, his statement that “saying you shouldn’t treat one person for cancer because you could treat 10 people for dysentery is valuing one life over another.” (paraphrased) is dangerously close to Heifer International’s “we can’t check how our programs compare to others, that would be experimenting on them and that would be wrong.” (paraphrased), which is dangerously close to Play Pump’s “fuck it, this seems right.” (paraphrased) as they rip out functional water pumps and replace them with junk.  So while Farmer is a strong argument against Effective Altruism as “the last social movement we will ever need” (because some people do the most good when they don’t compare what they’re doing to the counter-factuals), he’s not an argument against EA’s existence.  Someone has to run the numbers and shame Play Pump until they stop attacking Africans’ access to water.

It doesn't even make sense as a toy. The fun part of carousels is the coasting, not the wind up
It doesn’t even make sense as a toy. The fun part of carousels is the coasting, not the wind up

And just like you couldn’t improve Farmer by forcing to him do accounting, you can’t necessarily improve a given EA by making them sadder at the tragedy immediately in front of them.  EA is full of people who didn’t care about philanthropy until it had math and charts attached, or who find doing The Best Thing motivating.  We’re doing good work too. I understand why people fear doing the math on human suffering will make them less human, but that isn’t my experience.  I cry more now at heroism and sacrifice than I ever did as a child.

Ironically the one thing I was still angry at Farmer for at the end of the book was his most effective choice: neglecting his children in favor of treating patients and global health politics.  I could forgive it if he felt called to an emergency after the children were born, but he had kids knowing he would choose his work over them.  For me, no amount of lives saved can redeem that choice.  Maybe that is what he feels about letting that Haitian child die of cancer.

*This paper (PDF) estimates 142,000 deaths averted 2006-2015 by the program Farmer pioneered, and the program is still scaling up.    I estimate a drug resistant TB epidemic would cost a minimum 2.3 million lives/year (math below), although how likely that is to occur is a matter of opinion.  That’s ignoring his clinical work in Haiti, the long term effects of his pioneering HIV treatment in the 3rd world,the long term effects of his pioneering community-based interventions that increased treatment effectiveness,  infrastructure building in multiple countries, and refugee care.  I would love to give you numbers for those but neither Paul Farmer nor PIH believe in numbers, so the WHO evaluation of the TB program was the best I could do.

MATH

Untreated TB has a mortality rate of ~70%l

TB rates have been dropping since ~2002, but that’s due to aggressive
treatment. New infection rate held steady at ~150 people/100,000
from 1990-2002, so let’s use that as our baseline. With 7.3 billion people, that’s 11 million cases/year. 11 million * .7 chance of death = 7.7 million deaths. Per year. 25% of those are patients with AIDS who arguably wouldn’t live very long anyway, so conservatively we have ~5.7 million deaths. If I’m really generous and assume complete worldwide distribution of the TB vaccine (efficacy: 60%), that’s down to 2.3 million deaths. Per year.

For comparison, malaria causes about 0.5 million deaths/year.

Thanks to Julia Wise for recommending the book, Jai Dhyani and Ben Hoffman for comments on earlier versions, and Beth Crane for grammar patrol.

2 thoughts on “Mountains Beyond Mountains (Tracy Kidder) and the case against cost effectiveness”

  1. It’s possible you couldn’t correct his answer in the cancer case without also “correcting” his answer in the TB case.

    My problem with this is that it feels like talking about the lottery.

    Imagine that you see someone coasting by in a limo with a pool filled with banknotes. “How’d he get that?” “Oh, he played the lottery”

    Does this imply that the lottery is the right choice if you want to get rich? It obviously works, almost every week we can see someone for whom it worked.

    Yet in reality the expected return is negative. For every person we see coasting past in a limo after a lottery win there’s millions who are simply a little bit poorer.

    EA’s don’t have any kind of monopoly on achieving high impact interventions. If a hundred thousand non-EA people just try to do what “feels right” some small portion are going to have a high impact even if their choices were close to random.

    Pointing to one person who was extremely effective through a tactic which, while laudable, didn’t have good odds of being effective is like pointing to the lottery winner. Yes he won in this case but the approach isn’t reliable.

    Indeed it’s highly likely that in future the handful of most spectacular cases are still going to be non-EA’s in the same way that it’s rare to see hedge fund managers winning the lottery or making 100 million dollars from a 2 dollar investment.

    The hedge fund managers keep making smaller reliable investments making billions in the long term using far larger pools of money but their stories are boring.

  2. Agree with the above comment.

    Also, I agree that investing in our own mental health so that we’re effective in the long run is a good idea. But, if we’re consistent, we should be looking at the most effective ways to invest in our own effectiveness! Everybody needs an occasional vacation, but going camping in a nearby park is probably better from an EA perspective than a trip to Europe.

    Similarly, there are probably better investments in one’s emotional well-being than what Farmer spent on a single sick child.

Comments are closed.