Monthly Archives: September 2015

Mountains Beyond Mountains (Tracy Kidder) and the case against cost effectiveness

Mountains Beyond Mountains is a biography of Paul Farmer, an American doctor who founded a small clinic in Haiti and ended up saving hundreds of thousands of lives, possibly millions.  But that’s at the end.  The beginning is spent with him  doing obviously suboptimal things like spending $5000/year per patient treating AIDS patients in a country where people were constantly dying of malnutrition and diarrhea (average cost to treat: <$30), while baiting me by bragging about how cost ineffective it was.  I was very angry at him for a while, until it dawned on me that getting angry at a man for distributing lifesaving drugs probably said more negative things about me than it did about him.  Plus he did maybe avert a worldwide epidemic of untreatable tuberculosis, so perhaps I should stop yelling at the CD player and figure out what his thinking was.

Let’s take tuberculosis.  At some point in the story (it’s frustratingly vague on years), Farmer’s friend brings him to Peru, which had what was widely considered the world’s best TB containment system (called DOTS).  The WHO held it up as what the rest of the world should aspire to.  DOTS did many things right, like ensuring a continuous supply of antibiotics to patients and monitoring them to ensure compliance (intermittent treatment breeds resistance).    On the other hand, the prescribed response to someone failing to get better on drugs (indicating their infection was resistant to them) was “give them all the same, plus one more.”  This is exactly the right thing to do, if what you want is to make sure the bacteria evolve resistance to the new drug without losing its existing resistance.  The protocol specifically banned testing to see which drugs a particular patient’s infection was susceptible to, because that is expensive and potentially difficult in the 3rd world.

The WHO ignored the possibility of drug resistant TB because it was considered an evolutionary dead end.  Resistance had to evolve anew in each patient, and rendered the infection noncontagious.  I don’t know what evidence they based this on, but  at best it was a case of incorrectly valuing evidence over reason.  At worst, it indicates a giant sentient TB cell has infiltrated the WHO and is writing policy.

Hello fellow humans. I sure do hate death and suffering.
Hello fellow humans. I sure do hate death and suffering.

If your evidence says a contagious disease spontaneously becomes completely noncontagious at a convenient moment, your first thought should be “who do I fire?” because obviously something went wrong in the study design or implementation. If you check everything out and it genuinely is noncontagious, your next thought should be “wow, we really lucked out having all this extra time to prepare before it inevitably becomes contagious again.”  At no point should it be “sweet, cross that off the list”, because while you are not looking it will redevelop virulence and everyone will die.

Farmer’s response to the ban on treating multiple drug resistant TB was to steal >$100,000 worth of medicine and tests from an American hospital to treat a handful of patients.  When caught, a donor bailed him out.

Or take cancer.  A young child with weird symptoms shows up at his clinic.  He drops a few thousand dollars on tests to determine the child had a rare form of cancer.  60-70% survival rate in the US, no chance in Haiti.  He convinced an American hospital to donate the care, but when the child becomes too ill to travel commercially he spends $20,000+ on medical transport.

Both of these went against standard measures of cost effectiveness, as did Farmer’s pioneering work treating AIDS patients in the 3rd world.  But let’s look at his results:

  1. The WHO’s “yeah, it’s probably fine” approach to drug resistant TB worked as well as you would think.  Farmer proves it is contagious, is treatable, and drives down the price of treatment (to the point it is $/DALY competitive with standard health interventions). He goes on to change the international standard for TB treatment and lead the effort in several countries. Book gives no numbers but I estimate 142,000 lives and counting, plus avoiding an epidemic that could cost 2 million lives/year.*
  2. Kid’s cancer is inoperable, he dies in the US.  American hospital agrees to take a few more cases each year.

In retrospect his actions in the TB case seem pretty damn effective.  But he didn’t set out to change the world.  He stole those drugs for the exact reason he flew that boy to the US: someone was dying in front of him and it made him sad.  It’s possible you couldn’t correct his answer in the cancer case without also “correcting” his answer in the TB case. And while someone more math driven could have launched the world changing anti-MDR TB campaign, they didn’t.  Farmer did, and we need to respect that.

Lots of people in the philanthropic space, including Farmer and most EAs, say that it’s unreasonable to expect perfect altruism from everyone.  People need to spend money on themselves to keep themselves happy and productive, and constant bean counting about “do the morale effects of name brand toilet paper make up for the kids I won’t be able to deworm?” is counterproductive.   You put aside money for charity, and you put aside money for you to enjoy life, and you make your choices.  What if we view Farmer’s need to save the life in front of him as a morale booster that enables our preferred work (averting world wide incurable TB pandemic), rather than the work itself?  By that measure,  $150,000 on a single kid’s cancer and 7 hours doing a house call for one patient in Haiti is a steal.  Given that I pay my cats more (in food and vet care) than what 1/6 of the world survives on, I do not have a lot of room to judge Paul Farmer’s “saving children from cancer” hobby.

Farmer himself raises this point, in a way.  It turns out that effective altruism did not invent the phrase “that’s not cost effective.”  Lots of people with a lot of power (e.g. the WHO) have been saying it for a long time.  From Farmer’s perspective, it seems to be used a lot more to justify not spending money, rather than spending it on a different thing.  He also rejects the framing of the comparison: cancer treatment may save fewer lives per dollar than diarrhea treatment but it saves way more per dollar than a doctor’s beach house, so how come it only gets compared to the former?  Those are fair points.

It’s not clear he could have redirected the money even if he wanted to.  Most of the care for the cancer patient was donated in kind; there was no cash he could redirect to a better cause (although that’s not true for the cost of the medivac).  No one gave him $100,000 to spend on TB treatment, he stole drugs and got bailed out.  It’s not clear the donor would have been as moved to rescue him if he stole $100,000 worth of cheap antibiotics.

In essence, I’m postulating that Farmer operated under the following constraints.

  • Evaluating cost effectiveness is emotionally costly even in the face of very good information.
  • Low quality information on effectiveness
  • Financial discipline was emotionally costly
  • Some money was available for treatments of less relative effectiveness but could not be moved to the most effective thing.  But the money was not clearly labeled “the best thing” and “for warm fuzzies only”, he had to guess in the face of low information.

Under those things, evaluating cost effectiveness could easily be actively harmful.  Judging by the results, I think he did better following his heart.

Doing The Most Effective Thing is great, and I think the EA movement is pushing the status quo in the right direction. But what Farmer is doing is working and I don’t want to mess with it.  At the same time, his statement that “saying you shouldn’t treat one person for cancer because you could treat 10 people for dysentery is valuing one life over another.” (paraphrased) is dangerously close to Heifer International’s “we can’t check how our programs compare to others, that would be experimenting on them and that would be wrong.” (paraphrased), which is dangerously close to Play Pump’s “fuck it, this seems right.” (paraphrased) as they rip out functional water pumps and replace them with junk.  So while Farmer is a strong argument against Effective Altruism as “the last social movement we will ever need” (because some people do the most good when they don’t compare what they’re doing to the counter-factuals), he’s not an argument against EA’s existence.  Someone has to run the numbers and shame Play Pump until they stop attacking Africans’ access to water.

It doesn't even make sense as a toy. The fun part of carousels is the coasting, not the wind up
It doesn’t even make sense as a toy. The fun part of carousels is the coasting, not the wind up

And just like you couldn’t improve Farmer by forcing to him do accounting, you can’t necessarily improve a given EA by making them sadder at the tragedy immediately in front of them.  EA is full of people who didn’t care about philanthropy until it had math and charts attached, or who find doing The Best Thing motivating.  We’re doing good work too. I understand why people fear doing the math on human suffering will make them less human, but that isn’t my experience.  I cry more now at heroism and sacrifice than I ever did as a child.

Ironically the one thing I was still angry at Farmer for at the end of the book was his most effective choice: neglecting his children in favor of treating patients and global health politics.  I could forgive it if he felt called to an emergency after the children were born, but he had kids knowing he would choose his work over them.  For me, no amount of lives saved can redeem that choice.  Maybe that is what he feels about letting that Haitian child die of cancer.

*This paper (PDF) estimates 142,000 deaths averted 2006-2015 by the program Farmer pioneered, and the program is still scaling up.    I estimate a drug resistant TB epidemic would cost a minimum 2.3 million lives/year (math below), although how likely that is to occur is a matter of opinion.  That’s ignoring his clinical work in Haiti, the long term effects of his pioneering HIV treatment in the 3rd world,the long term effects of his pioneering community-based interventions that increased treatment effectiveness,  infrastructure building in multiple countries, and refugee care.  I would love to give you numbers for those but neither Paul Farmer nor PIH believe in numbers, so the WHO evaluation of the TB program was the best I could do.

MATH

Untreated TB has a mortality rate of ~70%l

TB rates have been dropping since ~2002, but that’s due to aggressive
treatment. New infection rate held steady at ~150 people/100,000
from 1990-2002, so let’s use that as our baseline. With 7.3 billion people, that’s 11 million cases/year. 11 million * .7 chance of death = 7.7 million deaths. Per year. 25% of those are patients with AIDS who arguably wouldn’t live very long anyway, so conservatively we have ~5.7 million deaths. If I’m really generous and assume complete worldwide distribution of the TB vaccine (efficacy: 60%), that’s down to 2.3 million deaths. Per year.

For comparison, malaria causes about 0.5 million deaths/year.

Thanks to Julia Wise for recommending the book, Jai Dhyani and Ben Hoffman for comments on earlier versions, and Beth Crane for grammar patrol.

EA Book Roundup

In the span of a few months, three intro-to-effective-altruism books came out- Peter Singer’s The Most Good You Can Do, Nick Cooney’s How to be Great at Doing Good, and Will MacAskill’s Doing Good Better.  After reading all three, my preference order is: How to be Great…Doing Good BetterThe Most Good…, with fairly large gaps between the three.  I actively disliked reading The Most Good….  On the other hand, if you measured the impact each person has had on moving money to effective charities, the order is most certainly Peter Singer >> Will MacAskill > Nick Cooney.*  This leads me to believe my preferences are not wildly predictive and shouldn’t be given much weight.   But I think I understand why I like How to be Great… best, and that information might be useful.

In a nutshell: Peter Singer tells you what to do, Nick Cooney teaches you a way to think about the problem.**  In particular: I hate the argument from a drowning child.  It works by evoking a primitive instinct that is almost impossible to translate to rigorous analysis completely separate from the part of my brain that plans effective giving.   The same instinct that tells you to ruin your suit to save a child will tell you to skip a one time opportunity to earn billions of dollars for charity to save a child.  We like saving children.  Meanwhile, Nick Cooney’s book is about building up the skills so you can look at tough choices without blinking.  To me, this came off as simultaneously less judgmental of my faults and yet a stronger push for me to do better.

[I suspect high scrupulousness plays a role here- I constantly felt like Singer was saying I was a bad person because I wasn’t doing the most goodest thing, and it made me want to think about giving less.]

As EA-the-movement grows, I think it’s going to have to have very different inward and outward faces.  On one extreme you have the very core- doing or planning on doing direct work, the kind of hardcore earning to give that depends on social support, organizing the movement, or spreading ideas outside it.   On the other extreme you have people who are going to donate a modest amount to charity and are choosing based mostly on warm fuzzies, but are influenced by EA to donate more, or more effectively.  I picture this group as slightly less rigorous than my parents, who I have convinced to donate to several EA charities, and who have done good research on charities themselves, but also still give money to my dad’s extremely well endowed alma matter.  How to be Great… is for preparing people to be core.  The Most Good… is for motivating people on the very fringe.  I would place Doing Good Better between them, but much closer to The Most Good… If one of my friends wanted to know about EA, I’d give them Doing Good Better, on the theory that my friends are a lot like me.  If a stranger wanted to know more I’d probably point them to The Most Good…, based on empirical results (unless they were highly conscientious).  If you enjoy my blog I suspect you’re a How to be Great type- even if you don’t want to drink the EA koolaid, you probably like being taught more than being ordered.

*Singer had major influences on Bill Gates and Dustin Moskovitz (funder of GoodVentures) and after he’s done that there’s no point even counting the other people he’s influenced.  Will MacAskill cofounded 80,000 hours and Giving What We Can.  Nick Cooney runs some relatively small budget animal charities.

** If I had to pick a description for MacAskill’s approach it would be “descriptive”, but it doesn’t contrast with the other books as much as they do with themselves.   Peter Singer also spends a lot of time describing the effective altruism movement but I found a lot of it to be wrong.