Seeing Like A State, Flashlights, and Giving This Year

Note (7/15/19): I’m no longer sure about Tostan as an organization. I would like to give more details on my current thinking, but they are hard to articulate and it seemed better to put up this disclaimer now than wait for my thinking to solidify.

Overview: The central premise of Seeing Like A State (James C. Scott, 1999) is that the larger an organization is, the less it can tolerate variation between parts of itself.  The subparts must become legible.  This has an extraordinary number of implications for modern life, but I would like to discuss the application to charity in particular.  I believe Tostan is pushing forward the art and science of helping people with problems that are not amenable to traditional RCTs, and recommend donating to them.  But before you do that, I recommend picking a day and a time to consider all of your options.

Legibility is easier to explain with examples, so let’s start with a few: 

• 100 small farmers can learn their land intimately and optimize their planting and harvest down to the day, using crop varieties that do especially well for their soil or replenish nutrients it’s particularly poor in.  Large agribusinesses plant the same thing over thousands of acres on a tight schedule, making up the difference in chemical fertilizer and lowered expectations.
• The endless mess of our judicial system, where mandatory sentencing ignores the facts of the case and ruins people’s lives, but judicial discretion seems to disproportionately ruin poor + minority lives.  
• Nation-states want people to have last names and fixed addresses for ease of taxation, and will sometimes force the issue.
• Money.  This is the whole point of money.

Legibility means it’s not enough to be good, you must be reliably, predictably good.*

I want to be clear that legibility and interchangeability aren’t bad.  For example, standardized industrial production of medications allows the FDA to evaluate studies more cleanly, and to guarantee the dosage and purity of your medication.  On the other hand, my pain medication comes in two doses, “still hurts some” and “side effects unacceptable”, and splitting the pills is dangerous.  

Let’s look at how this applies to altruism.  GiveWell’s claim to fame is demanding extremely rigorous evidence to make highly quantitative estimates of effectiveness. I believe they have done good work on this, if only because it is so easy to do harm that simply checking you’re having a positive effect is an improvement.  But rigor will tend to push you towards legibility.   

• The more legible something is, the easier it is to prove its effectiveness.  Antibiotics are easy.  Long term dietary interventions are hard.
• Legible things scale better/scaling imposes legibility.  There’s a long history of interventions with stunning pilots that fail to replicate.  This has a lot of possible explanations:
  • Survivorship bias
  • People who do pilots are a different set than people who do follow up implementations, and have a verve that isn’t captured by any procedure you can write down.
  • A brand new thing is more likely to be meeting an underserved need than a follow up.  Especially when most evidence is in the form of randomized control trials, where we implicitly treat the control group as the “do nothing group”.  There are moral and practical limits to our ability to enforce that, and the end result being that members of the “control group” for one study may be receiving many different interventions from other NGOs.  This is extremely useful if you are answering questions like “Would this particular Indian village benefit from another microfinance institution?”, but of uncertain value for “would this Tanzanian village that has no microfinance benefit from a microfinance institution?”
  • For more on this see Chris Blattman’s post on evaluating ideas, not programs, and James Heckman on Econtalk describing the limits of RCTs.

GiveWell is not necessarily doing the wrong thing here.  When you have $8b+ to distribute and staff time is your most limited resource, focusing on the things that do the most good per unit staff time is correct.

Meanwhile, I have a friend who volunteers at a charity that helps homeless families reestablish themselves in the rental market. This organization is not going to scale, at all. Families are identified individually, and while there are guidelines for choosing who to assist there’s a lot that’s not captured, and a worse social worker would produce worse results.  Their fundraising is really not going to scale; it’s incredibly labor intensive and done mostly within their synagogue, meaning it is drawing on a pool of communal good will with very limited room for expansion.

Theoretically, my friend might make a bigger difference stuffing envelopes for AMF than they do at this homelessness charity.  But they’re not going to stuff envelopes for AMF because that would be miserable.  They could work more at their job and donate the money, but even assuming a way to translate marginal time into more money, work is not necessarily overflowing with opportunities to express their special talents either.

Charities do not exist to give volunteers opportunities to sparkle.  But the human desire to autonomously do something one is good at is a resource that should not be wasted. It can turn uncompetitive uses of money into competitive ones.  It’s also a breeding ground for innovation.  GiveDirectly has done fantastically with very deliberate and efficient RCTs, but there are other kinds of interventions that are not as amenable to them.

One example is Medecins Sans Frontiers.  Leaving half of all Ebola outbreaks untreated in order to gather better data is not going to happen.  Even if it was, MSF is not practicing a single intervention, they’re making hundreds of choices every day.  85% of American clinical trials fail to retain “enough” patients to produce a meaningful result, and those are single interventions on a group that isn’t experiencing a simultaneous cholera epidemic and civil war.  MSF is simply not going to get as clean data as GiveDirectly.

This is more speculative, but I feel like the most legible interventions are using something up.  Charity Science: Health is producing very promising results with  SMS vaccine reminders in India, but that’s because the system already had some built in capacity to use that intervention (a ~working telephone infrastructure, a populace with phones, government health infrastructure, medical research that identified a vaccine, vaccine manufacture infrastructure… are you noticing a theme here?).  This is good.  This is extremely good.  Having that capacity and not using it was killing people.  But I don’t think that CS’s intervention style will create much new capacity.  For that you need inefficient, messy, special snowflake organizations.  This is weird because I also believe in iterative improvement much more than I believe in transformation and it seems like those should be opposed strategies, but on a gut level they feel aligned to me.

Coming at this from another angle: The printing press took centuries to show a macroeconomic impact of any kind (not just print or information related).  The mechanical loom had a strong and immediate impact on the economy, because the economy was already set up to take advantage of it.  And yet the printing press was the more important invention, because it eventually enabled so much more.  

I know of one charity that I am confident is building capacity: Tostan.  Tostan provides a three year alternative educational series to rural villages in West Africa.  The first 8 months are almost entirely about helping people articulate their dreams.  What do they want for their children? For their community?  Then there is some health stuff, and then two years teaching participants the skills they need to run a business (literacy, numeracy, cell phone usage, etc), while helping them think through what is in line with their values.

Until recently Tostan had very little formal data collection.  So why am I so confident they’re doing good work?  Well, for one, the Gates Foundation gave them a grant to measure the work and initial results are very promising, but before that there were other signs.

First, villages ask Tostan to come to them, and there is a waitlist.  Villages do receive seed money to start a business in their second year, but 6-9 hours of class/week + the cost of hosting their facilitator is kind of a long game. 

Second, Tostan has had a few very large successes in areas with almost no competitors.  In particular; female circumcision.  Tostan originally didn’t plan on touching the concept, because the history of western intervention in the subject is… poor.  It’s toxic and it erodes relationships between beneficiaries and the NGOS trying to help them, because people do not like being told that their cherished cultural tradition, which is necessary for their daughters to be accepted by the community and get good things in their life, is mutilating them, and western NGOs have a hard time discussing genital cutting as anything else.  But Tostan taught health, including things that touched on culture.  E.g. “If your baby’s head looks like this she is dehydrated and needs water with sugar and salt.  Even if they have diarrhea I know it seems weird to pump water into a baby that can’t keep it in, but this is what works.  Witch doctors are very good at what they do, but please save them for witch doctor problems.”  

And one day, someone asked about genital cutting.

[One of Tostan’s innovations is using the neutral term “female genital cutting”, as opposed to circumcision, which many people find to be minimizing, and mutilation, which others find inflammatory]

It’s obvious to us that cutting off a girl’s labia or clitoris with an unsterilized blade, and (depending on the culture) sewing them shut is going to have negative health consequences.  But if everyone in your village does it, you don’t have anything to compare it to.  Industrial Europeans accepted childbed fever as just a thing that happened despite having much more available counterevidence.*  So when Tostan answered their questions honestly- that it could lead to death or permanent pain at the time, and greatly increases the chances of further damage during childbirth- it was news.

The mothers who cut their daughters were not bad people.   If you didn’t know the costs, cutting was a loving decision.  But once these women knew, they couldn’t keep doing it, and they organized a press conference to say so.  To be clear, this was aided by Tostan but driven by the women themselves.

The press conference went… poorly.  A village deciding not to cut was better than a single mother deciding not to cut, but it wasn’t enough.  Intermarriage between villages was common and the village as a whole suffered reprisal.  In despair Tostan’s founder, Molly Melching, talked to Demba Diawara, a respected imam.  He explained the cultural issues to her, and that the only way end cutting was for many villages to end it at the same time.  So Tostan began helping women to organize mass refusals, and it worked.  So far almost 8000 villages in West Africa have declared an end to genital cutting, of which ~2000 come from villages that directly participated in Tostan classes (77% of villages that practice cutting that took part in Tostan), and ~6000 are villages adopted by the first set.

Coincidentally, at the same time Melching was testing this, Gerry Mackie, a graduate student, was researching footbinding in China and discovered it ended the exact same way; coordinated mass pledges to stop.  

This is not conclusive.  Maybe it’s luck that Melching’s method consistently ended female genital cutting where everyone else had failed, in a method that subsequently received historical validation.  But I believe in following lucky generals.

FGC is not the only issue Tostan believes it improves.  It believes it facilitates systemic change across the board, leading to better treatment of children, more independence for women, cleaner villages, and more economic prosperity.  But it doesn’t do every thing in every village, because each village’s needs are different, and because what they provide is responsive to what the community asks for.  So now you’re measuring 100 different axes, some of which take a long time to generate statistically significant data on (e.g. child marriage) some of which are intrinsically difficult to measure (women’s independence), and you can’t say ahead of time which axes you expect to change in a particular sample.  This is hard to measure, and not because Tostan is bad at measuring.  

That’s not to say they aren’t trying.  Thanks to a grant from the Gates Foundation, Tostan has begun before and after surveys to measure its effect.  In addition to the difficulties I mentioned above, it faces technical challenges, language issues, and the difficulty of getting honest answers about sensitive questions.  

There is a fallacy called the streetlight fallacy; it refers to looking for your keys under the lamppost, where there is light, rather than in the dark alley where you lost your keys.  The altruism equivalent is doing things that are legible, instead of following the need.  This is not categorically wrong- when it’s easy to do harm, it is correct to stay in areas where you’ll at least know if it happened.  But staying in the streetlight forever means leaving billions of people to suffer.

I believe Tostan is inventing flashlights so we can hunt for our keys in the woods.  It is hard, and it is harder to prove its effectiveness.  But ultimately it leads to the best outcomes for the world.  I am urging people to donate to Tostan for several reasons:

1. To support a program that is object level doing a lot of good
2. To support the development of flashlight technology that will help others do more good.
3. To demonstrate to the warmest, fuzziest, most culturally respecting of charities that incorporating hard data will get them more support, not less.

The traditional thing to do right now to encourage you to donate would be a matching pledge.  But more than I want money moved to Tostan, I want a culture of thoughtful giving, and charity-specific matching erodes that*.  Probably its best feature is that it can overcome inertia, but it does that regardless of charity quality.  So instead, let me encourage you to put time on your calendar to decide how much and where you will donate.  Seriously, right now.  If you can’t choose a time, choose a time to choose a time.  For those with company matching and tax concerns, this is noticeably more useful if it happens before Christmas.

If you are feeling extra motivated consider hosting a donation decision day or giving game.  If you would like to publicize your event, contact me at elizabeth @ this domain and I will post it here and to any contacts I have in your city.  

I also encourage you to write up your thought process regardless of the outcome, including not donating, and including thought patterns that are very different from my own or from established orthodoxy.  For some examples, see my posts in 2014 and 2015.  I will write up a separate post with every one of these someone sends me, assuming I’m sent any at all, which is not guaranteed.

The other prosocial purpose of matching challenges is to demonstrate how important you think an organization is by spending your own money.  I am going to skip the middle man and announce my contribution now: $19,750, plus $19,750 in company matching*, for a total of $39,500  This is everything I plan on donating between now and the end of 2017.

*I have a theory that much of the misery of modern jobs is from a need to make your work legible to others, which by necessity means doing things that are expected of the position, even if you’re bad at or dislike them, and shaving off the bits that you are especially good at and other people aren’t.  You may not even be allowed to do the things you are best at, and if you are the rewards are muted because no one is in a position to notice and reward the success.  This is pretty much a recipe for making yourself miserable.  It made me miserable at a large programming house famous for treating its employees wonderfully.  I think that company’s reputation is overblown as an absolute measure, but is probably still fair on a relative one, so I can only imagine how awful working in fast food is.  This does not actually have a lot to do with the point of this essay and will probably be cut in the version that goes on Tostan’s blog, but it was too interesting not to include.

*Postpartum infections were common in births attended by a physician because washing your hands between an autopsy and a birth was considered peasant superstition.  Midwives, who followed the superstition, had a lower death rate.  This discovery languished in part because the doctor who discovered it was an asshole and no one wanted to listen to him, and that’s why I don’t allow myself to dismiss ideas from people just because I don’t like them.    

*Charity-neutral matching, like that done by many employers, mostly doesn’t, although I worry it does anchor people’s charity budgets.

*If you are wondering why the number is weird: I donated $250 to a giving game earlier this year.

Relationship disclosures:  

Tostan’s Director of Philanthropy, Suzanne Bowles, has provided assistance on this post, in the form of answering questions about Tostan and reviewing this document (although she did not have veto power).  Suzanne and I have a friendly relationship and she has made some professional introductions for me.

I have several close friends who work or have worked for GiveWell, some of whom provided comments on this essay.  

Thanks to Justis Mills for copy editing and Ben Hoffman for feedback on earlier drafts.