Leaflets are Ineffective, Tell Your Friends

7679534638_59865527ef_oSomehow the meme got established in Effective Altruism communities that convincing people to go vegetarian or vegan is cheap and easy, and the only question is whether doing so as a substitute for reducing consumption yourself was ethical.  Me, John, Jai, and one shy friend dug deep into the research and discovered so many problems with the design of the studies showing this that we wrote them off entirely.  I wrote up a whole blog post explaining how Animal Charity Evaluator’s analysis was wrong, and leafleting was not effective.  In an act of thoroughness I would soon be very grateful for, I went to ACE’s website to make sure I was representing the studies exactly right.  Turns out ACE made essentially the same criticisms we did, and also concluded the studies were insufficient to show any net effect from leafleting.  If you go to the slate star codex link and follow the link to his sources, one has since renounced his math and the other says that his numbers are not meant so much to be “true” as to be “motivating in their concreteness”.

I did eventually find some organizations claiming leafleting was genuinely effective.  Vegan Outreach cites Farm Sanctuary, which uses the exact study Animal Charity Evaluators criticized.  ACE doesn’t go quite as far as I would: they note the 95% confidence interval of the effect and then the systemic biases of the study, whereas I would say “if you can’t get an effect size bigger than .001 in a study so egregiously biased towards your view, there is no effect.”  But the criticisms have always been there.

It’s not like anyone’s default belief is “lots of humans can be convinced to make enormous  permanent changes by one glossy 8.5×11 piece of paper,” so how this belief become established with so little data?  How did I dig into ACE’s data deeply enough to understand the design flaws myself without noticing they saw the flaws too? (partial answer: their calculator still shows leafleting having an effect ).  Unless there’s data I don’t know about, there doesn’t appear to be any support for the idea that leafleting reduces animal suffering.  We really need to figure out how this spread in a movement dedicated to quantification so we can fix the systemic issue.

EDIT: I’ve had a couple of requests to include the specific criticisms.  I originally didn’t because it felt mean to rehash ACE’s criticisms, but since the whole point is you can read their documentation without realizing them, that reasoning seems dumb now.

Everyone knows there’s a social desirability bias (reporting converting to veg*nism when you haven’t).  This is especially an issue for the people who report reducing but not eliminating animal products- it’s easy to lie about that, to yourself or others.  But the denominator (how many people received flyers total) is also unreliable, because there are a lot of reasons people who received a pamphlet will report they didn’t when asked two months later.  Maybe they threw it out without looking at it.  Maybe they read it and forgot.  Maybe they totally remember it and realize that if they say yes to the surveyor there will be a long conversation that implies they torture animals, and they would like to not do that.

We don’t have to assume this is a problem: one of ACE’s studies attempted to use a control group, and flat out couldn’t, because no one reported receiving the control flyer.  The lack of control group is a big problem, because it means you will give flyers credit for people that would have gone veg*n anyway.

Then you have to predict how long they stay veg*n.  ACE’s numbers are outlined here, and there’s several problems.  There’s social desirability bias again, and the samples are representative neither of the population at large or the population being leafleted.  I have a strong prior that people who make changes based on a flyer are less likely to stick with them than the general population

This is minor, but ACE doesn’t count the value of the leafleters time when calculating effectiveness.  Even if they’re volunteers, you need to consider the value of what else they could be doing with that time.

The Compassion Pain Scale

I never did do much with the kitten pain scale, because the pain meds’ effects were so striking there was no need for hour by hour monitoring.  But I’ve found another good marker.

When I first started at crisis chat I really really loved it, and would frequently stay hours past my scheduled shift.  I often left feeling energized*.  At some point that changed.  I put it down to a loss of novelty, or maybe nostalgia making me remember it as more fun than it was.  I kept going because it wasn’t about me having fun, it was about me helping people, but I was more conservative about the latest time I would start a new chat.

Then I got those really awesome pain meds in January, and suddenly I was staying late again.  But at some point it disappeared again.

One of the nice things about the meds is that they have a long lasting effecting.  They push the pain-tension cycle back, so I’m in less pain for days or even weeks after they wear off.  One of the bad things about one of them is that I hate being touched the next day.  That’s suboptimum on its own, but it scares me to think of what else it’s doing that I’m not noticing, so I try to go as long as possible without taking it.  I’m also very good at pushing away conscious knowledge of pain, even though it still effects me.  So I ended up going really way too long without taking the topical pain medication.

Finally I took it again, and what do you know, I stayed more than two hours past the end of my next chat shift.

It doesn’t surprise me that I’m better at chatting when I’m in less pain, but I am surprised by the way I’m better.  Ending chats is a tricky business.  You don’t want people to feel shoved out the door, but a good chunk of our target audience is having anxious ruminations.  The last two times I’ve been much better about recognizing when we’ve reached the end of the productive portion of the chat and wrapping it up.  A few people even seemed to take the nudge out as permission to relax.

*although not always.  Most days I had to call a child abuse report in were bad days

**Shift end times are a little fuzzy because of course you can’t leave in the middle of a chat.  If it’s 5 minutes before your shift ends, of course you don’t take a new chat unless you’re prepared to stay late.  But if it’s 30-45 minutes?  You’ll probably be done only a little after your stated end, but you never know which call is going to be a two hour active rescue.

Hunger Tracker 1.0

It’s out.  I made some UI tweaks and got the release build working, so you’re not including a bunch of debug code.  Here’s how I envision people using it:

1.  Install manually.  I’m working on getting it in the app store.

2. You should see something that looks like

hunger_tracker_screenshot

3.  To store your hunger:  enter the number in the upper text box and hit “store”

4.  Set an alarm for the next time you wish to track your hunger.

5.  A notification with a quiet beep and short vibrate will remind you at that time.

6.  When you want to retrieve your data, hit retrieve.

7.  When you’re done, hit clear.  This will erase all data, not just clear the screen.

Enjoy.

Xylitol for sinusitis

I’m taking on sinus inflammation because it’s a major contributor to my motion sickness, which is a major contributor to making commuting suck, and commuting is one of very few things that can actually depress your hedonic set point (psychologist talk for “make you miserable”).  My doctor has suggested xylitol nasal spray, which she claims inhibits irritation in the sinus cavities.  Quick googleing reveals it’s also considered useful for bacterial plagues on the teeth and in the arteries.  Let’s dig in.

Xylitol’s main claim to fame is as a calorie-less sweetener in humans. The claim is that it kills (many but not all strains of) bacteria via the same mechanism:  it can’t be converted into energy, so the bacteria starve to death.  This has to be to be missing a step.  Bacteria are surrounded by billions of molecules they can’t digest all the time, and they survive that.  If xylitol is to have an affect it must not only be indigestible, but inhibit digestion of actual sugar.  Off the top of my head there’s two ways that could happen.  In the human body, sugar is moved around by the blood.  If xylitol takes a sucrose molecule’s ticket to a particular area, there will be less sugar there for bacteria to eat.  The downside of this is that you might starve out your own cells.  Another option is that bacteria cells themselves become confused by xylitol.  The ideal would be if xylitol fit into a sugar receptor but couldn’t be taken into the cell, so the receptor was blocked indefinitely.  Or if it was taken in it could trigger a “yup, we got a sugar” reaction that caused the cell to take in sugar later, but I’m not sure why a bacteria would ever turn down calories.

I found a lot of studies on xylitol and dental use.  Most of what I learned is that dental abstracts are more like teasers than summaries, not cluttering up the space with numbers or sometimes even conclusions.  Overall there seems to be a mild consensus for xylitol mildly inhibiting cavities, although it’s certainly not a substitute for fluoride.  Also I totally should have been chewing xylitol while I was recovering from surgery, since is almost certainly disrupts oral plaques, although I worry about what it would do to the intestinal biome.

What about sinuses?  I found a lot of very small studies, but 5 studies of size n are not equivalent to one study of size 5n.  You don’t know how many more studies of size n were done but not published.

This study and this one found decreases in medical severity (as measured by the SNOT-20 score.  Yes that’s it’s real name), but not self-reported pain (as measured by the less well named VAS score).  This study in rabbits was well controlled (if small) and found significant decreases in bacteria.

rabbit
Rabbits self-reported pain scores were ambiguous

This study found that a nasal decongestant spray worked better than xylitol or saline spray, which worked about equally as well.  Nontheless it’s conclusion was that [name brand of xylitol spray] was an effective treatment for nasal congestion.  It also spelled spray with an ‘e’ .  Twice.

One interpretation of these results is xylitol helps impedes infection but irritates the sinuses such that there’s no change in pain levels.  Another is that people are really good at suppressing conscious knowledge of pain.  My experience has been I’m really good at suppressing moment-to-moment awareness of pain but I do notice when asked (which is how I went weeks without treating my dental neuralgia, and then suddenly noticed I was at 8 on the pain scale), and that the pain has a great deal of effect on my behavior and happiness whether I acknowledge it or not.   And if I keep using xylitol I need to change my brand to one that, when it buys positive press in a supposedly objective forum, spells its own name correctly.

More ways to make to the world better

Seattle Effective Altruists just got a bunch of new potential members from the very different spheres of “Peter Singer’s talk” and “the Harry Potter and the Methods of Rationality wrap party”.  I am planning an event targeted at these new people, with the dual goals of “make them slightly more effective even if they never come back” and “induce them to learn and do more about effective altruism”.  The specific plan is to present actions one can do right now to make the world a better place.  “Donating to GiveWell” will be there, but so will donating blood.  I am looking for things that run the full spectrum of cheap/expensive, difficult/easy, time consuming/not, so that everyone has something that is doable for them.  That especially means not having too many things that run on money, because not everyone has a lot of money to spare and we already spend a great deal of time on options for those that do.  Any other suggestions for what to include?  The list so far is:

  • Have you considered just giving money?
  • Use Amazon smile and a Charity Science referral link to buy from Amazon
  • Hold a birthday fundraiser
  • Donate blood, sign organ donor card
  • Reduce animal consumption/buy humane animal products.

I’ll add more as suggestions come in.

Autism as Developmental Injury

Left untreated, people with phenylketonuria (PKU) can develop intellectual disabilities, seizures, and “other medical problems”.  But PKU does not cause any of those.  Phenylketonuria + a normal diet causes a build of of phenylalinine in the body, which causes those problems.  If PKU is caught at birth and the sufferer is kept on a phenylanlinine-light diet, they will never develop these problems.

Henry Markram suggests that something analogous is going on with autism.  He and his collaborators think that the actual problem is that autistic babies have extraordinary sensory sensitivity, and this sensitivity causes defenses that cause them to miss certain critical information during developmental periods.  What is challenging but achievable (the zone necessary for learning) for other people is overwhelming for them, so they don’t learn.  The developmental window closes and they’ve lost their chance to truly master that skill.  But if they were given stimulus in their zone of achievable challenge, they would learn those skills and maintain them for life.  They might continue to need accommodations, the way phenylketonurics need to stay on a phenylalinine-light diet their whole life, but with those accommodations they could function “normally”.  This is known as the intense world hypothesis.

The example they give is the critical period for learning language.  You *can* learn a new language after the critical period, but it will never be as easy, most people will never attain genuine fluency, and if you never learn any language it may be truly impossible to pick one up later.  If normal human speech is overwhelming to an autistic infant they will miss that period and their language will be impaired for life.  But if they’re given regular access to speech they are comfortable with (probably quieter and slower) they could learn it just fine, the same way hearing impaired children do fine with sign language.

I was also really impressed with the writing of this lay-press article.  I’ve been avoiding doing take downs, especially of popsci articles, because there are millions of wrong things every day and criticizing them is easy.  For a while I could justify them as case studies in critical reading, but now it just feels bad.  This had led to a lot of aborted blog entries, as I read something amazing and then realize it’s too flawed to pass on uncritically.   I don’t agree with everything the article says (insisting there’s only one cause of autism strikes me less as brave and more as idiotic), but it lays out its case in an informative and responsible manner.

Hunger Tracker 0.3

I had this great essay written in my head about how good testing was great when you were learning, because you can’t copy paste tests like you can product code.  Then it turned out that it was impossible to test notifications.  Not difficult, literally impossible. The ability to test directly has been deliberately blocked for security reasons, and you can’t mock it because the relevant class doesn’t have a default constructor.  The most direct path I see right now is extending EasyMock to work on classes without default constructors, and I assume if that were easy someone would have done it already.

So here’s Hunger Tracker version 0.3, sans Aesop.  It has all the features I promised in 1.0, but there’s some UX work I want to do before calling it done.

Review: The Remedy: Robert Koch, Arthur Conan Doyle, and the Quest to Cure Tuberculosis (Thomas Goetz)

I love this book so much I gave it to my cats to cuddle, which would have made a more impressive visual if I hadn't gotten the kindle version.
I love this book so much I gave it to my cats to cuddle, which would have made a more impressive visual if I hadn’t gotten the kindle version.

I don’t even know where to start.  This book was fun to read and I felt like I learned a lot.  It covered both the specific facts of Robert Koch’s quest to prove germ theory and cure tuberculosis, and provided a good general sense of how science and medicine move forward and don’t.

A couple of specifically interesting points: doctors fought germ theory tooth and nail.  They also rejected stethoscopes as technological interlopers to be disposed of because they threatened the doctors importance, while using so many leaches prosperous countries had to import them.  The naive interpretation is “doctors are idiots, their reluctant to use quantified self data is proof they haven’t changed.”  This is the first time I’ve seen any hint as to why they found germ theory so implausible.  In the particular case of tuberculosis, everyone was exposed all the time, and it took the infection years to become symptomatic.  Preventing any one exposure wouldn’t have had noticeable results.  Another early-identified bacteria was Anthrax, which didn’t follow a typical exposure pattern either.  The doctors still come out looking pretty bad for refusing to wash their hands between autopsies and childbirth, but marginally less than they might have.

I knew this already, but it was good to have a reminder that the first person to suggest the germ theory of disease, Ignez Summelweis, died in an insane asylum.  This either means that people with truly visionary ideas can be broken when we reject them, or germ theory was so crazy it took a crazy person to see it.  Goetz doesn’t mention it but according to my dad Summelweis was also an asshole, which I try to remind myself every time someone mean says something I disagree with.

Remember last week when I suggested using microchips to force people to finish their antibiotics?  Several friends seriously questioned the effect of that, since they didn’t estimate the contribution of unfinished antibiotics to antibiotic resistance as very high.  The Remedy says that the current protocol for drug resistant TB is to have a medic visit a patient every day for 6 -24 months to observe them taking their pills, because drug resistant TB is that big a problem and the pills are that unpleasant.  So at least in that situation swallowable microchips would be an enormous improvement.

Apparently syphilis is always the [nationality] disease, where the nationality is not the speaker’s.  French is the most popular, but far from the only

I’ve always found the methods section the most boring of any paper or textbook.  I want to know what we learned, not how.  But The Remedy (and to a lesser extent Neanderthal Man, which I reviewed last week) made it seem interesting.  I’m still not terribly interested in microscopy, but it was deeply interesting to see how advances in technology enabled scientific advances.  Using or inventing new technology is how you move the world forward.  And when I thought about it, the modern field that most reminds me of the wide-open-ness of microbiology in the mid 1800s is programming.  That is where I get the most sense of possibility.  I still really care about translational health (in fact this book taught me that that is the word for what I am trying to do with this blog) and mental health, but I am feeling more and more like staying in programming would be the best way to accomplish that.

What We Talk About When We Talk About Effectiveness

The biological/scientific definitions of heredity and heritable differ slightly from the popular usage.  Lay people tend use it  to mean “how much is this caused by genes?”  In science, heredity is how closely people resemble their parents, divided by the total variation in the population.  Biological sex has almost zero heritability because knowing someone’s parents sex does not allow you to predict their own sex.*  Number of arms is barely heritable, because there’s almost no variation in number of arms among humans, and what variation exists is overwhelmingly caused by environment, not genetics.

A corollary to this is that a measure of heredity is only valid for the exact environment you measured it in.  If you plant a variety of seeds in identical pots and give them identical water and supplements, most variation will be due to genetics, and a small amount to chance (which will be counted as environment), so traits like height and time to flower will be highly heritable.  If you plant those same seeds in widely varying pots and vary the water and nutrients they get, a lot of the variation will be due to environment, and the heredity values of the same traits will be much lower.  Skin color in Norway is more heritable in the winter than in the summer, because teenagers deliberately tan more than their parents.

I have struggled before to make effectiveness estimates when the intervention’s usefulness depends on multiple factors.  Blood for car accident victims is only helpful in the context of emergency rooms and medical schools and sterile gauze.  Suicide hotlines require phones and electricity and suicidal people at a bare minimum, and active rescues require police and mental hospitals and often pharmaceutical research.  I think I’m just going to have to put effectiveness in the same category as heredity: the quantification is only valid for the environment in which it is measured.

I’ve worried before about Effective Altruism’s tendency to take the existing system as a given.  That was a reasonable simplification when the movement was first starting, and there was plenty of low hanging fruit that didn’t require more sophisticated analysis.  But I’m really happy to see organizations like the Open Philanthropy Project branch into studying how to change systems and how to measure the effectiveness of attempts to do so.

*Intersexuality confounds this a little but my impression is it’s mostly not a genetic issue, in part because intersex people generally have difficulty reproducing.

The Science of Blood Types

Yesterday I made an offhand remark that the Red Cross was so persistent in stalking me because I was O-.  Here’s why that’s important:

Your red blood cells can have two different proteins on them (A or B), or both, or neither.  If you receive a transfusion with red blood cells with a protein you don’t have, your immune system will mark it as a foreign invader and attack it.  If you get too much of these cells, you will die.  Parallel to this is another protein Rh factor, which you either have (Rh+) or you don’t (Rh-).  A- means you have A proteins, but not B or Rh factor.  If you are Rh- and receive Rh+ red blood cells, same problem.

foxindogs
You cannot bluff the immune sytem.

O- red blood cells have no immune-triggering proteins on it, and so can be given to anyone.  That makes O- blood extremely useful when you don’t have time to type a patient, or have limited carrying capacity.  Moreover, O- red blood cells are the only type O- patients can accept.  So O- is often the first to run out.

You might think this makes donations from AB+ people pointless, but that is not the case.  Their red blood cells are of limited use, but their blood plasma (which is the part that contains the antigens that trigger attacks on foreign matter) contains no antigens to any blood type.  Meanwhile everything looks like an invader to my O- plasma, and it will react accordingly.  This also means you can only do whole blood donations between complete type matches.  A B+ person can take my red blood cells, but would need plasma from someone else.

And that is why typed-matched blood is preferred and donations of all types are necessary.