Luck Based Medicine: No Good Very Bad Winter Cured My Hypothyroidism

I’ve previously written about Luck Based Medicine: the idea that, having exhausted all the reasonable cures for some issue, you are better off just trying shit rather than trying to reason more cures into existence. I share LBM success stories primarily as propaganda for the concept: the chance any one cure works for anyone else is <10% (sometimes much less), but a culture where people try things and share their results.

I’ve also previously written about my Very Unlucky Winter. My mattress developed mold, and in the course of three months I had four distinct respiratory infections, to devastating effect. A year later I am still working my way through side effects like asthma and foot pain. 

But, uh, I also appear to have cured my hypothyroidism, and the best hypothesis as to why is all the povidone iodine I gargled for all those respiratory infections illnesses.

Usually when I discuss fringe medicine I like to say “anything with a real effect can hurt you”, because it’s a nice catchall for potential danger. In this case, I can be more direct: anything that cures hypothyroidism has a risk of causing hyperthyroidism. The symptoms for this start with “very annoying” and end at “permanent disability or death”, so if you’re going to try iodine, it absolutely needs to be under medical supervision with regular testing. 

All that said…

I was first diagnosed with hypothyroidism 15 years ago, and 10 years ago tried titrating off medication but was forced back on. My thyroid numbers were in the range where mainstream MDs would think about treating and every ND, NP, or integrative MD would treat immediately. 

Low iodine can contribute to hypothyroidism, and my serum iodine tested at low normal for years, so we had of course tried supplementing iodine via pills, repeatedly, to no result. No change in thyroid and no change in serum iodine levels.

In January of the Very Unlucky Winter, I caught covid. I take covid hard under the best of circumstances and was still suffering aftereffects from RSV the previous month, so I was quite scared. Reddit suggested gargling povidone iodine and after irresponsibly little research, I tried it. My irresponsibility paid off in that the covid case was short and didn’t reach my lungs. I stopped taking iodine when I recovered but between all the illnesses, potential illnesses, and prophylactic use I ened up using it for quite a long period.

My memories of this time are very fuzzy and there were a lot of things going on, but the important bits are: I developed terrible insomnia, hand tremors, and temperature regulation issues. These had multiple potential explanations, but one of them was hyperthyroidism so my doctor had me tested. Sure enough, I had healed my thyroid to the point my once-necessary medication was giving me hyperthyroidism. 

Over the next few months I continued gargling with iodine and titrating my medication down. After ~6 months I was off it entirely. I’ve since been retested twice (6 weeks and 20 weeks after ceasing medication) and it looks like I’m clean. 

Could this have been caused by something besides iodine? I suppose, and I was on a fantastic number of pills, but I can’t figure out what else it could be. Hypothyroidism has a very short list of curable underlying causes, and none of them are treated by anything I was taking. 

So why did gargling iodine work when pills didn’t? It could be the formulation, but given my digestive system’s deepseated issues, I’m suspicious that the key was letting the iodine be absorbed through the mucous membrane of the throat, rather than attempting to through the gut. If that’s true, maybe I can work around my other unresponsive vitamin deficiencies by using sublingual multivitamins. I started them in June and am waiting to take the relevant test.

Thank you to my Patreon patrons for their support of this work. 

There is a $500 bounty for reporting errors that cause me to change my beliefs, and an at-my-discretion bounty for smaller errors. 

(Salt) Water Gargling as an Antiviral

Summary

Over the past year I’ve investigated potential interventions against respiratory illnesses. Previous results include “Enovid nasal spray is promising but understudied”, “Povidone iodine is promising but understudied” and “Humming will solve all your problems no wait it’s useless”. Two of the iodine papers showed salt water doing as well or almost as well as iodine. I assume salt water has lower side effects, so that seemed like a promising thing to check. I still believe that, but that’s about all I believe, because papers studying gargling salt water (without nasal irrigation) are few and far between. 

I ended up finding only one new paper I thought valuable that wasn’t already included in my original review of iodine, and it focused on tap water, not salt water. It found a 30% drop in illness when gargling increased in frequency from 1 time per day to 3.6 times, which is fantastic. But having so few relevant papers with such small sample sizes has a little alarm going off in my head screaming publication BIAS publication BIAS. So this is going in the books as another intervention that is promising but understudied, with no larger conclusions drawn. 

Papers

Estimating salivary carriage of severe acute respiratory syndrome coronavirus 2 in nonsymptomatic people and efficacy of mouthrinse in reducing viral load: A randomized controlled trial

Note that despite the title, they only gave mouthwashes to participants with symptoms.

This study had 40 participants collect saliva, rinse their mouth with one of four mouthwashes, and then collect more saliva 15 and 45 minutes later . Researchers then compared compared the viral load in the initial collection with the viral load 15 and 45 minutes later. The overall effect was very strong: 3 of the washes had a 90% total reduction in viral load, and the loser of the bunch (chlorhexidine) still had a 70% reduction (error bars fairly large). So taken at face value, salt water was at least as good as the antiseptic washes. 

(Normal saline is 0.9% salt by weight, or roughly 0.1 teaspoons salt per 4 tablespoons water)

[ETA 11/19: an earlier version of this post incorrectly stated 1 teaspon per 4 tablespoons. Thank you anonymous]

This graph is a little confusing: both the blue and green bars represent a reduction in viral load relative to the initial collection. Taken at face value, this means chlorhexidine lost ground between minutes 15 and 45, peroxide and saline did all their work in 15 minutes, and iodine took longer to reach its full effect.  However, all had a fairly large effect.

My guess is this is an overestimate of the true impact, because I expect an oral rinse to have a greater effect on virons in saliva than in cells (where the cell membrane protects them from many dangers). Saline may also inflate its impact by breaking down dead RNA that was detectable via PCR but never dangerous. 

The short-term effect of different chlorhexidine forms versus povidone iodine mouth rinse in minimizing the oral SARS-CoV-2 viral load: An open label randomized controlled clinical trial study

This study had a fairly similar experimental set up to the previous: 12 people per group tried one of three mouth washes, or a lozenge. Participants collected saliva samples immediately before and after the treatments, and researchers compared (a proxy for) viral loads between them.

Well, kind of. The previous study calculated the actual viral load and compared before and after. This study calculated the number of PCR cycles they needed to run before reaching detectable levels of covid in the sample. This value is known as cycle threshold, or Ct. It is negatively correlated with viral load (a smaller load means you need more cycles before it becomes detectable), but the relationship is not straightforward. It depends on the specific virus, the machine set up, and the existing cycle count. So you can count on a higher Ct count representing an improvement, but a change of 4 is not necessarily twice as good as a change of 2, and a change from 30->35 is not necessarily the same as a change from  20->25. The graph below doesn’t preclude them doing that, but doesn’t prove they did so either. My statistician (hi Dad) says they confirmed a normal distribution of differences in means before the analysis, which is somewhat comforting. 

This study found a significant effect for iodine and chlorhexidine lozenges, but not saline or chlorhexidine mouthwash. This could be accurate, an anomaly from a small sample size, or an artifact of the saline group having a higher starting Ct value (=lower viral load) to start from.

Prevention of upper respiratory tract infections by gargling: a randomized trial

This study started with 387 healthy volunteers and instructed them to gargle tap (not salt) water or iodine at least three times a day (the control and iodine group also gargled water once per day). For 60 days volunteers recorded a daily symptom diary. This set up is almost everything I could ask for: it looked at real illness over time rather than a short term proxy like viral load, and adherence was excellent. Unfortunately, the design were some flaws. 

Most notably, the study functionally only counted someone as sick if they had both nose and throat symptoms (technically other symptoms counted, but in practice these were rare). For a while I was convinced this was disqualifying, because water gargling could treat the pain of a sore throat without reducing viral load. However the iodine group was gargling as often as the frequent watergarglers, without their success. Iodine does irritate the throat, but gargling iodine 3 times per day produced about as much illness as water once per day. It seems very unlikely that iodine’s antiviral and throat-irritant properties would exactly cancel out. 

Taking the results at face value, iodine 3x/day + water 1x/day was no better than water 1x/day on its own. Water 3.6x/day led to a 30% reduction in illness (implicitly defined as lacking throat symptoms)

The paper speculates that iodine failed because it harmed the microbiome of the throat, causing short term benefits but long term costs. I liked this explanation because I hypothesized that problem in my previous post. Alas, it doesn’t match the data. If iodine traded a short term benefit for long term cost, you’d expect illness to be suppressed at first and catch up later. This is the opposite of what you see in the graph for iodine. However it’s not a bad description of what we see for frequent water gargling – at 15 days, 10% more of the low-frequency water garglers have gotten sick. At 50 days it’s 20% more – fully double the proportion of sick people in the frequent water gargler group. For between 50 and 60 days, the control group stays almost flat, and the frequent water garglers have gone up 10 percentage points. 

What does this mean? Could be noise, could be gargling altering the microbiome or irritating the throat, could be that the control group ran out of people to get sick. Or perhaps some secret fourth thing.

None of the differences in symptoms-once-ill were significant to p<0.05, possibly as a result of their poor definition of illness, or the fact that the symptom assessment was made a full 7 days after symptom onset.

Assuming arguendo that gargling water works, why? There’s an unlikely but interesting idea in another paper from the same authors, based on the same data. They point to a third paper that demonstrated dust mite proteins worsen colds and flus, and suggest that gargling helps by removing those dust mite proteins. Alas, their explanation of why this would help for colds but not flus makes absolutely no goddamn sense, which makes it hard to trust an already shaky idea. 

A boring but more reasonable explanation is that Japanese tapwater contains chlorine, and this acts as a disinfectant. 

Dishonorable Mention: Vitamin D3 and gargling for the prevention of upper respiratory tract infections: a randomized controlled trial

I silently discarded several papers I read for this project but this one was so bad I needed to name and shame.

The study used a 2×2 analysis examining vitamin D and gargling with tap water. However it was “definitively” underpowered to detect interactions, so they combined the gargling with and without vitamin D vs. no gargling with and without D into groups, without looking for any interaction between vitamin D and gargling. This design is bad and they should feel bad. 

Conclusion

Water (salted or no) seems at least as promising an antiviral as other liquids you could gargle, with a lower risk of side effects. So if you’re going to gargle, it seems like water is the best choice. However I still have concerns about the effect of longterm gargling on the microbiome, so I am restricting myself to high risk situations or known illness. However the data is sparse, and ignoring all of this is a pretty solid move. 

Thank you to Lightspeed Grants and my Patreon patrons for their support of this work. Thanks to Craig Van Nostrand for statistical consults.

There is a $500 bounty for reporting errors that cause me to change my beliefs, and an at-my-discretion bounty for smaller errors. 

Humming is not a free $100 bill

Last month I posted about humming as a cheap and convenient way to flood your nose with nitric oxide (NO), a known antiviral. Alas, the economists were right, and the benefits were much smaller than I estimated.

The post contained one obvious error and one complication. Both were caught by Thomas Kwa, for which he has my gratitude. When he initially pointed out the error I awarded him a $50 bounty; now that the implications are confirmed I’ve upped that to $250. In two weeks an additional $750 will go to either him or to whoever provides new evidence that causes me to retract my retraction.

Humming produces much less nitric oxide than Enovid

I found the dosage of NO in Enovid in a trial registration. Unfortunately I misread the dose-  what I original read as  “0.11ppm NO/hour” was in fact “0.11ppm NO*hour”. I spent a while puzzling out what this meant, with the help of Thomas Kwa, some guy on twitter, and chatGPT (the first time it’s been genuinely useful to me). My new interpretation is that this means “actual concentration upon application*1 hour/time at that concentration”. Since NO is a transient molecule, this means my guess for the amount of NO in Enovid was off by 2-3 orders of magnitude.

My estimates for the amount of NO released by humming may also be too high. I used this paper’s numbers for baseline NO concentration. However the paper I used to estimate the increase gave its own baseline number, which was an order of magnitude lower than the first paper.

This wasn’t intentional cherrypicking- I’d seen “15-20x increase in concentration” cited widely and often without sources. I searched for and spotchecked that one source but mostly to look at the experimental design. When I was ready to do math I used its increase but separately looked up the baseline concentration, and found the paper I cited.

I just asked google again and got an even higher estimate of baseline nasal concentration, so seems like there is a great deal of disagreement here.

If this were the only error I’d spend the time to get a more accurate estimate. But it looks like even the highest estimate will be a fraction of Enovid’s dose, so it’s not worth the energy to track down.

Using the new values, you’d need 28 minutes of humming to recreate the amount of NO in Enovid (spreadsheet here). That wouldn’t be so bad spread out over 4-6 hours, except that multiple breaths of humming in a row face diminishing returns, with recovery to baseline taking 3 minutes. It is possible to achieve this in 6 hours, but only just. And while it’s not consequential enough to bother to look it up, I think some of the papers applied Enovid more often than that.

This leaves humming in search of a use case. People who care a lot about respiratory illnesses are better off using Enovid or another nasal spray. People who don’t care very much are never going to carefully pace their humming; and the amount of humming they might do won’t be very effective. The only use case I see is people who care a lot and are pushed into a high risk situation without notice, or who want a feeling of of Doing Something even if it is not doing very much at all.

Reasons to not write off humming entirely

The math above assumes the effect is linear with the amount of NO released, regardless of application time. My guess is that frequent lower doses are more effective than the same amount as a one off. Probably not one effective enough to give humming a good non-emergency use case though.

Another possibility is that Enovid has more nitric oxide than necessary and most of it is wasted. But again, it would have to be a lot moreto make this viable.

Conclusions

Humming hasn’t been disproven as an anti-viral intervention, but the primary reason I believed it worked has been destroyed. I will be observing a six week period of mourning for both my hope in humming and generally feeling dumb.

The fact that I merely feel kind of dumb, instead of pricing out swords with which to commit seppuku, is thanks to the little angel that sits on my shoulder while I write. It constantly asks “how will you feel about this sentence if you turn out to be wrong?” and demands edits until the answer is either “a manageable amount of unhappy” or “That’s not going to come up”. This post thoroughly tested her work and found it exemplary, so she will be spending the next six weeks partying in Vegas.

[RETRACTED] Do you believe in hundred dollar bills lying on the ground? Consider humming

Introduction

[Reminder: I am an internet weirdo with no medical credentials]

A few months ago, I published some crude estimates of the power of nitric oxide nasal spray to hasten recovery from illness, and speculated about what it could do prophylactically. While working on that piece a nice man on Twitter alerted me to the fact that humming produces lots of nasal nitric oxide. This post is my very crude model of what kind of anti-viral gains we could expect from humming.

ETA 6/6: I made a major error in this post and its numbers are incorrect. The new numbers show that matching Enovid’s nitric oxide content, or even getting close enough for a meaningful effect, takes way more humming than anyone is going to do.

I’ve encoded my model at Guesstimate. The results are pretty favorable (average estimated impact of 66% reduction in severity of illness), but extremely sensitive to my made-up numbers. Efficacy estimates go from ~0 to ~95%, depending on how you feel about publication bias, what percent of Enovid’s impact can be credited to nitric oxide, and humming’s relative effect. Given how made up speculative some of these numbers are, I strongly encourage you to make up  speculate some numbers of your own and test them out in the guesstimate model.

If you want to know how nitric oxide reduces disease, check out my original post.

Math

Estimating the impact of Enovid 

I originally estimated the (unadjusted) efficacy of nitric oxide nasal sprays after diagnosis at 90% overall reduction in illness, killing ~50% of viral particles per application. Enovid has three mechanisms of action. Of the papers I looked at in that post, one mentioned two of the three (including nitric oxide) a second mechanism but not the third, and the other only mentioned nitric oxide. So how much of theat estimated efficacy is due to nitric oxide alone? I don’t know, so I put a term in the guesstimate with a very wide range. I set the lower bound to ⅓ (one of three mechanisms) to 1 (if all effect was due to NO). 

There’s also the question of how accurate the studies I read are. There are only two, they’re fairly small, and they’re both funded by Enovid’s manufacturer. One might reasonably guess that their numbers are an overestimate. I put another fudge factor in for publication bias, ranging from 0.01 (spray is useless) to 1 (published estimate is accurate).

How much nitric oxide does Enovid release?

This RCT registration uses a nitric oxide nasal spray (and mentions no other mechanisms). They don’t give a brand name but it’s funded by the company that produces Enovid. In this study, each application delivers 0.56 mL of nitric oxide releasing solution (NORS) (this is the same dose you get from commercial Enovid), which delivers “0.11ppm [NO]*hrs”. 

There’s a few things that confusing phrase could mean:

  • The solution keeps producing 0.11ppm NO for several hours (very unlikely). 
  • The application produces 0.88ppm NO almost immediately (0.11*8, where 8 hours is the inter-application interval), which quickly reacts to form some other molecule. This is my guess, and what I’ll use going forward. It won’t turn out to matter much. 
  • Some weirder thing. ETA 5/25: Thomas Kwa points out that the registration says “0.11ppm*hrs” not “0.11ppm/hr”. I’m on a tight deadline for another project so haven’t been able to look into this; it definitely seems like my interpretation is wrong, but I’m not sure his is right. I’ve reached out to some biology friends for help.

How much nitric oxide does humming move into the nose?

Here we have much more solid numbers. NO concentration is easy to measure. Individuals vary of course, but on average humming increases NO concentration in the nose by 15x-20x. Given baseline levels of (on average) 0.14ppm in women and 0.18ppm in men, this works out to a 1.96-3.42 ppm increase. More than twice what Enovid manages.

The dominant model is that the new NO in the nose is borrowed from the sinuses rather than being newly generated. Even if this is true I don’t think it matters; sinus concentrations are 100x higher than the nose’s and replenish quickly. 

Estimating the impact of humming

As far as I can find, there are no published studies on  humming as an antimicrobial intervention. There is lots of circumstantial evidence from nasal vs. mouth breathing, but no slam dunks. So I’m left to make up numbers for my Guesstimate:

  • Paper-reported decline in illness due to spray (0.9) 
  • Proportion of effect due to NO (0.33 to 1)
  • Adjustment for publication bias (.01 to 1)
  • Adjustment for using prophylactically rather than after diagnosis (0.75 to 2.5) (set this to 1 if you want to consider post-diagnosis use)
  • Bonus to humming due to higher NO levels and more frequent application (1 to 5) 
  • I capped the results so they couldn’t suggest that the effect size was less than 0  or greater than 1, and then applied the nasal-infection discount. 
  • Proportion of infections starting in the nose (because infections in the throat should see no effect from humming) (0.9 to 1) (set this to 1 if you believe the spray effect estimate already includes this effect)

From that I get an estimate of effect of 0 to 0.98, with an average of 0.67. This is of course incredibly sensitive to assumptions I pulled out of my ass. If you prefer numbers from your own ass, you can enter them into my model here. For comparison, microcovid.org estimates that masks have an efficacy against of 33% (for thick, snug cloth masks) to 87% (well-sealed n95s). 

How to hum

Here is what I’ve advised my elderly parents, and will use myself once I find a way to keep it from activating the painful nerve damage in my jaw:

  • This really is normal humming, just be sure to exhale entirely through your nose.
    • If you google “how to hum” you will mostly get results on singing exercises, which I think are suboptimal. This very annoying video has decent instructions on how to hum with your lips sealed. 
    • Higher pitch (where the vibration lives more in the nose and less in the throat) should be more effective, but making it easy to do is probably more important.
    • You only need to do one breath per session, after that you face diminishing returns.
  • Once per hour is probably overkill, but it’s also easy to remember. Alternately, pick a trigger like entering a room or opening Twitter.
    • A beta reader asked if it was worth waking up in the middle of the night to hum. I’m still not a doctor, but my immediate reaction was “Jesus Christ no”. Sleep is so important, and once per hour is a number I made up for convenience. However if you happen to wake up in the middle of the night, I expect that’s an especially valuable time to hum.
  • The less time between exposure and humming, the better. Since you can’t always know when you’ve been exposed, this suggests humming during and after every high risk event, or making it an everyday habit if you find it cheap.
  • How long after? For Enovid I made up a plan to use it for one full day after the last high risk period, which my very crude math estimates gives your body an extra day to ramp up your immune system. 

Are there downsides?

Everything with a real effect has downsides.  I’m struggling to come up with ones that won’t be immediately obvious, like vibrating a broken nose or annoying your spouse, but I’ve been surprised before.

One possible source of downsides is that the nitric oxide was more valuable in the sinuses than the nose. This doesn’t worry me much because sinus levels are 100x nasal levels, and judging from the exhalation experiments sinus levels completely recover in 3 minutes. 

The barest scraps of other evidence

This (tiny) study found that Bhramari Pranayama (which includes humming) reduced sinusitis more than conventional treatment. But the same size of 30 (per group) and lack of a no-treatment group makes this hard to take seriously.

There appeared to be a plethora of literature that nasal breathers had fewer respiratory infections than mouth breathers. I wouldn’t find this convincing even every study showed a strong effect (because it’s over such a long time period and impossible to track causality), so I didn’t bother to investigate. 

Some dude may or may not have eliminated his chronic rhinosinusitis (inflammation of nose and sinuses) that may or may not have had an infectious component by humming, which may or may not have worked by increasing nasal nitric oxide. He used a very different protocol that to my eye looks more likely to work via sheer vibration than by nitric oxide, especially because a lot of his problem was located in the sinuses.

Reasons to disbelieve

  1. If my model is correct, humming is the equivalent of finding a paper sack full of hundred dollar bills on the ground. Both the boost from humming and the immune function of NO have been known for decades; medical research would have to be really inadequate to produce so little data on this. 
  2. All of the data on the impact of nasal nitric oxide is on covid; maybe NO is less effective on other viruses.
  3. If nasal nitric oxide is so great, why did evolution give us the nasal NO concentration it did?
    1. I love me a good evolution-based argument, but I think they’re at their weakest for contagious diseases. Relative to the ancestral environment we have a much easier time finding calories to fuel our immune system and diseases with which to keep it busy, so we should expect our immune systems to be underpowered. 
  4. If humming has any effect outside the nose, it has got to be tiny. 

Conclusion

Hourly nasal humming might be as effective as masks at reducing respiratory infections. The biggest reasons to disbelieve are the paucity of data, and skepticism that society would miss something this beneficial. If you’re the kind of person who looks at an apparent hundred dollar bill on the ground and gets excited, humming seems like an unusually good thing to try. But if the pursuit of loose bills feels burdensome or doomed, I think you should respect your instincts.

I have an idea for how to generate more data on humming and respiratory illnesses, but it requires a large conference in winter. If you’re running a conference with 500+ nerds, in your local winter, with a majority of attendees coming from locations in local winter, I’d love to chat. You can reach me at elizabeth@acesounderglass.com.

Betadine oral rinses for covid and other viral infections

Before we get started, this is your quarterly reminder that I have no medical credentials and my highest academic credential is a BA in a different part of biology (with a double major in computer science). In a world with a functional medical system no one would listen to me. 

Tl;dr povidone iodine probably reduces viral load when used in the mouth or nose, with corresponding decreases in symptoms and infectivity. The effect size could be as high as 90% for prophylactic use (and as low as 0% when used in late illness), but is probably much smaller. There is a long tail of side-effects. No study I read reported side effects at clinically significant levels, but I don’t think they looked hard enough. There are other gargle formulas that may have similar benefits without the risk of side effects, which are in my queue to research.

Benefits

Math

One paper found a 90% decrease in salivary viral load after mouthwash use (which probably overestimates the effect). Another found a 90% reduction in bad outcomes, with treatment (in mouth, nose, and eyes) starting soon after diagnosis. I suspect both of these are overestimates but 1. 90% reduction is a fantastic upper bound to have 2. Neither of these looked at prophylactic use. A third study found a significant reduction in viral DNA after usage, but did not quantify that by viral load or outcomes. 

I feel like if povidone iodine was actually that good we’d have heard about it before. OTOH mouthwash formulations are barely available in the US, and most of these studies were in Asia, so maybe it went to fixation there years ago and the west is just catching up. 

So I’m going to call this 9-45% reduction in illness timeXintensity when used after symptom onset. Before onset ought to be higher, my wild ass guess is up to 90%. 

One reason I think earlier use is better is that, at least with covid, most of the real damage happens when the virus reaches the lungs. If iodine gargles can form a firewall that prevents an upper respiratory infection from becoming a lower respiratory infection, you’ve prevented most (although not all) of the worst outcomes.

Papers

I livetweeted every paper I read, collected here. I don’t want to brag, but those tweets were very popular among ladies with large boobs and 10 numbers in their twitter handles. So if that’s your type you should definitely check out those threads. Everyone else will probably find them tedious, so I’m going to summarize the most relevant papers here.

Estimating salivary carriage of severe acute respiratory syndrome coronavirus 2 in nonsymptomatic people and efficacy of mouthrinse in reducing viral load: A randomized controlled trial

This study had participants rinse their mouth with one of four mouthwashes, and compared the pre-mouthwash salivary viral load with the viral load 15 and 45 minutes later. The overall effect was very strong: 3 of the washes had a 90% total reduction, and the loser of the bunch still had a 70% reduction (error bars fairly large). 

Note that despite the title, they only gave mouthwashes to participants with symptoms.

My guess is this is an overestimate of impact, because I expect an oral rinse to have a larger effect on saliva than on cellular levels. I wish they’d tested 4-6 hours later, after the virus had had some time to regrow.

Effect of 1% Povidone Iodine Mouthwash/Gargle, Nasal and Eye Drop in COVID-19 patient 

On one hand, this paper features significant ESL issues, missing data, terrible presentation of other data, and was published in a no-name journal. On the other hand, it had one of the best study designs and 30x the number of participants of other studies. I’d love to discard this paper but there aren’t better options.

We see an almost 90% reduction in testing positive on the third day. I suspect that overstates the results because it lowers salivary or nasal fluid viral load more than cellular load, so let’s look at outcomes:

90% reduction in hospitalization, 85% reduction in oxygen use, and  88% reduction in death. 

I was skeptical of these numbers at first, especially because they only tell you the total number of an age/sex group in the study, and the number of people in a demographic group with a bad outcome. Their percentages also don’t work out properly, making it hard to see the real impact. 

Luckily almost everyone in the control group was still PCR positive on day 3, which is almost like having a participant count. The number of control participants still sick on day 3 is indeed about half of every demographic. This doesn’t rule out trickier stuff like putting people at the higher end of their age band in the control group, but it’s a good deal better than that one paper where the youngest person in the control group was a year younger than the oldest person in the treatment group. 

The short-term effect of different chlorhexidine forms versus povidone iodine mouth rinse in minimizing the oral SARS-CoV-2 viral load: An open label randomized controlled clinical trial study

I originally ignored this paper, because it only reported Ct values and not outcomes or viral load.* However the previous two papers are from the same author and have shockingly concordant results, and I wanted a second opinion. 

[*Ct value = how often you have to run the PCR machine on a sample to get over a particular threshold. This corresponds to viral load but the relationship is complicated and variable. A higher Ct value means lower viral load]

The most important finding is that Ct went up by 3.3 (S genes) and 4.4 (E genes). 

N=12 so I’m not thrilled with this study, but pickings are slim. 

Side Effects, Or: Should I just gargle iodine all the time then?

Barring very specific circumstances, I wouldn’t. There are several issues that give me pause about long term continuous use.

Hyperthyroidism

Povidone iodine skin washes can cause hyperthyroidism in infants. Among adults, many studies found increases in Thyroid Stimulating Hormone (an indicator of issues but not itself terrible), but not T3 or T4 (directly casual to outcomes). These studies tend to be small and in some cases used the wrong statistical test that missed a long tail clearly visible in their plots, so I assume there exist people for whom this creates a clinically significant effect, especially after prolonged use.

I didn’t include this paper when calculating health benefits, because its control group was too different from its treatment group. But it’s still potentially useful for tracking side effects (although at n=12, it’s still pretty limited). It found a 50% increase in TSH after a week of treatment, but no change in T3 or T4. TSH returned to normal within 12 days of ceasing treatment. That’s not worrisome for healthy people on its own, but could easily reach worrisome with longer use or a vulnerable patient. 

Tissue damage could leave you worse off?

There is a long history of aggressive use of topical antimicrobial treatments leaving users worse off due to long term tissue irritation. This is why proper wound treatment changes every decade. That same study looked at this and found no increase in cellular irritation in the throat after six months of use. It’s possible they didn’t look hard enough, or they didn’t have sufficient sample size to catch the effect. It’s also possible the species that invented ghost peppers for fun has a throat surface built to handle irritation and iodine is too weak to hurt us

Oral microbiome damage could leave you worse off?

No one studied this at all, but it looks to me like an obvious failure point. I already use oral probiotics, but if I didn’t I would add them in while using iodine.

How to use

0.5% povidone iodine is sold under the brand name Betadine. You can also buy more concentrated povidone iodine and dilute it yourself. You might be tempted to use a higher concentration, but: 1. Remember the long tail of side-effects. 2. There’s some weird evidence that higher concentrations are less effective. I didn’t dig into this very weird claim but you probably should if you plan to try it. 

The Betadine bottle recommends gargling 10ml for 30s, 4x/day. The short term studies used 4-6x/day. Spacing that out is nontrivial attention tax, so when I was sick I just put the bottle on my bathroom sink and used it every time I used the bathroom. This probably comes out to more than 6x/day (especially when I’m sick and chugging fluids), but I also didn’t use a full 10ml and rarely made it to a full 30s, so hopefully it balanced out. 

More Data Needed

The state of existing knowledge around iodine gargles is poor. This is especially frustrating because I don’t think it should be that challenging to gather more. I’m toying with a plan to fix this, but will publish separately since it’s not specific to iodine. 

For financial support I would like to thank my Patreon supports and Lightspeed grants.

Nitric oxide for covid and other viral infections

Epistemic status: I spent about 5 hours looking into this, and the next day developed covid myself.  I did a bit more research plus all of the writing while sick. So in addition to my normal warning that I have no medical credentials, you should keep in mind that this knowledge may be cursed. 

ETA 4-30-24: In this post I used “nitric oxide spray” and “enovid” as synonyms. I’ve since learned this is incorrect, NO is one of several mechanisms Enovid uses. The other mechanisms weren’t mentioned in the papers I cite so it’s possible these are accurate for NO alone.

Introduction

Nitric Oxide Nasal Spray, sold under the brand name Enovid, is a reactive compound that kills viruses (and I suspect taxes your nasal tissue). It has recently been tested and marketed for treatment of covid. The protocol I found in papers was 2 sprays per nostril every 2-3 hours, after you develop symptoms. Enovid’s instructional pamphlets say twice per day, also after you get sick. This seems a little late to me.

I suspect the real power of NONS lies in use before you develop symptoms, ideally as close to exposure as possible. This is difficult because you don’t know when you would have gotten sick, and I suspect there are costs to indefinite use. I initially thought (and told people, as a tentative guess) that one round of 4 total sprays after a high risk event was a good trade off. After doing the math for this post, that intervention seems much less helpful to me, and picking the right length of post-exposure prophylaxis depends on equations for which we lack good numbers. I pulled some numbers out of my ass for this post, but you should not trust them. 

My guess is NONS is minimally useful once covid has reached the throat, unless you combine it with a separate disinfectant of the throat. I hope to write up a report on one such disinfectant soon, although TBH it’s not looking good. 

NONS can lead to false negatives on any test based on a nasal swab, because it breaks the relationship between nasal viral load and overall load.

How does it work?

First, nitric oxide is highly reactive, which makes it destructive to anything organic. Virions are fragile to this kind of direct attack, and certain immune cells will produce nitric oxide to kill bacteria, viruses, and your own diseased cells.

First-and-a-half, nitric oxide may alter the pH of your nose, and this effect may last well past the death of the original NO molecules. This was an aside in one paper, and I haven’t followed up on it. 

Second, nitric oxide is a signaling molecule within your body, probably including but definitely not limited to the immune system. I assume the immune system uses it as a signal because it serving a functional purposes. For the rest of body the selling point appears be that it crosses membranes easily but dies quickly, making it useful when the body wants the signal to fade quickly. Viagra works by indirectly increasing your body’s synthesis of nitric oxide. 

How well does it work?

Good question, and it depends a lot on how you use it.

My best guess is that a single application (2 sprays in each nostril) of Envoid ~halves the viral load in your nose. Covid doubles in 36 hours, so that’s how much extra time you’ve bought your immune system to ramp up defenses. If you follow the more aggressive protocols in the literature and apply that treatment 6 times per day, you wipe out 95% of covid in the nose. I will attempt to translate this an efficacy estimate in that mythical future, but in the meantime siderea has a write-up on why reducing viral load is valuable even if you can’t destroy it entirely

Sometimes you will see very impressive graphs for Enovid’s impact; these are inevitably looking at the results of nasal swabs. Since even in the best case scenario NONS doesn’t affect spread once an infection has reached the throat, this doesn’t feel very relevant to me. 

Sometimes you will see very unimpressive graphs, from the rare studies that looked at transmission or symptoms. These effects are so weak, in such small studies, that I consider them essentially a null result.

…Except that these studies all started treatment days after symptoms emerged. In one case it was a minimum of 4 days. Another said “0-3 days” after symptoms, but since it takes time to see a doctor and be recruited into a study I expect the average to be on the high end of that. Additionally, both studies showed a downward slope in infection in both treatment and control groups. This is a big deal because I expect the largest effect to come if NONS is used before exponential growth really takes off. If they’re seeing a decline in viral load in their control arm, they either administered treatment too late or their placebo isn’t. 

[I think this reasoning holds even if immune overreaction is part of the problems with long covid. Long covid is correlated with severity of initial infection.]

To figure the impact of prophylactic use, I’m going to have to get, uh, speculative. Before I do that, let me dig into exactly what the data says. 

Effect size on nasal viral load

This has very solid data: even under the unfavorable circumstances of a strong infection, a day of usage drops viral load by 90-95%

Paper 1 says 95% reduction in one day, 99% in two. They took samples from the nose and throat but don’t clarify which location that applies to. If I had the energy I’d be very angry about that right now. 

(Their placebo was a saline spray, which other people claim is an antimicrobial in its own right, so this may understate the effect)

Paper 2 finds an adjusted 93-98% decline after 1 day’s use of NONS. 

Effect on symptoms/transmission, as measured by poorly designed studies

Paper 1 did track time to cure, but with a 40% response rate on a sample size of 40 in the treatment arm I can’t bring myself to care.

Paper 2 reported a couple of metrics. One is “Time to cure (as defined by PCR results)” which is still worthless because it’s still using a nasal swab. Another is clinician-assessed improvement; this effect seemed real but not huge. 

They also checked for spread to close contacts, but not very well. Contacts had to take the initiative to get tested themselves, and AFAICT they didn’t establish if they were infected before or after treatment started.  You can try to factor that out by only looking at the last day of recorded data, but the difference appears to start on day 1 of treatment, when there absolutely shouldn’t be an effect. 

Other Diseases

NONS has been studied against other infections and I fully meant to look at that data. Now that I have actual covid I consider it kind of a race to get this post out before I’m too tired, so this will come later if at all.

My wild ass guess of impact

What does a single dose do? I did a very stupid model assuming six doses over 24 hours each having the same proportionate effect, and found that halving viral load with each application was a perfect match with the data. I expect the first dose of the day has a larger effect and each one is a little less effective until you sleep and the virus has some time to marshal forces, but barring better data I’m going to treat Enovid as rolling back one doubling. 

[I want to emphasize I didn’t massage this to make the math easier. I tried .9 in my naive spreadsheet knowing it wouldn’t work, and then tried 0.5 to find it perfectly matched the data]

If my covid infection starts in the nose and I take a full course of treatment immediately after exposure, <10% chance I get sick. But that’s unachievable without constant use, which I think is a bad idea (see below).

What if you’re infected, but only in your nose? It’s a 95% reduction per day. It’s anyone’s guess how much that reduces the chance of spread to your throat; I’d say 95% is the upper bound, and am very arbitrarily setting 50% as the lower bound for the first day (this time I am trying to make the math easier). But you’re also reducing the cumulative load; on day three (after two days of treatment), your viral load is 99% lower than it would otherwise be, before you take any new doses.

I suspect the real killer app here is combining Enovid with a throat disinfectant, and am prioritizing a review of at least one throat disinfectant in a future post. 

Can I get this effect for free, without the painful stinging or logistical hassle of a nasal spray?

Maybe. Your nose already naturally produces nitric oxide, and you can increase this by 15x by humming. I haven’t been able to find the dosage of a single spray of Enovid to compare, but humming doesn’t sting so I assume it’s a lot less. On the other hand, you can hum more often than six times per day. On the third hand, I can’t tell if humming causes you to produce more NO or just release it faster, in which case chronic humming might deplete your stores. 

A quick search found multiple published articles suggesting this, but none actually studying it. The cynic in me says this is because there’s no money in it, but this study would take pennies to run and be so high impact if it worked that I suspect this is less promising than it seems. 

Update 2024-10-01: No.

Thank you to Michael Tontchev on twitter for pointing me towards humming.

Should I just use this all the time?

I don’t regularly use Envoid, despite having a shit immune system. The history of treatments like this is that long term use causes more problems than it solves. They dry out mucous membranes, or kill your own immune cells. I think the rest of you should seriously consider developing a humming habit; alas I have nerve damage in my jaw that makes vibration painful so not an option for me. 

I do think there’s a case for prophylactic use during high risk situations like conferences or taking care of a sick loved one. 

Where can I buy Enovid?

Amazon has it, but at $100/bottle it’s quite expensive. You can get it from other websites for half the price but longer shipping times; my friend used israelpharm.com and confirms he got his shipment. 

Inositol Non-Results

Three months ago I suggested people consider inositol for treating combined vague mood issues and vague stomach issues. I knew a few people who’d really benefited from it, and when one talked about it on his popular Twitter account several more people popped up thanking him for the suggestion, because it fixed their lives too. But those reports didn’t come with a denominator, which made it hard to estimate the success rate; I was hoping mentioning it on my blog and doing a formal follow-up to capture the non-responders would give a more accurate number.

Unfortunately, I didn’t get enough people to do anything useful. I received 7 responses, of which 3 didn’t have digestive issues and thus weren’t really the target. The low response rate might be a consequence of giving the wrong link in the original follow-up post, or maybe it just wasn’t that interesting. I’m reporting the results anyway out of a sense of civic virtue. 

Of those 4 remaining responses:

  • 2 rated it exactly 5 out of 10 (neutral)
  • 1 rated it as 6, which was not strong enough for them to try it a third time.
  • 1 rated it as 3- not bad enough that they spontaneously noticed a problem, but they did detailed mood tracking and the linear regression was clearly bad. 

That response rate isn’t really low enough to prove anything except that anything with a real effect can hurt you, and the value of detailed data. So for now we just have David’s estimate that 5% of people he inspired to take inositol benefited from it. 

Change my mind: Veganism entails trade-offs, and health is one of the axes

Introduction

To me, it is obvious that veganism introduces challenges to most people. Solving the challenges is possible for most but not all people, and often requires trade-offs that may or may not be worth it.  I’ve seen effective altruist vegan advocates deny outright that trade-offs exist, or more often imply it while making technically true statements. This got to the point that a generation of EAs went vegan without health research, some of whom are already paying health costs for it, and I tentatively believe it’s harming animals as well. 

Discussions about the challenges of veganism and ensuing trade-offs tend to go poorly, but I think it’s too important to ignore. I’ve created this post so I can lay out my views as legibly as possible, and invite people to present evidence I’m wrong. 

One reason discussions about this tend to go so poorly is that the topic is so deeply emotionally and morally charged. Actually, it’s worse than that: it’s deeply emotionally and morally charged for one side in a conversation, and often a vague irritant to the other. Having your deepest moral convictions treated as an annoyance to others is an awful feeling, maybe worse than them having opposing but strong feelings. So I want to be clear that I respect both the belief that animals can suffer and the work advocates put into reducing that suffering. I don’t prioritize it as highly as you do, but I am glad you are doing (parts of) it.

But it’s entirely possible for it to be simultaneously true that animal suffering is morally relevant, and veganism has trade-offs for most people. You can argue that the trade-offs don’t matter, that no cost would justify the consumption of animals, and I have a section discussing that, but even that wouldn’t mean the trade-offs don’t exist. 

This post covers a lot of ground, and is targeted at a fairly small audience. If you already agree with me I expect you can skip most of this, maybe check out the comments if you want the counter-evidence. I have a section addressing potential counter-arguments, and probably most people don’t need to read my response to arguments they didn’t plan on making. Because I expect modular reading, some pieces of information show up in more than one section. Anyone reading the piece end to end has my apologies for that. 

However, I expect the best arguments to come from people who have read the entire thing, and at a minimum the “my cruxes” and “evidence I’m looking for” sections. I also ask you to check the preemptive response section for your argument, and engage with my response if it relates to your point. I realize that’s a long read, but I’ve spent hundreds of hours on this, including providing nutritional services to veg*ns directly, so I feel like this is a reasonable request. 

My cruxes

Below are all of the cruxes I could identify for my conclusion that veganism has trade-offs, and they include health:

  • People are extremely variable. This includes variation in digestion, tastes, time, money, cooking ability… 
  • Most people’s optimal diet includes small amounts of animal products, but people eat sub-optimally for lots of reasons and that’s their right. Averting animal suffering is a better reason to eat suboptimally than most. 
  • Average vegans and omnivores vary in multiple ways, so it’s complicated to compare diets. I think the relevant comparison healthwise is “the same person, eating vegan or omnivore” or “veganism vs. omnivorism, holding all trade-offs but one constant”.
  • For most omnivores who grew up in an omnivorous culture, going vegan requires a sacrifice in at least one of: cost, taste (including variety), health, time/effort.
    • This is a mix of capital investments and ongoing costs – you may need to learn a bunch of new recipes, but if they work for you that’s a one time cost.
    • Arguments often get bogged down around the fact that people rarely need to sacrifice on all fronts at once. There are cheap ways for (most) people to eat vegan, but they either take effort and knowledge, or they’re bad for you (Oreos are vegan). There are vegan ways for most people to be close to nutritionally optimal, but they require a lot of planning or dietary monotony.
    • Some of the financial advantage for omnivores is due to meat subsidies that make meat artificially cheap, but not all of it, and I don’t know how that compares to grain subsidies.
  • There are vegan sources of every nutrient (including B12, if you include fortified products). There may even be dense sources in every or almost every nutrient. But there isn’t a satisfying plant product that is as rich in as many things as meat, dairy, and especially eggs. Every “what about X?” has an answer, but if you add up all the foods you would need to meet every need, for people who aren’t gifted at digestion, it’s far too many calories and still fairly restrictive.
    • “Satisfying” matters. There are vegan protein shakes and cereals containing ~everything, but in practice most people don’t seem to find these satisfying.
    • There isn’t a rich vegan source of every vitamin for every person. If there are three vegan sources and you’re allergic to all of them, you need animal products.
    • The gap between veganism and omnivorism is shrinking over time, as fortified fake meats and fake milks get better and cheaper. But these aren’t a cure-all.
      • Some people don’t process the fortified micronutrients as well as they process meat (and vice-versa, but that’s irrelevant on an individual level).
      • Avoiding processed foods or just not liking them is pretty common, especially among the kind of people who become vegan. 
      • Brands vary pretty widely, so you still need to know enough to pick the right fortified foods.
      • Fake meats are quite expensive, although less so every year.
        • I want to give the people behind fake meat a lot of credit. Making meat easier to give up was a good strategy for animal protection advocates.
  • Veganism isn’t weird for having these trade-offs. Every diet has trade-offs. I can name many diets I rank as having worse average trade-offs than veganism or a lower ceiling on health.
    • Carnivore diet, any monotrophic diet, ultralow calorie diets under most circumstances, “breathetarian”, liquid diets under most circumstances, most things with “cleanse” or “detox” in the name, raw foodism…
    • And even then, several of these have someone for whom they’re the best option.
  • The trade-offs vary widely by person. Some people have the digestive ability and palate of a goat and will be basically fine no matter what. Some people are already eating monotonous, highly planned diets and removing animal products doesn’t make it any harder. Some people are already struggling to feed themselves on an omnivore diet, and have nothing to replace meat if you take it away.
    • Vegan athletes are often held up as proof veganism can be healthy, with the implication that feeding athletes is hard mode so if it works for them it must work for everyone. But being a serious athlete requires a lot of the same trade-offs as veganism: you’re already planning diets meticulously, optimized for health over taste, with little variety, and taking a lot of supplements. If there are plant foods that work for you, swapping them in may be barely a sacrifice. Also, athletes have a larger calorie budget to work with.
  • Lots of people switch to vegan diets and see immediate health improvements.
    • Some improve because veganism is genuinely their optimal diet.
    • Others improve because even though their hypothetical optimal diet includes meat, the omnivore diet they were actually eating was bad for them and removing meat entirely is easier than eating good forms in moderation.
    • Others improve because they are putting more effort into their vegan diet, and they would be doing even better if they put that much effort into their omnivore diet.
    • Others see short-term improvement because animal products have both good points and bad points, and for some people the bad parts decay faster than the good parts. If your cholesterol goes down in a month and your B12 takes years to become a problem, it is simultaneously true that going vegan produced an immediate improvement, and that it will take a health toll.
  • Vegetarianism is nutritionally much closer to omnivorism than it is to veganism.
  • There exist large clusters of vegans who do not talk about nutrition and are operating naively. As in, no research into nutrition, no supplements, no testing, no conscious thought applied to their diet.
    • One of these clusters is young effective altruists whose top priority is not animal welfare (but nonetheless feel compelled to go vegan). 

Those are my premises. Below are a few conclusions I draw from them.  I originally didn’t plan on including a conclusion, but an early reader suggested my conclusions were milder than they expected and it might be good to share them. So: 

  • People recruiting for veganism should take care to onboard people in a responsible way. This could be as simple as referring people to veganhealth.org frequently enough that they actually use it.
    • Recruiting means both organized efforts and informal encouragement of friends. 
  • Diet issues are a live hypothesis suggested to vegans with health problems, especially vague, diagnosis-resistant ones.
    • This one isn’t vegan specific, although I do think it’s more relevant to them.
  • False claims about vegan nutrition should be proactively rejected by the vegan community, in both formal and informal settings, including implicit claims. This includes:
    • Explicit or implicit claims veganism is healthy for everyone, and that there is no one for whom it is not healthy.
    • Explicit or implicit claims veganism doesn’t involve trade-offs for many people. 
    • Motte and baileys of “there is nothing magic about animal products, we can use technology to perfectly replace them” and “animal products have already been perfectly replaced and rendered unnecessary”.

My evidence

One is first principles. Animal products are incredibly nutrient dense. You can get a bit of all known nutrients from plants and fortified products, and you can find a vegan food that’s at least pretty good for every nutrient, but getting enough of all of them is a serious logic puzzle unless you have good genes. Short of medical issues it can be done, but for most people it will take some combination of more money, more planning, more work, and less joy from food. 

“Short of medical issues” is burying the lede. Food allergies and digestion issues mean lots of people struggle to feed themselves even with animal products; giving up a valuable chunk of their remaining options comes at a huge cost.

[Of course some people have issues such that animal products are bad for them and giving them up is an improvement. Those raise veganism’s average health score but don’t cancel out the people who would suffer]

More empirically, there is this study from Faunalytics, which found 29% of ex-vegans and ex-vegetarians in their sample had nutritional issues, and 80% got better within three months of quitting. Their recorded attrition rate was 84%, so if you assume no current veg*ns have issues that implies a 24% of all current and former veg*ns develop health issues from the diet (19% if you only include issues meat products cured quickly). I’m really sad to only be giving you this one study, but most of the literature is terrible (see below).

The Faunalytics study has a fair number of limitations, which I went into more detail on here. My guess is that their number is a moderate underestimate of the real rate, and a severe underestimate of the value for naive vegans in particular, but 24% is high enough that I don’t think the difference matters so I’ll use that for the rest of the post.

Evidence I’m looking for

The ideal study is a longitudinal RCT where diet is randomly assigned, cost (across all dimensions, not just money) is held constant, and participants are studied over multiple years to track cumulative effects. I assume that doesn’t exist, but the closer we can get the better. 

I’ve spent several hours looking for good studies on vegan nutrition, of which the only one that was even passable was the Faunalytics study. My search was by no means complete, but enough to spot some persistent flaws among multiple studies. I’ve also spent a fair amount of time checking citations made in support of vegan claims, only to find the study is either atrocious or doesn’t support the claim made (examples in the “This is a strawman…” section). There is also some history of goalpost moving, where an advocate cites a study, I criticize it, and they say it doesn’t matter and cite a new study. This is exhausting. 

I ask that you only cite evidence you, personally, find compelling and are willing to stand by, and note its flaws in your initial citation. That doesn’t mean the study has to be perfect, that’s impossible, but you should know the flaws and be ready to explain why you still believe the study. If your belief rests on many studies instead of just one (a perfectly reasonable, nee admirable, state), please cite all of them. I am going to be pretty hard on people who link to seriously flawed studies without disclosing the flaws, or who retract citations without updating their own beliefs.

A non-exhaustive list of common flaws:

  • Studies rarely control for supplements. I’m tentatively on board with supplements being enough to get people back to at least the health level they had as an omnivore, but you can’t know their effect with recording usage and examining the impact.
  • I’ve yet to see a study that controlled for effort and money put into diet. If vegans are equally healthy but are spending twice as much time and money on food, that’s important to know.
  • Diet is self-selected rather than assigned. People who try veganism and stick with it are disproportionately likely to find it easy.
    • I don’t expect to find a study randomly assigning a long term vegan diet, but I will apply a discount factor to account for that. 
  • Studies are snapshots rather than long-term, and so lose all of the information from people who tried veganism, found it too hard, and quit.
    • Finding a way around this is what earned Faunalytics my eternal gratitude.
  • Studies don’t mention including people with additional dietary challenges, which I think are a very big deal.
  • Veganism status is based on self-identification. Other studies show that self-identified vegans often eat enough meat to be nutritionally relevant.
  • Studies often combine veganism and vegetarianism, or only include vegetarians, but are cited as if they are about veganism alone. I think vegetarianism is nutritionally much closer to omnivorism than veganism, so this isn’t helpful.
  • All the usual problems: tiny samples, motivated researchers, bad statistics. 
  • Some studies monitor dietary intake levels rather than internal levels of nutrients (as measured by tests on blood or other fluids). There are two problems with this:
    • Since RDA levels run quite high relative to average need, this is unfairly hard on vegan diets. 
    • Nutrition labels aren’t always corrected for average bioavailability, and can’t be corrected for individual variation in digestion. Plant nutrients are on average less bioavailable (although I think there are broad exceptions, and certainly individuals vary on this), so that’s perhaps too easy on plant-based diets.
  • Most studies are done by motivated parties, and it’s too easy to manipulate those. I wouldn’t have trusted the Faunalytics study if it had come from a pro-meat source.

A non-exhaustive list of evidence I hope for:

  • Quantifying the costs (across all dimensions) of dietary changes, even if the study doesn’ control for them
  • AFAICT there is no large vegan culture- the closest is lacto-vegetarian with individuals choosing to aim higher, and cultures that can’t afford meat often. Evidence of cultures with true, lifelong veganism (excluding mother’s milk) would be very interesting.
  • Studies that in some way tracking people who quit veganism, such that it could detect health issues driving people to quit. 
  • What happens to health when a very poor community earns enough to have access to occasional meat?
  • What happens when people from a lacto-vegetarian or meat-sparse culture move to a meat-loving one?
  • Studies on the impact of vegan nutritional education- how much if any does it improve outcomes?
  • What happens to people who are forced to give up animal products suddenly, for non-ethical reasons? I’m thinking of things like Alpha-gal Syndrome creating an immune response to red meat, adult onset lactose intolerance, or moving to a country that deemphasizes meat.
  • Ditto for the reverse.
    • I’m especially interested in people with dietary difficulties.
  • Studies comparing veganism and vegetarianism, especially in the same person.

 Preemptive responses to counter-arguments

There are a few counter-arguments I’ve already gotten or expect to get shortly, so let me address them ahead of time. 

“You’re singling out veganism”

Multiple people have suggested it’s wrong for me to focus on veganism. If I build enough trust and rapport with them they will often admit that veganism obviously involves some trade-offs, if only because any dietary change has trade-offs, but they think I’m unfairly singling veganism out.

First off, I’ve been writing about nutrition under this name since 2014. Earlier, if you count the pseudonymous livejournal. I talk about non-vegan nutrition all the time. I wrote a short unrelated nutrition post while this one was in editing. I understand the mistake if you’re unfamiliar with my work, but I assure you this is not a hobby I picked up to annoy you.

It’s true that I am paying more attention to veganism than I am to, say, the trad carnivore idiots, even though I think that diet is worse. But veganism is where the people are, both near me and in the US as a whole. Dietary change is so synonymous with animal protection within Effective Altruism that the EAForum tag is a wholly-owned subsidiary of the animal suffering tag. At a young-EA-organizer conference I mentored at last year, something like half of attendees were vegan, and only a handful had no animal-protecting diet considerations. If keto gets anywhere near this kind of numbers I assure you I will say something.

“The costs of misinformation are small relative to the benefits of animals”

One possible argument for downplaying or dismissing the costs of veganism is that factory farming is so bad anything is justified in stopping it. I’m open to that argument in the abstract, but empirically I think this isn’t working and animals would be better off if people were given proper information. 

First, it’s not clear to me the costs of acknowledging vegan nutrition issues are that high. I’ve gotten a few dozen comments/emails/etc on my vegan nutrition project of the form “This inspired me to get tested, here are my supplements, here are my results”. No one has told me they’ve restarted consuming meat or even milk. It is possible people are less likely to volunteer diet changes, although I do note I’m not vegan.

But even if education causes many people to bounce off, the alternative may be worse. 

That Faunalytics study says 24% of people leave veg*nism due to health reasons. If you use really naive math, that suggests that ignoring nutrition issues would need to increase recruitment by 33%, just to break even.  But people who quit veganism due to health issues tend to do so with a vitriol not seen in people leaving for other reasons. I don’t have numbers for this, but r/exvegans is mostly people who left for health reasons (with a smattering of people compelled by parents), as are the ex-vegans angry enough to start blogs. Even if they don’t make a lifestyle out of it, people who feel harmed are less likely to retry veganism, and more likely to discourage their friends.

I made a toy model comparing the trade off of education (which may lead people to bounce off) vs. lack of education (which leads people to quit and discourage others). The result is very sensitive to assumptions, especially “how many counterfactual vegans do angry ex-vegans prevent?”. If you put the attrition rate as low as I do, education is clearly the best decision from an animal suffering perspective. If you put it higher it becomes very sensitive to other assumptions. It is fairly hard to make a slam-dunk case against nutritional awareness, but then, (points at years of nutrition blogging) I would say that.

“The human health gains are small relative to the harms to animals” 

I think this is a fair argument to make, and the answer comes down to complicated math. To their credit, vegan EAs have done an enormous amount of math on the exact numeric suffering of farmed animals. But honest accounting requires looking at the costs as well.

“The health costs don’t matter, no benefit justifies the horror of farming animals”

This is a fair argument for veganism. But it’s not grounds to declare the health costs to be zero.

It’s also not grounds to ignore nutrition within a plant-based diet. Even if veganism is healthy for everyone and no harder a switch than other diets, it is very normal for dietary changes to entail trade-offs and have some upfront costs.  The push to deny trade-offs and punish those who investigate them (see below) is hurting your own people. 

“This is a strawman, vegans already address nutrition” 

I fully acknowledge that there are a lot of resources on vegan nutrition, and that a lot of the outreach literature at least name-checks dietary planning. But I talk to a lot of people (primarily young EAs focused on non-animal projects) with stories like this one, of people going vegan as a group without remembering a single mention of B12 or iron. I would consider that a serious problem even if I couldn’t point to anything the movement was doing to cause it.

But I absolutely can point to things within the movement that create the problem. There are some outright lies, and a lot more well-crafted sentences that are technically correct but in aggregate leave people with deeply misleading impressions. 

While reading, please keep in mind that these are formal statements by respected vegans and animal protection organizationss (to the best of my ability to determine). All movements have idiots saying horrible things on reddit, and it’s not fair to judge the whole movement by them. But please keep that context in mind while reading: these were not off-the-cuff statements or quick tweets, but things a movement leader thought about and wrote down. 

  • There are numerous sources talking about the health benefits of veganism. Very few of them explicitly say “and this will definitely happen with no additional work from you, without any costs or trade-offs”, but some do, and many imply it.
    • Magnus Vindling, who has published 9 books and co-founded the Center for Reducing Suffering, says :”Beyond the environmental effects, there are also significant health risks associated with the direct consumption of animal products, including red meatchicken meatfish meateggs and dairy. Conversely, significant health benefits are associated with alternative sources of protein, such as beansnuts, and seeds. This is relevant both collectively, for the sake of not supporting industries that actively promote poor human nutrition in general, as well as individually, to maximize one’s own health so one can be more effectively altruistic.”
  • This Facebook post from Jacy Reese Anthis, saying vegan dogs and cats can be perfectly healthy. Jacy was a leader among animal EAs until he left for unrelated reasons in 2019. He cites two sources, one of which supports only a subset of his claims, and the other of which actively contradicts them.
      • Apologies for the tiny image, WordPress is awful. If you right-click>open in new tab it will load a larger version.
    • His first source does say veganism can work, in dogs, but says nothing about cats.
    • His second source cites one person who says her cat is fine on a vegan diet but she doesn’t tell vets about it. The veterinarians quoted say dogs can be vegetarian and even vegan with some work. The statement on cats is ambiguous: it might be condemning only vegan diets, or both vegan and vegetarian. It rules out even vegetarian diets for young or breeding animals.

      The piece ends with “When people tell me they want to feed [their pet] a vegan diet, I say, ‘Get a goat, get a rabbit”.
    • Normally I would consider a 7 year old Facebook off-limits, but Jacy has a blue check and spent years doing very aggressive vegan advocacy on other peoples’ walls, most of which he has since deleted, so I think this is fair game. 
  • There is a related problem of motte-and-baileying “one day we will be able to have no-trade-off vegan diets, thanks to emerging technologies” and “it’s currently possible with no trade offs right this second”, e.g.: “Repudiating what “obligate carnivore” means – Kindly, but stridently, we have to correct folks that obligate carnivore stems from observation, not a diet requirement. This outdated thinking ignores the fundamental understanding of biochemistry, nutrition, and metabolism, which has only developed since the initial carnivore classification.”
  • In Doing Good Better, EA leader Will MacAskill advocates for a vegan diet to alleviate animal suffering, without mentioning any trade-offs. In isolation I don’t think that would necessarily be the wrong choice; the book is clearly about moral philosophy and not a how-to guide. But it is pushing individuals to change their personal diet (as opposed to donating to vegan recruitment programs), so I think it should at least mention trade-offs.
    • Apologies for the tiny image, WordPress is awful. If you right-click>open in new tab it will load a larger version.
  • Animal-ethics.org name-checks “a balanced diet” but the vibe is strongly “veganism is extra health with no effort”:
    • “According to the Academy of Nutrition and Dietetics, a well-planned vegan diet is nutritionally adequate and appropriate for individuals during all stages of the life cycle, including pregnancy, lactation, infancy, childhood, and adolescence, and for athletes.1 Everyone should have a balanced diet to be healthy, not only vegans. In fact non-vegans may well have unbalanced diets which are not good for their health. In order to be healthy we don’t need to consume certain products, but certain nutrients. Vegans can ingest those nutrients without having to eat animal products.”
    • “Being vegan is easier than you may think. Finding vegan food and other alternative products and services that do not involve animal exploitation is increasingly easier. It is true that some people may experience a lack of support from their family or friends or may find it extra challenging to stop eating certain animal products. However, other people can help you with that, especially today, given that internet and social networks have made it possible to get information and help from many other people. It is important to identify the factors that may be hindering your transition to veganism and look for assistance and encouragement from other people.”
    • Do I need to consult a doctor or nutritionist before becoming vegan?
      While this can be useful, as in the case of a planned non-vegan diet, it is not necessary. A vegan diet is suitable for people of all ages and conditions. A vegan nutritionist may help plan custom menus to meet specific requirements – for instance, if you are an athlete or if you want to gain or lose a lot of weight as a vegan. It is always advised to consult a nutritionist regularly for a check-up. However, it is important to note that some nutritionists are biased and don’t know a lot about vegan nutrition. Note also that medical doctors are often not experts on nutrition.”
  • EA-Foundation says veganism requires “appropriate planning”, but that this is easy 
  • That Faunalytics vegan study, which I mostly loved, contains the following: “Former vegetarians/vegans were asked if they began to experience any of the following when they were eating a vegetarian/vegan diet: depression/anxiety, digestive problems, food allergies, low cholesterol, an eating disorder, thyroid problems, protein deficiency, B12 deficiency, calcium deficiency, iron deficiency, iodine deficiency, vitamin A deficiency, vitamin D deficiency, zinc deficiency. The findings show that: – 71% of former vegetarians/vegans experienced none of the above. It is quite noteworthy that such a small proportion of individuals experienced ill health.”
    • 29% isn’t small. You can argue that’s an overestimate, but they’re accepting the 29% number, and are saying it doesn’t matter. 

Why is this so hard to talk about?

This is probably the least important section. I’m including it mostly in the hope it lowers friction in the object-level conversation. 

The stakes are so high

Hardcore vegan advocates believe we are surrounded by mass torture and slaughter facilities killing thousands of beings every day. That’s the kind of crisis that makes it hard to do really nuanced math people may use to justify ignoring you. 

Vegans are sick of concern trolls

Vegans frequently have to deal with bad-faith interrogation of their choices (“wHxt ABuoT proTEIn?!?!”). I imagine this is infuriating, and I’ve worked really hard to set myself apart by things like investing hundreds of hours of my time, much of which was unpaid, and working to get vegans the nutrition they needed to stay healthy and vegan.

Typical minding/failure of imagination

People who find veganism easier are disproportionately likely to become and stay vegan. That’s what the word “easy” means. Then some of them assume their experiences are more representative than they are, and that people who report more difficulty are lying. 

E.g. this comment on an earlier post (not even by a vegan- he was a vegan’s partner) said “there is nothing special one needs to do to stay healthy [while eating vegan]” because “most processed products like oat milk, soy milk, impossible meat, beyond meat, daiya cheese are enriched with whatever supplements are needed”. Which I would describe as “all you need to do to stay healthy while vegan is eat fortified products”. That’s indeed pretty easy, and some people will do it without thinking. But it’s not nothing, especially when “no processed foods” is such a common restriction. Sure enough, Faunalytics found that veg*ns who quit were less likely (relative to current veg*ns) to eat fortified foods. 

That same person later left another comment, conceding this point and also that there were people the fortified foods didn’t work for. Which is great, but it belonged in the first comment.

Or this commenter, who couldn’t imagine a naive vegan until an ex-vegan described the total ignorance they and their entire college EA group operated under. 

Lies we tell omnivores

Ozy Brennan has a post “Lies to cis people”. They posit that trans advocates, faced with a hostile public, give a story of gender that is simplified (because most people won’t hear the nuance anyway), and prioritizes being treated well over conveying the most possible truth. The intention is that an actual trans person or deeply invested ally will go deeper into the culture and get a more nuanced view. This can lead to some conflict when a person tries to explore gender with only the official literature as their guide.

Similarly, “veganism requires no sacrifice on any front, for anyone” is a lie vegans tell current omnivores. I suspect the expectation, perhaps subconscious, is that once they convert to veganism they’ll hang around other vegans and pick up some recipes, know what tests to get, and hear recommendations for vegan vitamins without doing anything deliberately. The longer sentence would be “for most people veganism requires no sacrifice beyond occasional tests and vitamins, which is really not much work at all”. 

But this screws over new vegans who don’t end up in those subcultures. It’s especially bad if they’re surrounded by enough other vegans that it feels like they should get the knowledge, but the transmission was somehow cut off. I think this has happened with x-risk focused EA vegans, and two friends described a similar phenomenon in the straight-edge punk scene

Failure to hear distinctions, on both sides

I imagine many people do overestimate the sacrifice involved in becoming vegan. The tradeoff is often less than they think, especially once they get over the initial hump. If omnivores are literally unable to hear “well yes, but for most people only a bit”, it’s very tempting to tell them “not at all”. But this can lead even the average person to do less work than they should, and leaves vegans unable to recognize people for whom plant based diets are genuinely very difficult, if not impossible.

Conclusion

I think veganism comes with trade-offs, health is one of the axes, and that the health issues are often but not always solvable. This is orthogonal to the moral issue of animal suffering. If I’m right, animal EAs need to change their messaging around vegan diets, and start self-policing misinformation. If I’m wrong, I need to write some retractions and/or shut the hell up.

Discussions like this are really hard, and have gone poorly in the past. But I’m still hopeful, because animal EAs have exemplified some of the best parts of effective altruism, like taking weird ideas seriously, moral math, and checking to see if a program actually worked. I want that same epistemic rigor applied to nutrition, and I’m hopeful about what will happen if it is. 

Thanks to Patrick La Victoire and Raymond Arnold for long discussions and beta-reading, and Sam Cotrell for research assistance.

Lessons learned from offering in-office nutritional testing

Introduction

I’ve talked previously about my concerns with nutritional deficiencies in effective altruists who go vegan for ethical reasons, especially those who don’t have a lot of contact with the broader vegan culture. No one else seemed very concerned, so I launched a tiny project to test people in this group and see if they were in fact deficient. This is a report on the latest phase of the project. 

To cut to the chase:

  • It was very easy to find lots of deficiencies, although due to a severely heterogenous sample and lack of a control group this doesn’t provide useful information about if veganism is at fault.
  • Finding these deficiencies probably leads to useful treatment, but not as much as I’d hoped.
  • There are still a lot of operational issues to work out. My guess is that the ideal would require more work (to encourage participants to act on their results) or less (by focusing on education but not providing testing). 
  • I am currently looking for a co-founder to properly investigate the impact of veganism on nutrition. 

My main question here was “is there low-hanging fruit in treating nutritional deficiencies in this group, and if so how do we pluck it?” An important part of that is “how prevalent are deficiencies?”, but I had substantially more uncertainty around “do people treat deficiencies you find?” and “does the treatment lead to improvements in anything we actually care about?” That prioritization (and budget issues) led the experimental design to focus on operational issues and outcomes, and deprioritized getting the kind of clean data that would let me compare vegan and non-vegan outcomes. Similarly this write-up is mostly focused on showing the problem exists at all and building metis of investigation and treatment, rather than estimating prevalence. 

Which is to say to to everyone planning on @ing me to complain about the sample size, heterogeneity, or mediocre statistics: you are right that this sample is not very informative about base rates of deficiencies in vegans or anyone else. If someone claimed it was, they would be committing an epistemic sin. However, this particular post is focused on “how much effort is it to get nutritional issues addressed, and is that effort worth it?”. Given that, any complaints about the terrible sampling will be considered to be offers of assistance in running the much larger study that could answer the prevalence question

Background 

(if you’ve read the previous posts, this will be review)

Last year I worked in a co-working space focused on existential risks, which is anything that might really end everything. Because x-risk is a popular topic within the effective altruism movement, many participants in the space were EAs. Another big topic within effective altruism is animal suffering, especially farmed animal suffering. This has led many EAs in the office to go vegan or at least vegetarian for ethical reasons, without making animals the focus of their lives. And when I asked them, many had put no thought into how to give up animal products in a healthy way.

Which of course would have been fine if they were eating well. My personal opinion is that most people’s optimal diet contains small amounts of animal products, but lots of people are eating suboptimally for lots of reasons and I don’t consider it my problem. But the number of fatigue and concentration issues were… I don’t actually know if they were high. I don’t know what baseline to compare them to. But it wasn’t low enough to reassure me. Neither did the way the vegans talked about nutrition, and in particular the fact that more than one person was doing keto vegan and still wasn’t investigating nutrition. 

None of this meant there was necessarily a problem, but I was suspicious. So I got a small grant to solve this with numbers. 

First I tried broad spectrum testing, which led me to identify iron deficiency as the main concern.

Then I did a short investigation into the actual costs of iron deficiency, which motivated a number of people to get tested, some of whom got in touch after the fact. 

In my last post I implied I was stopping the project, and wrote next steps as if someone else would be doing them. I ended up getting a follow on grant so pushed forward with round two on my own, which is what this post is about. 

Round 2 concrete steps

I followed the steps laid out in my previous post almost exactly.

  1. I brought in a company to draw blood from participants in the Lightcone office. This was expensive, probably 2-4x having people buy lab orders online, which is still more expensive than doctor-ordered tests with insurance.
    1. I say I brought them in – I in fact hired someone to handle the company and interface with participants. Aaron Silverbook was great, pretty much best case scenario for hiring a subcontractor, although still not as good as an invested co-founder.
  2. Based on previous results and nutritional folk wisdom my test priorities were ferritin (the best proxy for iron), B12, and Vitamin D. I threw in some other iron tests because they were very cheap.
  3. We had applicants fill out a form detailing their diet and any fatigue issues.
  4. Aaron picked 20 participants and made a schedule for testing them. Prioritization was: Fatigue issues > vegan > vegetarian > whoever happened to be in the office that day.
    1. This was my favorite part of having a contractor because scheduling involved some very intense math around prioritization and availability and from my perspective it was no work at all.
    2. Prioritizing people with fatigue issues probably made the immediate impact higher, and was useful for estimating the upper bound of possible impact, but ruined the sample for calculating base rates. It means I can’t really compare vegans vs. omnivores, because both were selected for having potentially-nutrition-caused problems. I had hoped to maybe compare vegans with fatigue to omnivores with fatigue, because the tired vegans had more nutritional issues that would be pretty suggestive, but ultimately didn’t have enough data to bother.
    3. I flat out didn’t have enough money to do a good study capable of establishing high confidence prevalence rates, even if that had been my primary goal. Putting aside sample size, good sampling requires getting representative participants, many of whom have no reason to get tested. They’re not intrinsically motivated, so you have to pay them to get tested. 
  5. The company comes into the office. There are a bunch of day-of headaches that Aaron handles beautifully and I am completely uninvolved with because he is good at his job.
  6. Results go out. Some participants received results over email, some were told to go to the company’s portal, some may never have gotten a notification at all. More on this in the Difficulties section.
  7. I ask Aaron to get results from participants to me, for analysis. This goes less well- I wasn’t sufficiently clear on what information I wanted in the form, and he’s gotten busy with other stuff and had less time to devote to my project, which has already gone longer than anticipated due to issues with the lab. Response rate is poor- we eventually get 13 people out of 20.
  8. I send out a second form asking for more information, and a bunch of emails harassing people to do both forms. Response rate continues to be poor, only 8 people this time. 
  9. 3 months after testing I followed up with the people with the worst scores to see if they had gotten treatment.

Project Difficulties

Other People’s Fault

I cannot say enough bad things about BayAreaPLS, the company we hired to come into the office and do testing.

First and most importantly, they just forgot to run the ferritin test for half the participants. They did the other tests, there was no reason to not do that one in particular, they just… didn’t. Their initial attempt to make up for this was an offer to not charge me for tests they didn’t do. I pushed back fairly hard and they agreed to some actual discounting. It’s been 2.5 months and they have yet to send me that follow-up invoice, which I guess is technically good for me but makes me feel worse overall.

Second, the results were hard to retrieve. Some participants never got results at all even after following the instructions, and I have no way to debug whose fault that is. Others didn’t bother to retrieve results because it was too hard. The happiest people were the ones who got their results over email, which is a HIPAA violation the company claims to have done by accident. Much like the lack of the second invoice, the emailed results are technically good for me but leave me more concerned about the company.

Even for people who did get results it took almost two weeks, which was not great for momentum.

Lastly the company used different deficiency thresholds for different people. They say this was based on sex and age, but they didn’t get that information from everyone (sounds like incompetence on their part), so some people got a list of thresholds. It ended up creating a moderate amount of confusion and friction, especially because I couldn’t tell what variation came from age/sex (which I mostly want to ignore) vs. the norms of a particular test (which I very much don’t). People are slow to act on results in general so the additional friction was quite costly. 

My Fault or Inherent Difficulties

Not all of these were literally caused by me, but they all fall under “my responsibility to anticipate and handle”.

I mentioned in my last post that some participants were kind of insistent on getting a lot of help from me, even after I explicitly told them something was outside my bailiwick and needed a doctor. I tried to fix that this time by hiring Aaron. That worked on that one issue, but made it harder to catch new problems. If the plan had been for people to directly show me their results and receive coaching I would have caught the missing test values much earlier. I’m not sure there is a way to have someone available enough to get all the relevant questions from participants without also having to deal with a bunch of irrelevant ones, including inappropriately persistent ones. The difference between the two just isn’t obvious with the knowledge most participants have, and emotions run so high around health stuff.

It’s hard to estimate the rate of acting on results because the people who don’t act are less likely to fill out the follow-up surveys, but my sense is it’s not good, and probably <50%. I also strongly suspect the rate of acting on results would have been higher if there had been in-person follow-up. 

The guidelines I sent participants emphasized vitamin D and ferritin because it didn’t occur to me anyone could see an anemia result on the page and not rush to treat it, but at least two people scored as anemic and, as of three months later, had not treated it. 

The lab obeyed HIPAA enough to only send results to participants, not directly to me, so I needed participants to forward them. Of 20 participants, 13 did so. Only 8 participated in the separate follow-up questionnaire (arguably my fault for asking for two separate forms, but there was good reason to ask for results quickly and do the follow-up a little later). 20 people with varying motivations for testing was never going to be strong evidence for anything systemic, but the low response rate makes it even harder to draw conclusions.

My emails to participants sometimes went to spam, possibly due to use of BCC, which was necessary to meet the privacy commitments I made in the application.

What were the total costs? 

As a ballpark:

  • ~$270 per participant (if they had done all the tests for everyone).
  • 0.5 hours per participant, including follow-up and the disruption to their schedule. If you want to be really conservative you could call this an hour to account for the disruption from transitions.
  • ~10 hours of Aaron’s time
  • ~20 hours of my time
  • $30 per deficient person for supplements (not covered by the study)

If you value everyone’s time roughly the same, that means that to break even we need to save one person 30 hours + ~$5700 (ignoring the information gained). 

If you want to complicate that math you can add any of: discount rate on time or money, exchange rate between ops time/my time/participant time, cost of unnecessary or counterproductive treatment, knowledge gained from this round can make the next round cheaper, the fact that most of my time went to a write-up that shouldn’t really be billed to participants, but a better program would have spent more time with participants.

Results and Benefits

Test Results

I should remind you here that the sample was a mix of 20 ethical EA vegans, vegetarians, people with fatigue issues, and people who happened to be in the office. Even a very large sample wouldn’t be perfectly predictive unless you had the exact same mix of participant types, and this sample was tiny. So the right way to look at these results is “is this enough to think there’s a problem?” and “did people do helpful things with the information?” not “at what exact rate does this population develop ferritin issues?”

With those caveats, here is the data from participants that reported back. I have ferritin results for 8 people and all other results for 13. 

  • 85% reported energy problems
  • 15% were honest-to-god anemic
  • 65% had low ferritin (30% clinically deficient) (none of the anemic people had ferritin tests done, so there is no overlap between this group and the anemics. There were non-anemic people without ferritin tests, so I assume this is a coincidence)
  • 60% had low vitamin D (15% clinically deficient)
  • A total of 80% had low scores in at least one of hemoglobin, ferritin, or vitamin D
  • 0% had low B12 (many were on supplements, but I haven’t correlated that with serum B12 levels because at this sample size and heterogeneity there is no point)

Some of the non-reporting was random due to the lab’s incompetence, but it’s not impossible unhealthy participants were more likely to report back. If you want to be extremely conservative and assume every missing value was A+ healthy, the results are still quite concerning: 

  • 10% anemic
  • 25% low-in-my-opinion ferritin, 10% clinically deficient (still no overlap with the anemics)
  • 35% low-in-my-opinion vitamin D, 10% clinically deficient
  • B12 is still great, good job everyone

I Lead This Horse To Water – You Won’t Believe What Happened Next

But finding results isn’t very meaningful if no one acts on them. Of the 8 people who filled out the follow-up survey, 75% changed either diet or supplements, and 1 additional person kept going with supplements they would otherwise have dropped. I assume this is overreporting because people who changed things are more likely to respond, but if you assume none of the nonresponders did anything that’s still 30% of people changing something.

Of the 5 who changed something and answered the relevant question, 40% said they thought they saw an improvement (~1 month after they received results). I don’t consider that particularly strong evidence in either direction- it can take time for deficiencies to heal, but it’s also easy to placebo yourself into seeing an improvement that isn’t there. The real test will be the six-month follow-up. 

Three months after testing I followed up with the two identified anemics to make sure they were getting treatment. Neither was, despite having health issues plausibly caused by anemia. They’ve both indicated vague plans to follow up now that I’ve pushed them on it. 

Is in-office testing worth it?

I believe this round of testing was better than not doing in-office testing, but there is a lot of room for improvement.

My absolute wild-ass-guess is that this saved between one and ten people (out of twenty participants) from anemia or a moderate iron deficiency, and this improved their life and productivity by 10% to 100% (mean around 20%). I acknowledge these are large ranges, but some problems are bigger than other problems. I’m ignoring vitamin D entirely here because I haven’t even attempted to quantify its value and its ardent fanclub has poisoned the literature.

Even in the worst case, a 5% chance at a 10% improvement is a big deal, so I think this was obviously worth it from a participant perspective. I think it’s a toss-up if it would be worth it for apparently healthy omnivores: my expected value for them is much lower, but people don’t always realize they’re operating at a deficit and catching them requires testing actually-healthy people as well.

I said above that we needed to save someone 30 hours + $6000 for the project to break even. Even one successfully treated anemic will blow that out of the water, so I don’t feel the need to do the more complicated math with discount rates and relative value of time, especially because any future round should either be less work or have a higher response rate.

Of course “break even” isn’t a very high bar. To go even further out on a limb: the median case of mild anemia easily costs someone two hours/day (source: had anemia one time).  This testing easily caught the anemia at least six months before it would otherwise have been caught (because everyone who was going to get tested on their own did so when I published my iron post).  That is, at a bare minimum, 360 hours someone otherwise wouldn’t have had. That’s a pretty great return rate for 40 hours (my time + Aaron’s time + participant time) +$6000.

Was the project overall worth it?

I would bet on yes, although a lot of information has yet to come in. 

I expect this project to have 6 lasting impacts:

  1. The treated health problems of participants.
  2. The secondary impact via participants’ work. 
  3. Public blog posts I write. My iron deficiency post, which motivated many people to get tested themselves, quite possibly more than received tests directly.
  4. Knowledge of how to do this more in the future. 
  5. Influence on Effective Altruism vegan culture.
  6. Animal suffering averted by making veganism more sustainable for participants.

#1 is what I calculated above, and think was already a sound success although not resounding.

#2 and #3 require making assessments of all individuals who received testing from the project or due to blog posts I wrote. That’s a combination of “vast amounts of missing information” and “judging individual merit” that makes it really uncomfortable to talk about in a public post. 

To be totally honest I’m on an “everything is bullshit” kick right now so deep in my soul I don’t think this paid off, but intellectually I think my standards are too high and this was a better project than average project in the space of existential risk. 

#4 and #5 depend on other people following up on this project. I absolutely believe they should, for all the other reasons but also because the return via #6 seems pretty good- health reasons are a common reason vegans go back to eating meat [It was hard to find a good source, but this shitty source says 26%. I’ll acknowledge that’s probably an overestimate, since health is the most virtuous reason to go back to eating meat.]

I can’t estimate #6 myself. I’m not familiar enough with vegan literature to sort good from bad here, and it wasn’t my main goal.

Then there are costs. The total grant was for a little less than $25k. I didn’t track my time very closely to avoid depressing myself, but my compensation is going to work out to a fraction of my normal hourly rate. If you count foregone client work you could argue the true cost was as high as $50k. 

My gut feeling is the project was straightforwardly worth it if you don’t track the foregone work. If you do, impact is dependent on having at least one of:

  • At least one of the impaired and successfully treated participants goes on to do high-impact work.
  • The iron post inspires at least 40 tests total, with a similar rate of finding and treating problems.
  • Follow-up projects exist and do good work. 
  • This work leads to more veganism, and you value that a lot. 

So can we blame veganism for the deficiencies?

This study doesn’t say anything one way or the other, which means I still think yes but you shouldn’t change your opinion based on the results. The sample is too small and skewed to compare deficiency rates in vegans and nonvegans. There were energetic omnivores with deficiencies and tired vegans with perfect scores so it’s clearly not deterministic. 

Next steps

I see three possible follow ups to this project:

Nutrition blogging

This is my default, although I don’t plan on writing many of these because there is only so much low hanging fruit and people have a very limited attention budget.  I have to be very judicious in what I suggest.

Mass testing to investigate deficiencies in effective altruism populations

Get the money to test a large enough representative sample, and run the tests with a proper control to actually estimate the cost of nutritional veganism. 

This is most useful if there are EA vegan leaders who won’t act on nutritional concerns now but would if the study demonstrated a problem. If this is you, I would love to talk to you about what you would consider sufficient to act on. 

Assuming the demand for this information is there, I still don’t think I want to run this project alone. First, it is a lot of work. Second…I know I said “assuming demand is there”, but I can’t picture a scenario where demand exists but no animal EAs consider this project worth working on. A collaborator would be both proof of investment and much better positioned than I am to get the information acted on. 

To that end, here is an ad for a co-founder. I will post it on this blog in a few days.

In office testing with real nutritional counseling

This can work, but only in a limited number of situations. You need a reason  (uninformed veganism, high fatigue rates) to suspect nutritional issues in lots of people sharing a space. There needs to be a reason people aren’t getting tested themselves that won’t also inhibit follow up (probably lack of money and existing relationship with a doctor). And even then it’s more of a hits-based model than a sure thing.  

My decision is easy because the office I was working out of closed, and in general I think most of the people in the bay area I would want to help have already been reached. The market is saturated for at least a year. There are other offices elsewhere in the world, and if you want to run this yourself I’m happy to act in an advisory capacity (especially if you share data), but it can’t really be an ongoing project in any one city. 

Conclusion

I finished most of this post planning on it being the end of my part of the project. I had hopes I would convince someone else to pick up the torch, and maybe even act as an advisor, but it seemed like the biggest problem was participant motivation, which I don’t feel equipped to solve. It was while I was writing this that I realized I wasn’t ready to let the broader issue of vegan nutrition go. I still believe the problem that offended my morals and epistemics is there and worth acting on.

But doing so is still very annoying, which is why I’m looking for someone to partner with on this. Someone who can handle the parts I’m bad at, point out where I’m wrong, and interface with the vegan EA community to get the results acted upon. If you’re interested, please reach out to elizabeth@acesounderglass.com.

Thank you to the Survival and Flourishg Fund for funding this research, and Lightcone Infrastructure for hosting the grant and testing. I inflicted this draft on a number of people but want to especially thank Gavin Bishop. Daniel Filan didn’t beta read this post but he did vegan-check my co-founder ad and suggest the title “I Lead This Horse To Water – You Won’t Believe What Happened Next”.

Product Endorsement: Apollo Neuro

Short version: This $310 vibrating bracelet dramatically improved my sleep and moderately improved my emotional regulation. The return policy is pretty liberal so if this seems at all appealing I recommend trying it, or one of the cheaper alternatives I haven’t investigated. Between now and Mothers’ Day they are $300. 

[note: the link I use here is an affiliate link that gives you a $40 discount and me a $50 Amazon gift card] 

EDIT 2023-06-07: they recently updated the Android app and it’s quite bad now. It stalls while loading, or connecting to the device, and I swear it’s sometimes playing the wrong program. It’s still easily worth it for me, but if you’re on the fence I would wait until the new app is polished.

Backstory

As a strong believer in luck-based medicine, I have a pretty liberal threshold for trying shit Facebook advertises to me. Most of it is crap, but every once in a while there is something amazing that justifies all the work and return fees. Previous purchases include resistance band clips that measure force applied, sleep-safe headphones that claimed to measure your HRV, and an infrared heat massage. But when I first looked at the Apollo Neuro’s website, it was too dumb even for me. The explanation for how it worked was a mix of absent and stupid, and the rush to provide scientific evidence was somehow worse than nothing. I only tried it because a friend raved about it. He had also thought it sounded deeply stupid and only bought it because his friend raved about it, and I can only assume she also thought it was stupid until someone raved about it to her, in a great circle of life. Now it is your turn to be told it sounds stupid but it works.

[For some people. My sample size is only three people.]

Benefits to me

My sleep improved a lot. I don’t have precise metrics for this because fitbit is stupid, but 1-3 times a week I wake up feeling drugged (positive valence) because my muscles are so relaxed. This never happened before the neuro unless I took actual drugs*. I also estimate my number of remembered wake-ups has been ~halved. 

The Neuro has about 8 hours of battery. When I started using it I always woke up with the battery drained, meaning I’d activated it repeatedly. I now wake up with it at 70-90% charge, partially because I wake up less and thus am not turning it back on, and partially because I use less intense vibration.

[*TBF I regularly take supplements for sleep, so what I really mean here is “unless I took more drugs than baseline”]

My emotional regulation improved as well. There were a number of stressful things I handled better than my baseline. Some of this is subjective, but there are a few things with obvious before-pictures. Most notably:

I have pretty bad medical anxiety, due most notably to dental malpratice leaving me with painful nerve damage, but also some other stuff. Last fall I had a doctor’s appointment on Friday, followed by friend’s child’s medical emergency on Saturday that, due to their newness to the country, I needed to be in charge for. I did it, but I was wrecked for at least a day afterwards, possibly more, and my partner had to put in a lot of emotional energy helping me recover. ~4 weeks into using the Apollo Neuro I had a dentist appointment in the afternoon, followed 8 hours later by a friend’s medical emergency requiring my attention and eventually a 3AM field trip to the ER for which I was the only available driver. I did have a little freak out once I was home, but I recovered to normal faster than I did from the incident in the fall despite that having been a less intense day with more help. 

I also found myself more decisively liking and disliking things, and noticing when things shifted. At parties the transition from “I’m enjoying this” to “I’m done” is clearer, without a lot of “maybe I should hold out and it will get better”. 

Mechanics

The website is stupid, so let me tell you how it really works. The Neuro is shaped like a watch with a large rectangular face. You can wear it around your wrist or ankle with a band, or clip it to clothes. When activated, it vibrates with oscillating intensity. There are 7 programs with varying oscillation patterns and durations: the wake-up program lasts five minutes with a short peak and shifts between off and on quickly, sleep spends longer in both phases and shifts between them much more slowly. Sleep is the longest program but also lowers intensity over time. You can configure the peak intensity but not duration or pattern, which I feel very oppressed by. 

You can change intensity, and pause and restart the last program from the watch, but to choose a program you need the smartphone app. It is impossible to use the watch without a smartphone, which is a serious quality of life issue for those of us with insomniac older relatives. 

The website talks about cumulative effects a lot, and is clearly pushing you to try for several weeks before judging it. There’s even some gamification for the first N hours. This felt to me exactly as necessary as a punch card from a heroin dealer. I loved it from the moment I put it on and found the little badges cheapening of my relationship with my device. But the cumulative effects part was true: as previously mentioned I needed less and less work from the Neuro to sleep, and lowered the intensity setting over time. When I first started most of the programs besides sleep and wake-up ran together, but at 6 weeks in I started really distinguishing the other programs and having strong preferences about which program, which changed over time. 

The website also advertises the Apollo for concentration problems, but I don’t know anyone who’s really tested that.

One thing I want to give the Apollo is that there is no subscription fee. You give them money and they give you the whole product and app. That should be standard but very much isn’t in the as-seen-on-FB crowd.

Cheaper Alternatives?

The Neuro is very expensive and seems like it can’t possibly cost that much to manufacture. For me paying them for the R+D was worth it. It would even have been worth a second one at the same inflated price, had my sleep not improved to the point I didn’t need it. 

Back when I thought I would need a second one to cover a full night of sleep I looked around for cheaper alternatives. None of them quite worked and I gave up when I no longer personally needed it, but if you’re motivated some might be worth checking out. 

The Senate works on a similar principle, but is no cheaper. Its programs are only 10 minutes, which is much too short for me. 

I tried a few sexual vibrators but even those with intermittent patterns transition from on to off much too quickly, which is how I figured out the gentle transition is so important. Presumably there are some that transition gently but I have no idea how to search for them. 

There are apps to vibrate your phone but they are mostly ad-ridden messes I couldn’t deal with. The one I managed to test had the same problem as the sexual vibrators. 

Products aimed at babies: these mostly run on external batteries, but they can be pretty cheap and many don’t require a smart phone. If you try any of these let me know because the alternative is teaching my elderly aunt to use a smart phone and I am not looking forward to it.

Conclusion

This product is very much in “immensely valuable to some people, price and quality of life issues limit its market for now”. If you have anxiety or sleep issues I seriously recommend trying it; the return fee looks to be about $20, although they’re less forthcoming than one might hope. If money is an issue or you’re just feeling curious you could also try the vibrating baby soothers. Amazon sells several and returns are usually free. 

If you do try the Apollo or any other product in the category I’d love to hear from you so I can share the information (please decide in your heart if you want to share your results before you try it, to avoid biasing the data). You can report your experience here

Thanks to my Patreon patrons for supporting this write up, and J for suggesting the Apollo to me in the first place.