Vegan Nutrition Testing Project: Interim Report

Introduction

Reducing consumption of animal products is a choice with both moral and practical consequences. Last summer I found myself in contact with many vegans who cared a lot about the moral consequences, but had put little effort into learning about or managing the practical consideration of removing animal products from their diet. I’ve suffered a lot due to bad nutrition, so this made me very concerned. With a grant from the Survival and Flourishing Fund, I launched small a pilot project to give nutritional tests to 5 vegans and near-vegans from the Lightcone Office, which they could use to choose supplements that would hopefully improve their health.

My long-term goal was for everyone to have accurate information on their personal nutritional costs of veganism and make informed choices about how to handle them, with the first line solution being supplements. My goal for the pilot was to work out practical issues in testing, narrow the confidence interval on potential impact, and improve the nutrition of the handful of people. This report is on phase 1: getting the testing done and supplements started. It is aimed at people who might want to run a similar program at scale; if you are interested in running this for yourself I recommend checking out Tuesday’s post on iron deficiency.

Tl;dr: I found rampant iron deficiencies, validating the overall concern. The procedure I used has a lot of room for improvement. 

The Experiment

I gave nutrition tests to 6 people in the Lightcone office. 

The ideal subject was completely vegan, had never put any effort or thought into their diet, and was extremely motivated to take a test and implement changes. This person does not volunteer for studies, so I ended up with 4 vegans or near-vegans who had put somewhere between 0 and a lot of thought into their diet, 1 vegetarian, and 1 extremely motivated omnivore I used to test out the process.  In addition, one hardcore vegan contributed results from private testing. I did not poll the ~vegans on their exact diets.

Unless otherwise stated the results exclude the omnivore.

I gave each of these six people a Genova Metabolomix+ test, ordered from walkinlabs.com, with the iron add-on. This test was selected for being recommended by doctors I trust (in part because they prefer urine to blood testing), having extremely easy-to-read results, being nearly comprehensive (with the unfortunate absence of vitamin D), and because I hoped urine collection at home would be easier than blood draws at a lab. Foreshadowing: I was wrong about that last part.

I also gave people the option of an add-on to determine what variant of the MTHFR gene they have. MTHFR can affect how one processes certain B vitamins, and certain variants can necessitate a more expensive form of supplements.

Several people (although not everyone) scored with undetectably low iron. I offered them follow-up blood tests, which one person accepted. An additional vegan contributed blood test results without urine results.

As of publication all subjects have received their first round of results and started supplements of their choosing. 

The original plan was to retest in 3-6 months after people began supplements, using the same urine tests.

My initial predictions

I expected the big shortages to be B12, iron, and vitamin D, the first of which has very few* natural vegan sources and the latter two of which are scarce, although not absent, in vegan sources. This makes it pretty unfortunate the original test did not include vitamin D. 

[*B12 is naturally found in some (but not all) seaweeds and algaes, in at least one kind of mushroom, and in nutritional yeast. It’s also added to many wheat products in the US, so if you eat enough wheat and aren’t going out of your way to get unfortified wheat that’s a strong source]

Relative to the mainstream I wasn’t very concerned about protein consumption. Vegan proteins are a little less abundant, a little harder to digest, and have a less ideal distribution of amino acids, but are basically fine as long as you don’t pile on additional constraints.

One reason I was concerned was that lots of people I polled were piling on additional constraints, like keto or gluten-free, and still not doing anything to manage nutrition. I expected a smattering of deficiencies from these people, and to a lesser extent from everyone, as their restrictions and tastes cut off random nutrients. These could have been in any almost nutrient.

I expected everyone to be fine on vitamin C because it is abundant in both produce and processed food (where it’s used as a preservative).

Results

(including only vegans and near-vegans)

  1. ¾  vegan testers had severe iron deficiencies in their urine tests.
    1. The one who didn’t had both a stunning dietary intake of iron, and a parent who 23andMe believes to have a genetic predisposition to excessive absorption of iron.
    2. An additional vegetarian tester was not deficient.
    3. One of these retested with a blood test and scored low normal (~30). However this person was already taking iron supplements at the time of the test. 
    4. A bonus blood-only participant tested between 13 and 20, meaning they’d be considered deficient by some standards but not others.
  2. There were no B12 deficiencies, probably because everyone was already on B12 supplements. 
  3. One tester had a lot of deficiencies, including vitamin C, to the point I suspect it’s a problem with digestion rather than diet. 
  4. Everyone had at least one amino acid deficiency, including the person eating over 100g of protein/day. I don’t know how big a deal this actually is.
  5. The urine test did not include vitamin D.  Of the 2 blood tests, both had low-normal vitamin D.
  6. Excluding the person with across-the-board deficiencies, there were scattered other deficiencies but nothing else to consistently worry about. People were mostly in their tests’ green zone, with occasional yellow and red.

What does this mean?

Only one near-vegan out of 5 had solidly good ferritin levels. As I discuss here, that’s a very big deal, potentially costing them half a standard deviation on multiple cognitive metrics. 

There’s no control group, so I can’t prove that this is a veganism problem. But I’m quite suspicious.

There were no other consistent problems, so broad-spectrum testing is probably overkill for people with no known problems. 

Retrospective on the project

What worked

I consider the core loop of the study as vindicated as can it be at this stage. 

  • Deficiencies were identified, and the primary one was one of the three I predicted.
    • And another of the three, B12, was probably absent because people treated it preemptively. Note that people were inconsistent in what they took so I can’t say definitively what they were on during testing.
  • In the counterfactual timeline the shortages were probably identified much later if at all. No one who participated had any plans for testing, including people with obvious symptoms and people whose doctors had previously recommended testing.

This will be less impressive if supplementation doesn’t turn out to fix anything, but it’s an extremely solid start.

Other things that went well:

  • Having the room in my budget for unplanned additional testing, so I could add in serum iron tests when it became obvious they were necessary.
  • Creating a shopping list with links. I was worried this was somehow taking advantage of people (since I used affiliate links), but removing a decision and several steps from the ordering process seems to have been pretty crucial. 
  • Bypassing the need for doctors’ visits to get a test. Given how long it took people to order tests I think doctors’ appointments would have killed the project entirely. 
  • The Lightcone ops team was extremely cooperative and got all of the vitamins I suggested into the office.

Difficulties + possible changes

Potential changes are framed as recommendations because I am deeply hoping to hand off this project to animal advocates, who caused the veganism in the first place. 

  1. The test ordering workaround was not as good as I had hoped
    1. I’d originally hoped to just hand participants a box, but they had to order the tests themselves.
    2. In order to get iron + genetics tests people had to call rather than order online. This is non-standard for the provider and two people had to call twice to insist on what they wanted.
    3. Tests took a long time to ship, and a long time to return results after shipping. The lab alleges this is a supply chain issue and there’s nothing to be done about it. 
    4. Those two together turned into a pretty big deal because they made it very hard to plan and people lost momentum.
    5. In combination with the results showing few problems beyond iron I recommend deemphasizing full spectrum urine tests and focusing on blood tests for iron (and vitamin D), and making those convenient, perhaps by bringing a phlebotomist to the office.
    6. Another option would be to bring in a medical practitioner, who can order tests for other people, to manage tests so the office can be stocked with them. This of course fails to solve the problem for anyone not in the office.
    7. There are home tests for vitamin D and iron specifically, but I have no idea if they’re any good.
  2. Ideal test subjects (completely vegan, never done nutritional testing or interventions, promptly puts in the effort to do these tests and act on them once I suggest it) were even thinner on the ground than anticipated.
    1. I knew there wouldn’t be many, but I didn’t think it would be so hard to get five people pretty close to that profile. 
    2. I loosened restrictions and still consistently found problems, so recommend lowering the eligibility bar for testing in future rounds, especially since that was always the plan. The strict requirements in this round were an attempt to make the signal as loud as possible.
  3. Getting everyone tested was like herding cats. Beyond the problems with the test distributor, some participants needed repeated reminders to order, one lost a test, results went missing… it was kind of a nightmare.
    1. One advantage of focusing on blood tests would be to cut down on this, especially if you bring the phlebotomist to the office.
  4. At points I was uncomfortable with the deference some participants showed me. I was as clear as I possibly could be that this was a best-effort from a knowledgeable amateur kind of thing; they were responsible for their own health and I was a nonexpert trying to provide some logistics help. I nonetheless got more than one person bringing me problems not even related to the nutrition project, and insisting I tell them what to do.
    1. Recommendation: bring in a skilled nutritionist. They can both give better advice than me and devote more time to helping people. 
  5. I initially misread the protein results (which are delivered in terms of “how deficient are you?” rather than “what’s your current level?”, making 0 the best possible score). Luckily I knew I was confused from the beginning and no one had taken any actions based on my misinterpretation. More broadly, I’m just a woman who’s had some problems and read some stuff, I expect my suggestions to be better than nothing but far from the maximum good it would be possible to do.
    1. Recommendation: bring in a skilled nutritionist
  6. I underestimated the amount of time and especially emotional labor this project would need. I was hoping to bluff my way through that until people got on supplements, at which point the improvements in health would be their own motivation. I think I always overestimated how well that would work, but it was especially wrong because all the problems with the tests drained people’s momentum.
    1. Recommendation: I still think you should bring in a skilled nutritionist
  7. Many of the participants were moving frequently and not in the office by the time their results came in (because they took so long…), so they had to buy supplements themselves. Given the option I would have selected people consistently in the office, but as mentioned I was already managing trade-offs around participants.
    1. Recommendation: ask for more money to give everyone their first month of supplements and a convenient pill planner.

Next Steps

I previously planned to give people the same urine test 3-6 months after they started supplements. That no longer seems worth it, relative to the cheaper and more convenient blood tests. 

It’s not actually clear a formal follow-up is that useful at all. I initially planned that because I expected a wide range of shortages such that literature reviews wouldn’t be helpful. But there was only one real problem, and it has a richer literature than almost any micronutrient. So I don’t think another 5 people’s worth of scattered data is going to add much information. 

So the next step for this as a project would be mass blood testing for B12, iron, and vitamin D. 

Feeling motivated?

If this has inspired you to test your own nutrition, I haven’t done anything you can’t do yourself. Both the urine and blood tests are available at walkinlabs.com, and if you have a doctor they’re quite likely to agree to testing, especially if you’re restricting meat products or fatigued. I have a draft guide of wisdom on supplementation I’ve picked up over the years here, although again, I’m not a doctor and only learned how to digest food last May, so use at your own risk. 

Thank you to the Survival and Flourishing Fund for funding this project, Lightcone for hosting, and all the participants for their precious bodily fluids.

Iron deficiencies are very bad and you should treat them

In brief

Recently I became interested in what kind of costs were inflicted by iron deficiency,  so I looked up studies until I got tired. This was not an exhaustive search, but the results are so striking that even with wide error bars I found them compelling. So compelling I wrote up a post with an algorithm for treating iron deficiency while minimizing the chance of poisoning yourself. I’ve put the algorithm and a summary of potential gains first to get your attention, but if you’re considering acting on this I strongly encourage you to continue reading to the rest of the post where I provide the evidence for my beliefs.

Tl;dr: If you are vegan or menstruate regularly, there’s a 10-50% chance you are iron deficient. Excess iron is dangerous so you shouldn’t supplement blindly, but deficiency is easy and cheap to diagnose with a common blood test. If you are deficient, iron supplementation is also easy and cheap and could give you a half standard deviation boost on multiple cognitive metrics (plus any exercise will be more effective). Due to the many uses of iron in the body, I expect moderate improvements in many areas, although how much and where will vary by person. 

Note that I’m not a doctor and even if I was there isn’t good data on this, so it’s all pretty fuzzy. The following is an algorithm for treating iron deficiency that I’ve kludged together from various doctors. I strongly believe it is a lot better than nothing on average, but individuals vary a lot and you might be unlucky. 

  1. Take a serum ferritin test. If you have a doctor they will almost certainly say yes to a request, or you can order for yourself at walkinlab.com
  2. If your results show a deficiency (<20ug/L), increase iron intake through diet or supplements such as Ferrochel, taking the default dose once per day, with a meal.
    1. The definition of deficiency can vary by study, lab and goal. I picked <20ug/L because it’s the highest level I have concrete evidence is insufficient, but personally believe people are likely to benefit from iron beyond that and am taking pills accordingly.
  3. If you experience negative effects after taking the pills, stop immediately. Give yourself a week to recover, then you can try other brands, be more careful to eat with a full meal, etc. 
  4. If you are experiencing the symptoms of iron poisoning (listed below), stop pills and see a doctor now. Iron poisoning is a very big deal, which is why step 1 of this algorithm is “get tested” not “gobble pills”.  Unfortunately several of these are pretty generic, but I’m never going to feel bad about telling people with seizures to seak medical attention:
    1. Nausea
    2. Vomiting
    3. Abdominal pain
    4. Dizziness
    5. Low blood pressure and a fast or weak pulse
    6. Headache
    7. Fever
    8. Shortness of breath and fluid in the lungs
    9. Grayish or bluish color in the skin
    10. Jaundice (yellowing of the skin due to liver damage)
    11. Seizures
    12. Black or bloody stools
  5. Retest at 8-12 weeks, ideally at the same lab as before.
  6. Continue to retest every 8-12 weeks.
  7. If you increase by 20ug from your starting value without noticing any improvements to your cognition or overall energy levels; low ferritin is probably not your bottleneck.
    1. If you believe it’s not a problem at all, quit.
    2. If you believe it is a problem but another problem is limiting your gains, stay on a maintenance dosage but don’t put more time into managing this. Verrrrry roughly, divide your current dosage by your currently monthly gains (so If you take one RDA/day and gain 10ug/month, your result is 0.1), and take that much. This hopefully keeps you from losing ground, without gaining so quickly it could become a problem.
  8. If you’re getting improvements, keep going until those taper off. I personally would exercise caution and investigate the downsides of iron once I reached 80ug/L, but I’ve never gotten close to that so it hasn’t come up. 
  9. Continue to retest and adjust until you’ve found a dose on which your values are stable and healthy.

[Note: I provided links to supplements because I found people follow through more when I do, and because it’s easy to buy worthless supplements. There are other good supplements out there and if you have a reason to prefer one, take that instead. Links are affiliate.]

Research summary

Iron’s most famous use in the body is in hemoglobin, which your blood uses to transport oxygen. Oxygen is extremely important [citation needed], so it makes sense that low hemoglobin (aka anemia) gets a lot of attention, and everyone agrees anemia is very bad. But what the studies I read found was that even among people who started with adequate hemoglobin, a low ferritin score still predicted they would benefit from supplementation. And it’s not because of a bad definition of “adequate”; people saw benefits even when their hemoglobin didn’t change. So what else does iron do? 

Iron is one of a small number of elements that can safely accept electrons in reduction-oxidation reactions. Free electrons are quite damaging, so iron’s ability to safely contain them is important. Some specific usages:

  • The enzyme catalase, which converts caustic H2O2 to harmless water.
    • Fun fact: Catalase is the least important enzyme whose name and purpose I can recall offhand. Other enzymes achieved that status by being very important (DNA polymerase), or having self explanatory names (carbohydrase), but catalase achieved this by sounding kind of similar to a song I was into the summer I took microbiology, and I made up alternate lyrics about the enzyme.
  • Multiple points in DNA synthesis and repair, including keeping DNA polymerases in their correct shapes.
  • Myoglobin: similar to hemoglobin it binds oxygen, but instead of blood it stays in muscles, holding oxygen in reserve until it is needed.
  • Regulation of many components of the immune system.
  • Proline and lysyl hydroxylases, both used to build collagen.
  • Please enjoy this list of 80 enzymes that use iron as a cofactor.

I’d say “that’s a lot” but honestly it’s not, everything in the body is like this, it was not built to be understood.

Standard tests for anemia only look at hemoglobin. Ferritin tests are considered to be a much better measurement of cellular iron levels. There’s suspicion, although not proof, that your body prioritizes hemoglobin production above other uses of iron, so it will undersupply these other uses in order to maintain hemoglobin levels. This suggests that if you have normal hemoglobin but low ferritin, additional iron will find many uses. Unfortunately, those uses and their effects are so varied I can’t really predict what any particular person will experience.

There are any number of studies showing correlations between low ferritin and low functioning, but I don’t find those very useful. The people in those studies might have any number of deficiencies for multiple reasons, or low ferritin levels could just be a proxy for poverty. In my research I stuck to actual experiments, with controls, that gave iron to subjects and checked for an improvement in function, not just test scores. Unfortunately, there were not that many of them.

The only study I liked on the cognitive effects found an absolutely enormous effect. Successful iron supplementation led to improvements averaging >0.5 standard deviations in attention, learning, and memory. I have qualms about this study and expect the results are cherrypicked, but it’s also not necessarily the full size of the effect, because they stopped after a set amount of time rather than waiting for effects to plateau.

There were multiple studies on iron supplementation and exercise. In a nutshell: everyone’s endurance improves when they exercise. Giving people with iron deficiency but not anemia (IDNA) iron supplements increases that effect. In the strongest study, people treated for iron deficiency for 6 weeks improved their 15km time by 10%, compared to 5% in the control group. Another study (which didn’t involve exercise training) showed no improvement in time to complete a given distance, but did find the treatment group used about 5% less energy while doing so. 

Iron deficiency rates vary a lot by population, but with the patterns you’d expect. Vegans are more deficient than vegetarians, who are more deficient than omnivores. People who regularly menstruate (or give blood) are more likely to be deficient. I found the baseline rate of omnivorous men in rich countries to be somewhere between 0-11%. For a female omnivore it’s 9-22% (these numbers include people already taking supplements; it’s presumably higher if you don’t). Young female vegans who were not already supplementing were at least 50% deficient, plausibly more. Data for non-supplementing male vegans was not available, but let’s ballpark it at 5-25%, based on the ratio between men and women in the general public.

People in poor countries are much more likely to be iron deficient and anemic, due to poor diet and more physical exertion. 

Caveats

I am not a doctor, my most relevant credential is a BA in a different part of biology, the fact that I couldn’t find a decent resource and had to make it myself is a sign of civilizational inadequacy.

Normally not being a doctor inhibits me from giving medical advice, but I am going to go ahead and say that iron poisoning is extremely bad and not that hard to induce with pills, don’t do that. Iron poisoning is why you need to be careful your kid only gets one multivitamin a day, and why men can’t use women’s multivitamins (which should actually be “menstruator’s multivitamins”, since the relevant issue is monthly blood loss). 

The papers are very finicky and boring and this was really important, so I’ve tried to frontload my conclusion. This is a delicate balancing act of readability and accuracy. I did my best but some trade-offs are unavoidable. 

This lit review was done with a focus on people with low iron intake, especially vegans. None of the studies I looked at filtered on dietary intake versus absorption issues. This means they probably underestimate the impact of supplementing for healthy people.

Do not take the dosages in the studies literally, especially if you don’t menstruate. The right dosage depends on the form and your personal needs. I suggest operating based on RDA percentages rather than raw chemical weights.

Details

Definitions

There are a lot of ways to measure iron and iron-related levels in the body. The two most important are hemoglobin (the protein red blood cells use to carry oxygen) and ferritin (the protein your cells use to store oxygen, but also present in blood). There are some other numbers I’m going to ignore.

Hemoglobin and ferritin are both testable via blood sample, and the tests have something called “reference ranges”, which are supposed to be the healthy range of values. Whether the ranges actually capture that is a matter of great controversy, with various people alleging the minimum is what you need to avoid hardcore deficiency diseases, but won’t get you optimum functioning, to people claiming low scores are fine and anyone who says otherwise is a psyop from Big Vitamin. And then there’s individual variation.

Hemoglobin’s reference range is 120g/L-170g/L.  Ferritin’s reference range starts between 10 and 20 ug/L, and ends at 150-200ug/L, depending on who you ask.  It’s possible to have low hemoglobin (aka anemia) without an iron deficiency or vice versa. Low hemoglobin with adequate iron typically means you’re having trouble manufacturing hemoglobin and is beyond the scope of this post. Low iron with adequate hemoglobin is more controversial. Top explanations include “the tests aren’t that good”, “you’re deficient but your body is prioritizing hemoglobin production”, “you’re about to develop anemia” and “low iron is fine, actually”.  

Impact Data

When looking at studies I used the following selection criteria:

  • Examining iron deficiency without anemia. We can assume that anemic cases will benefit more from iron, unless the anemia is unrelated to the iron deficiency.
  • No co-morbidities.
  • On adult humans (in practice this almost always means women).
  • In the developed world.
  • RCTs only, no correlational studies.

This didn’t leave a lot of studies, and I had to accept some other flaws. 

Mental

Murray-Kolb and Beard (2007)

This study was by far the best study of cognitive function, maybe the only one that tested an intervention rather than merely looking at correlations. I don’t love it. The data presentation is obviously leaving a lot of information out, I assume to dramatize results. But those results are very dramatic. 

This study allowed for mild anemia (hb < 120 but >105), but separated anemic and non-anemic subjects. The paper, uh, doesn’t mention its threshold for iron deficiency; another paper from the same authors set it at serum ferritin <=12ug/L, which is in line with the aggregated averages.

The study included a double control group that started with sufficient iron and hemoglobin. Each group (no deficiency (n=42), iron deficiency without anemia (n=73), and iron deficiency with anemia (n=34)) was split into treatment and placebo groups. 

Iron supplementation increased ferritin levels in everyone. People with iron deficiency without anemia (IDNA) increased serum ferritin (sFt in the table) 2.5x more than their placebo group; people with iron deficiency and anemia (IDA) improved ferritin levels almost 4x more than their placebos. Neither group got anywhere close to the ferritin levels of the no-deficiencies group. The treatment group was given 160 mg of ferrous iron daily.

In baseline cognitive testing, IDNA women scored about the same or slightly worse as healthy women, and IDA women scored much worse than both. This is probably an underestimate of the effect, because the study was heavily recruited from students at a single university, who can be expected to be selected for the same range of competence.

The study separately evaluated treatment-group women who had increased ferritin levels from those who didn’t. The former group had large improvements in their cognitive test results, the latter very modest ones. I think separating out non-responders is fair: if there’s a problem interfering with iron absorption that doesn’t tell you anything about the effect of increasing ferritin levels, and I am studying this mostly for the benefit of people with insufficient dietary intake.

Among ferritin responders, attention, memory, and learning increased from .5 to .75 standard deviations (although somehow that .75 is at p<0.07). That effect size is the equivalent of 7.5-12.5 IQ points or 1.1-1.6 inches in height. Of course the test could be bullshit, but it’s not out of line with anecdotes I hear. Additionally, the treatment groups did not reach the ferritin levels of the healthy group, indicating potentially more gains to be had.

Hemoglobin responders also saw more improvement than non-responders, but the effect size was smaller than with ferritin, indicating an effect of iron beyond increasing hemoglobin.

(Note that the axis has changed from performance to time required, making negatives good. Yes, I am suspicious that they presented total score for one metric and time to completion for another).

Physical

Zhu and Haas (1998)

20 women with normal hemoglobin (Hb >120 g/l) but low ferritin (serum ferritin <= 16 ug/l) were given 135 mg ferrous iron supplements for eight weeks and instructed to take with citrus juice. 17 women were given placebos as a control (random assignment, double-blinded). They were given both blood and athletic tests before and after treatment.

Treatment group hemoglobin and iron binding capacity were unchanged. Serum ferritin was up 250% for the treatment group (compared to 30% for the control). Their athletic test results did not improve any faster than the controls, however they needed less energy (2.0kj/min) and oxygen (5%) to get those same results. 

Hinton et al (2000)

22 women with normal hemoglobin (Hb >12 g/dl) but low ferritin (serum ferritin <= 16 ug/l) were given 100mg ferrous iron supplements for six weeks. 20 women were given placebos as a control (random assignment, double-blinded). They were given both blood and athletic tests before and after treatment.

Iron supplementation did not change hemoglobin or iron binding capacity levels, but did increase serum ferritin by about 50%, and transferrin saturation by 70%. Note that their ending ferritin levels (19.4) were still barely above the bottom of the reference range, indicating there was probably much more room for growth. 

The control group went marginally up on some measurements and marginally down on others, I’ve treated their changes as noise.

Both treatment and placebo groups were given 4 weeks of exercise treatment; the treatment group showed about double the athletic improvement. Endurance saw a bigger improvement than initial performance.

Brownlie et all (2004)

~20 women, with serum ferritin concentration < 16 ug/L and a hemoglobin concentration > 120 g/L were given 100mg ferrous iron/day for 6 weeks. Of that, 4 weeks also included exercise training.

Once again we see an improvement in ferritin but not hemoglobin or binding capacity.

The treatment group experienced ~30% more improvement in their trial times than the control, or 800% if they started with elevated ferritin. I’m suspicious of this posthoc subgroup analysis, but on the other hand, the bar in this graph is very big.

Hinton and Sinclair (2006)

The good news: this study has men! I did not think I was going to find any of those!

The bad news: this study has 20 people, total.

No meaningful change in hemoglobin or binding capacity, near doubling of serum ferritin in the treatment group, 15% drop in ferritin in the control group. 

Results are basically identical for the control and placebo groups.

Prevalence Data

Estimates for the prevalence of iron deficiency vary a lot by study and population. 

In the first paper I found, the estimate was 9-22% among menstruating women in the general public, and 1-2% among adult men (non-menstruating women were not included but I expect “do you lose 2-4 tablespoons of blood every month?” and “do you occasionally host demanding parasites?” to be more important than hormones or gender identification). Note that this number includes both anemic and non-anemic iron deficiency. 

Vegans are at much more risk. One German study of vegan women found a median serum ferritin level of 14 ug/L, a level that is above their reference range and LabCorp’s but below the cut-off in several of the studies cited above. They found 40% of young women fell below their threshold for deficiency (12ng/ml) and 11% of older women (presumably mostly post-menopausal) did so. Women taking iron supplements were excluded from this study.

A second German study (why are they all from Germany?) that allowed supplements and had an even gender split found rates of iron deficiency slightly lower in vegans than omnivores, but both had higher means than anyone in any of the impact studies I found. Nonetheless, 10% of vegans were iron deficient.

My own study (data forthcoming) had 3-4 male vegans and a deficiency rate of 25%-75%, depending on how you count. 

Thank you Survival and Flourishing Fund for funding this research and Lightcone Infrastructure for providing a home for it.

Thanks to Andrew Rettek for help making my normally very in-the-weeds style more accessible.

Follow up to medical miracle

The response to my medical miracle post has been really gratifying. Before I published I was quite afraid to talk about my emotional response to the problem, and worried that people would strong arm in the comments. The former didn’t happen and the latter did but was overwhelmed by the number of people writing to share their stories, or how the post helped them, or just to tell me I was a good writer. Some of my friends hadn’t heard about the magic pills or realized what a big deal it was, so I got some very nice messages about how happy they were for me.

However, it also became clear I missed a few things in the original post.

Conditions to make luck-based medicine work

In trying to convey the concept of luck-based medicine at all, I lost sight of traits I have that made my slot machine pulls relatively safe. Here is a non-exhaustive list of traits I’ve since recognized are prerequisites for luck based medicine:

  • I can reliably identify which things carry noticeable risks and need to be assessed more carefully. I feel like I’m YOLOing supplements, but that’s because it’s a free action to me to avoid combining respiratory depressants, and I know to monitor CYP3A4 enzyme effects. A comment on LessWrong that casually suggested throwing activated charcoal into the toolkit reminded me that not everyone does this as a free action, and the failure modes of not doing so are very bad (activated charcoal is typically given to treat poison consumption. Evidence about its efficacy is surprisingly equivocal, but to the extent it works, it’s not capable of distinguishing poison, nutrients, and medications).
    • This suggests to me that an easy lever might be a guide to obvious failure modes of supplements and medications, to lower the barrier to supplement roulette. I am not likely to have the time to do a thorough job of this myself, but if you would like to collaborate please e-mail me (elizabeth@acesounderglass.com).
  • A functioning liver. A lot of substances that would otherwise be terribly dangerous are rendered harmless by the human liver. It is a marvel. But if your liver is impaired by alcohol abuse or medical issues, this stops being true. And even a healthy liver will get overwhelmed if you pile the load high enough, so you need to incorporate liver capacity into your plans.
  • A sufficiently friendly epistemic environment. If it becomes common and known that everyone will take anything once, the bar for what gets released will become very low. I’m not convinced this can get much worse than it already has, but it is nonetheless the major reason I don’t buy the random health crap facebook advertises to me. The expected value of whatever it is probably is high enough to justify the purchase price, but I don’t want to further corrupt the system. 
  • Ability to weather small bumps. I’m self-employed and have already arranged my work to trade money for flexibility so this is not a big concern for me, but a few days off your game can be a big deal if your life is inflexible enough. Somehow I feel obliged to say this even though I’ve lost work due to side effects exactly once from a supplement (not even one I picked out; a doctor prescribed it) and at least three times from prescription medications.
  • A system for recognizing when things are helping and hurting, and phasing treatments out if they don’t justify the mental load. It’s good to get in the habit of asking what benefits you should see when, and pinning your doctor down on when they will give a medication up as useless.
    • Although again, I’ve had a bigger problem with insidious side effects from doctor-initiated prescription meds than I ever have with self-chosen supplements.
    • Probably there are other things I do without realizing how critical they are, and you should keep that in mind when deciding how to relate to my advice. 

Feel free to add your own conditions in the comments and I’ll add my favorites to this list.

Ketone Esters

Multiple people have asked for details on the ketone esters thing, and I sure hope that’s because I convinced them to try stuff rather than somehow sold ketone esters in particular as good. Answers to the common questions:

  • I use KE4, but I haven’t tried any others. I think when I originally looked it was the only one available without caffeine, but I could be wrong, or that could have changed.
  • When I first started and was doing longer intermittent fasting I’d do 10-15ml at night, 5-10 in the morning, and 5-15 before workouts (all on an empty stomach). I currently only do 5ml, before bed, to smooth out blood sugar issues whle sleeping.
    • The change is partially because I’m recovering from an injury and that does not mix with intermittent fasting, and partially because KE seems to have caused durable changes so there’s less point. I went from 3-4 sodas a day to none a few days after starting KE4 and it’s never reverted. The only caffeine I’ve had is incidentally in chocolate, and after the Bospro I’ve barely even had that.

Minimal Potato Diet

Again I am not recommending this, but if you would like to know what I’m doing:

  1. I use small potatoes- ideally the really tiny ones, but half-a-fist size at most. And I aim for a variety of color potatoes. These are out of a not particularly verified belief that skin has more vitamins than the core and that color means vitamins, or at least antioxidants. I also prefer the way the small ones cook.
  2. I cook the potatoes as soon as I receive them. If that’s not possible they might spend a few days in the fridge. When I let them age enough to get eyes they upset my stomach.
    1. A lot of people on the potato diet had to skin their potatoes to prevent feeling ill. I am curious if that would have been required if they’d used very new potatoes.
  3. I cook the potatoes by throwing them onto a cookie sheet and roasting at 350F for 45 minutes. I do this because it’s really quick and I prefer the dry texture.
    1. I cook 3 pounds a time because that is both the size of the bag they come in and about what my cookie sheet can hold.
    2. I tried gnocchi, but the additional flavor made me get tired of it faster. Also maybe my weight loss slowed around then but the potato weight loss has been weirdly punctuated so I dunno.
    3. I wish I could share a graph of just how weird the weight loss has been – same weight for 1-2 weeks, then 3 pounds in 4 days. Unfortunately, I keep changing my creatine dosage which ruins the aesthetics with a lot of water weight changes.
  4. The cooked potatoes spend at least a day in the fridge before eating, and ideally several. This is out of a slightly verified belief that the post-cooking cold converts some of the starches from digestible to indigestible, which lowers calories while doing something vaguely good for my digestive tract. But since I’m cooking much less often than eating they inevitably log a lot of fridge time anyway.
  5. Originally I ate about 100g/day, mostly in the morning but if I woke up craving something I’d start with that. For a few days now I’ve been experimenting with eating smaller amounts of potato more times per day and that’s maybe driven calorie consumption further down, but far too early to say for certain, and it’s not totally clear that would be desirable.
    1. This is based on my hypothesis that potatoes reduce calorie consumption in me by being a relatively bland food with (small amounts of) lots of different micronutrients, plus some help from the fiber. 
    2. Slime Mold Time Mold thinks it’s potassium and is testing that now. 
  6. I originally described myself as making no other changes. That was 100% true in the beginning, I will admit I now check in with my food diary calorie total and adjust a bit (including upwards, although not sure about the relative frequency). The point of the food diary is micronutrient tracking but it’s hard to avoid reacting to the calorie number once it’s there. I’m not sure that’s actually affecting things much – on days I happen to have a high count I eat much less the next few days without thinking about it. 
  7. My food diary is very clear I am not reliably hitting the RDA for most vitamins. I think you can do it on my calorie count but it would be a lot of effort and planning and I’m on vitamins anyway. Hopefully I get nutrition test results in the next month, although that will be much more a referendum on the Bospro than the potatoes. 
average nutritional intake for the last two weeks

A male friend lost 4 pounds on a 50% potato diet and then plateaued (but that could be from an injury). A female friend tried my minimal potato diet and experienced no change.  I think if that worked reliably we would already know about it.

Bonus

Shout to reader George who connected me with an offline friend who had similar symptoms with the same cure, who has done a ton of research into mechanisms and suggested some follow-ups. They’re not guaranteed to work but this feels like a rich vein to me. Thanks George and offline friend!

Guesstimate Algorithm for Medical Research

This document is aimed at subcontractors doing medical research for me. I am sharing it in the hope it is more broadly useful, but have made no attempts to make it more widely accessible. 

Intro

Guesstimate is a tool I have found quite useful in my work, especially in making medical estimates in environments of high uncertainty. It’s not just that it makes it easy to do calculations incorporating many sources of data; guesstimate renders your thinking much more legible to readers, who can then more productively argue with you about your conclusions. 

The basis of guesstimate is breaking down a question you want an answer to (such as “what is the chance of long covid?”) into subquestions that can be tackled independently. Questions can have numerical answers in the form of a single number, a range, or a formula that references other questions. This allows you to highlight areas of relative certainty and relative uncertainty, to experiment with the importance of different assumptions, and for readers to play with your model and identify differences of opinion while incorporating the parts of your work they agree with.

Basics

If you’re not already familiar with guesstimate, please watch this video, which references this model. The video goes over two toy questions to help you familiarize yourself with the interface.

The Algorithm

The following is my basic algorithm for medical questions:

  1. Formalize the question you want an answer to. e.g. what is the risk to me of long covid?
  2. Break that question down into subquestions. The appropriate subquestion varies based on what data is available, and your idea of the correct subquestions is likely to change as you work.
    1. When I was studying long covid last year, I broke it into the following subquestions
      1. What is the risk with baseline covid?
      2. What is the vaccine risk modifier?
      3. What is the strain risk modifier?
      4. What’s the risk modifier for a given individual?
  3. In guesstimate, wire the questions together. For example, if you wanted to know your risk of hospitalization when newly vaccinated in May 2021, you might multiply the former hospitalization rate times a vaccine modifier. If you don’t know how to do that in guesstimate, watch the video above, it demonstrates it in a lot of detail.
  4. Use literature to fill in answers to subquestions as best you can. Unless the data is very good, these probably include giving ranges and making your best guess as to the shape of the distribution of values.
    1. Provide citations for where you got those numbers. This can be done in the guesstimate commenting interface, but that’s quite clunky. Sometimes it’s better to have a separate document where you lay out your reasoning. 
    2. The reader should be able to go from a particular node in the guesstimate to your reasoning for that node with as little effort as possible.
    3. Guesstimate will use log-normal distribution by default, but you can change it to uniform or normal if you believe that represents reality better.
  5. Sometimes there are questions literature literally can’t answer, or aren’t worth your time to research rigorously. Make your best guess, and call it out as a separate variable so people can identify it and apply their own best guess.
    1. This includes value judgments, like the value of a day in lockdown relative to a normal day, or how much one hates being sick.
    2. Or the 5-year recovery rate from long covid- no one can literally measure it, and while you could guess from other diseases, the additional precision isn’t necessarily worth the effort.
  6. Final product is both the guesstimate model and a document writing up your sources and reasoning.

Example: Trading off air quality and covid.

The final model is available here.

Every year California gets forest fires big enough to damage air quality even if you are quite far away, which pushes people indoors. This was mostly okay until covid, which made being indoors costly in various ways too. So how do we trade those off? I was particularly interested in trading off outdoor exercise vs the gym (and if both had been too awful I might have rethought my stance on how boring and unpleasant working out in my tiny apartment is).

What I want to know is the QALY hit from 10 minutes outdoors vs 10 minutes indoors. This depends a lot on the exact air quality and covid details for that particular day, so we’ll need to have variables for that.

For air quality, I used the calculations from this website to turn AQI into cigarettes. I found a cigarette -> micromort converter faster than cigarette -> QALY so I’m just going to use that. This is fine as long as covid and air quality have the same QALY:micromort ratio (unlikely) or if the final answer is clear enough that even large changes in the ratio would not change our decision (judgment call). 

For both values that use outside data I leave a comment with the source, which gives them a comment icon in the upper right corner.

But some people are more susceptible than others due to things like asthma or cancer, so I’ll add a personal modifier.  I’m not attempting to define this well: people with lung issues can make their best guess. They can’t read my mind though, so I’ll make it clear that 1=average and which direction is bad.

Okay how about 10 minutes inside? That depends a lot on local conditions. I could embed those all in my guesstimate, or I could punt to microcovid. I’m not sure if microcovid is still being maintained but I’m very sure I don’t feel like creating new numbers right now, so we’ll just do that. I add a comment with basic instructions.

How about microcovids to micromorts? The first source I found said 10k per infection, which is a suspiciously round number but it will do for now. I device the micromorts by 1 million, since each microcovid is 1/1,000,000 chance of catching covid.

They could just guess their personal risk modifier like they do for covid, or they could use this (pre-vaccine, pre-variant) covid risk calculator from the Economist, so I’ll leave a note for that.

But wait- there are two calculations happening in the microcovids -> micromorts cell, which makes it hard to edit if you disagree with me about the risk of covid. I’m going to move the /1,000,000 to the top cell so it’s easy to edit.

But the risk of catching covid outside isn’t zero. Microcovid says outdoors has 1/20th the risk. I’m very sure that’s out of date but don’t know the new number so I’ll make something up and list it separately so it’s easy to edit

But wait- I’m not necessarily with the same people indoors and out. The general density of people is comparable if I’m deciding to throw a party inside or outside, but not if I’m deciding to exercise outdoors or at a gym. So I should make that toggleable.

Eh, I’m still uncomfortable with that completely made up outdoor risk modifier. Let’s make it a range so we can see the scope of possible risks. Note that this only matters if we’re meeting people outdoors, which seems correct.

But that used guesstimate’s default probability distribution (log normal). I don’t see a reason probability density would concentrate at the low end of the distribution, so I switch it to normal.

Turns out to make very little difference in practice.

There are still a few problems here. Some of the numbers are more or less made up, and others have sources but I’ve done no work to verify them, which is almost as bad.

But unless the numbers are very off, covid is a full order of magnitude riskier than air pollution for the scenarios I picked. This makes me disinclined to spend a bunch of time tracking down better numbers.

Full list of limitations:

  • Only looks at micromorts, not QALYs
  • Individual adjustment basically made up, especially for pollution
  • Several numbers completely made up
  • Didn’t check any of my sources

Example: Individual’s chance of long covid given infection

This will be based on my post last year, Long covid is not necessarily your biggest problem, with some modification for pedagogical purposes. And made up numbers instead of real ones because the specific numbers have long been eclipsed by new data and strains. The final model is available here

Step one is to break your questions into subquestions. When I made this model a year ago, we only had data for baseline covid in unvaccinated people. Everyone wanted to know how vaccinations and the new strain would affect things. 

My first question was “can we predict long covid from acute covid?” I dug into the data and concluded “Yes”, the risk of long covid seemed to be very well correlated with acute severity. This informed the shape of the model but not any particular values. Some people disagreed with me, and they would make a very different model. 

Once I made that determination, the model was pretty easy to create: It looked like [risk of hospitalization with baseline covid] * [risk of long covid given hospitalization rate] * [vaccination risk modifier] * [strain hospitalization modifier] * [personal risk modifier]. Note that the model I’m creating here does not perfectly match the one created last year; I’ve modified it to be a better teaching example. 

The risk of hospitalization is easy to establish unless you start including undetected/asymptomatic cases. This has become a bigger deal as home tests became more available and mild cases became more common, since government statistics are missing more mild or asymptomatic cases. So in my calculation, I broke down the risk of hospitalization given covid to the known case hospitalization rate and then inserted a separate term based on my estimate of the number of uncaught cases. In the original post I chose some example people and used base estimates for them from the Economist data. In this model, I made something up.

Honestly, I don’t remember how I calculated the risk of long covid given the hospitalization rate. It was very complicated and a long time ago. This is why I write companion documents to explain my reasoning. 

Vaccination modifier was quite easy, every scientist was eager to tell us that. However, there are now questions about vaccines waning over time, and an individual’s protection level is likely to vary. Because of that, in this test model I have entered a range of vaccine efficacies, rather than a point estimate. An individual who knew how recently they were vaccinated might choose to collapse that down. 

Similarly, strain hospitalization modifiers take some time to assess, but are eventually straightforwardly available. Your estimate early in a new strain will probably have a much wider confidence interval than your estimate late in the same wave. 

By definition, I can’t set the personal risk modifier for every person looking at the model. I suggested people get a more accurate estimate of their personal risk using the Economist calculator, and then enter that in the model.

Lastly, there is a factor I called “are you feeling lucky?”. Some people don’t have anything diagnosable but know they get every cold twice; other people could get bitten by a plague rat with no ill effects. This is even more impossible to provide for an individual but is in fact pretty important for an individual’s risk assessment, so I included it as a term in the model. Individuals using the model can set it as they see fit, including to 1 if they don’t want to think about it.

When I put this together, I get this guesstimate. [#TODO screenshot]. Remember the numbers are completely made up. If you follow the link you can play around with it yourself, but your changes will not be saved. If anyone wants to update my model with modern strains and vaccine efficacy, I would be delighted.

Tips and Tricks

I’m undoubtedly missing many, so please comment with your own and I’ll update or create a new version later.

When working with modifiers, it’s easy to forget whether a large number is good or bad, and what the acceptable range is. It can be good to mark them with “0 to 1, higher is less risky”, or “between 0 and 1 = less risk, >1 = more risk”

If you enter a range, the default distribution is log-normal. If you want something different, change it. 

The formulas in the cells can get almost arbitrarily complicated, although it’s often not worth it. 

No, seriously, write out your sources and reasoning somewhere else because you will come back in six months and not remember what the hell you were thinking. Guesstimate is less a tool for holding your entire model and more a tool for forcing you to make your model explicit.

Separate judgment calls from empirical data, even if you’re really sure you are right. 

Acknowledgements

Thanks to Ozzie Gooen and his team for creating Guesstimate.

Thanks to the FTX Regrant program and a shy regrantor for funding this work.

Quick Look: Asymptomatic Herpes Shedding

Tl;dr: Individuals shed and thus probably spread oral HSV1 while completely asymptomatic.

Introduction

“Herpes virus” can refer to several viruses in the herpes family, including chickenpox and Epstein-Barr (which causes mono). All herpesviridae infections are for life: once infected, the virus will curl up in its cell of choice, possibly to leap out and begin reproduction again later. If the virus produces visible symptoms, it is called symptomatic. If the virus is producing viable virions that can infect other people, it’s called shedding. How correlated symptoms and shedding are is the topic of this post. 

When people say “herpes” without further specification, they typically mean herpes simplex 1 or 2. HSV1 and 2 are both permanent infections of nerve cells that can lay dormant forever, or intermittently cause painful blisters on mucous membranes (typically mouth or genitals, occasionally eyes, very occasionally elsewhere). There are also concerns about subtle long-term effects, which I do not go into here.

There are two conventional pieces of conventional wisdom on HSV: “you can shed infectious virus at any time, even without a sore. Most people who catch herpes catch it from an asymptomatic individual” and “99.9% of shedding occurs during or right before a blister and there are distinct signs you can recognize if you’re paying attention. If you can recognize an oncoming blister the chances of infecting another human are negligible.” At the request of a client I performed two hours of research to judge between these.

It is definitely true that doctors will only run tests looking for the virus directly (as opposed to antibodies) if you have an active sore. However when researchers proactively sampled asymptomatic individuals using either genetic material tests (PCR/NAAT, which look for viral DNA in a sample) or viral culture (which attempt to breed virus from your test sample in a petri dish), they reliably found some people are shedding virus. 

HSV1 prefers the mouth but is well known to infect genitals as well. HSV2 is almost exclusively genital. Due to a dearth of studies I’ve included some HSV2 and genital HSV1 studies. 

Studies

Tronstein et al: This paper stupidly lumped in “0% shedding” with “>0% shedding” and I hate them. Ignoring that, they found that 10% of all days recorded from individuals with asymptomatic genital HSV2 involved shedding, and these were distributed on a long tail, with the peak at 0-5%. I cannot tell if they lumped 0% and 0.1% together because 0% never happens, or because they hate science. 

your buckets are bad and you should feel bad

Bowman et al: 14% of previously symptomatic genital HSV2 patients shed isolate-able virus (sampled every 8 weeks over ~3 years) while on antivirals. This study reports “isolating” virus without further details; I expect this means viral culture. 

Sacks et al: citing another paper: shedding across 6% of days in oral HSV1 patients (using viral culture). It also found the following asymptomatic shedding rates for genital herpes

Spruance: oral HSV1 patients shed isolatable virus 7.4% of the time (including while symptomatic). 60% of this occurred while experiencing mild symptoms that could have indicated an upcoming sore, but never developed into a sore.

Tateish et al: tested 1000 samples from oral surgery patients (not filtered for HSV infection status). 4.7% had PCR-detectable herpes DNA, and 2.7% had culturable virus. This includes patients without herpes (about 50% of people in Japan, where the research was done), but oral surgery is stressful and often stems from issues that make it easier to shed herpes, so I consider those to ~cancel out. 

Conclusion

My conclusion: it is definitely possible to shed HSV while asymptomatic, including if you are never symptomatic. The daily shedding rate is something like 3-12%, although with lots of interpersonal variability. This doesn’t translate directly to an infectiousness rate: human mouths might be harder or easier to infect than petri dishes (my guess is harder, based on the continued existence of serodiscordant couples). It may be possible for people who are antibody positive for HSV to never shed virus but we don’t know because no one ran the right tests. 

Thanks to anonymous client for funding the initial research and my Patreon patrons for supporting the public write-up.

New Water Quality x Obesity Dataset Available

Tl;dr: I created a dataset of US counties’ water contamination and obesity levels. So far I have failed to find anything really interesting with it, but maybe you will. If you are interested you can download the dataset here. Be warned every spreadsheet program will choke on it; you definitely need to be use statistical programming.

Photocredit: DALL-E and a lot of coaxing 

Many of you have read Slime Mold Time Mold’s series on the hypothesis that environmental contaminants are driving weight gain. I haven’t done a deep dive on their work, but their lit review is certainly suggestive. 

SMTM did some original analysis by looking at obesity levels by state, but this is pretty hopeless. They’re using average altitude by state as a proxy for water purity for the entire state, and then correlating that with the state’s % resident obesity. Water contamination does seem negatively correlated with its altitude, and its altitude is correlated with an end-user’s altitude, and that end user’s altitude is correlated with their average state altitude… but I think that’s too many steps removed with too much noise at each step. So the aggregation by state is basically meaningless, except for showing us Colorado is weird.

So I dug up a better data set, which had contamination levels for almost every water system in the country, accessible by zip code, and another one that had obesity prevalence by county. I combined these into a single spreadsheet and did some very basic statistical analysis on them to look for correlations.

Some caveats before we start:

  • The dataset looks reasonable to me, but I haven’t examined it exhaustively and don’t know where the holes are. 
  • Slime Mold Time Mold’s top contender for environmental contagion is lithium. While technically present in the database, litium had five entries so I ignored it. I haven’t investigated but my guess is no one tests for lithium.
  • It’s rare, but some zip codes have multiple water suppliers, and the spreadsheet treats them as two separate entities that coincidentally have the same obesity prevalence.
  • I’ve made no attempt to back out basic confounding variables like income or age.
  • “% obese” is a much worse metric than average BMI, which is itself a much worse metric than % body fat. 
  • None of those metrics would catch if a contaminant makes some people very fat while making others thin ( SMTM thinks paradoxical effects are a big deal, so this is a major gap for testing their model).
  • Correlation still does not equal causation.

The correlations (for contaminants with >10k entries):

ContaminantCorrelation# Samples
Nitrate-0.03921430
Total haloacetic acids (HAAs)0.05514666
Chloroform0.04615065
Barium (total)0.04017929
Total trihalomethanes (TTHMs)0.11721184
Copper-0.00217113
Dibromochloromethane0.08013856
Nitrate & nitrite0.03511902
Bromodichloromethane0.07914238
Lead (total)-0.00613031
Dichloroacetic acid-0.00310159

Of these, the only one that looks interesting is trihalomethanes, a chemical group that includes chloroform. Here’s the graph:

Visually this looks like the floor is rising much faster than the ceiling, but in a conversation on twitter SMTM suggested that’s an artifact of the biviariate distribution, it disappears if you look at log normal. 

Very casual googling suggests that TTHMs are definitely bad for pregnancy in sufficient quantities, and are maybe in a complicated relationship with Type 2 diabetes, but no slam dunks.

This is about as far as I’ve had time to get. My conclusions alas are not very actionable, but maybe someone else can do something interesting with the data.

Thanks to Austin Chen for zipping the two data sets together, Daniel Filan for doing additional data processing and statistical analysis, and my Patreon patrons for supporting this research.

Home Antigen Tests Aren’t Useful For Covid Screening

Epistemic status: I strongly believe this is the right conclusion given the available data. The best available data is not that good, and if better data comes out I reserve the right to change my opinion.

EDIT (4/27): In a development I consider deeply frustrating but probably ultimately good, the same office is now getting much more useful information from antigen tests. They aren’t tracking with same rigor so I can’t comapre results, but they are now beating the bar of “literally ever noticing covid”.

In an attempt to avoid covid without being miserable, many of my friends are hosting group events but requiring attendees to take a home covid test beforehand. Based on data from a medium-sized office, I believe testing for covid with the tests people are using, to be security theater and provide no decrease in riskAntigen tests don’t work for covid screening. There is a more expensive home test available that provides some value, and rapid PCR may still be viable.

It’s important to distinguish between test types here: antigen tests look for viral proteins, and genetic amplification tests amplify viral RNA until it reaches detectable levels. The latter are much more sensitive. Most home tests are antigen tests, with the exception of Cue, which uses NAAT (a type of genetic amplification). An office in the bay area used aggressive testing with both Cue and antigen tests to control covid in the office and kept meticulous notes, which they were kind enough to share with me. Here are the aggregated numbers: 

  • The office requested daily Cue tests from workers. I don’t know how many people this ultimately included, probably low hundreds? I expect compliance was >95% but not perfect.
    • The results are from January when the dominant strain was Omicron classic, but no one got strain tested.
  • 39 people had at least one positive Cue test, all of which were either asymptomatic or ambiguously symptomatic (e.g. symptoms could be explained by allergies) at the time, and 27 of which had recent negative cue tests (often but not always the day before, sometimes the same day)
  • Of these, 10 definitely went on to develop symptoms, 7 definitely did not, and 18 were ambiguous (and a few were missing data).
  • 33 people with positives were retested with cue tests, of which 9 were positive. 
  • Of those 24 who tested positive and then negative, 4 tested positive on tests 3 or 4.
  • Of the 20 people with a single positive test followed by multiple negative retests, 6 went on to develop symptoms.
  • 0 people tested positive on antigen tests. There was not a single positive antigen test across this group. They not only didn’t catch covid as early as Cue did, they did not catch any cases at all, including at least 2 people who took the tests while experiencing definitive systems.
    • Antigen tests were a mix of Binax and QuickVue.
    • Early cases took multiple antigen tests over several days, later cases stopped bothering entirely.
    • The “negative test while symptomatic” count is artificially low because I excluded people with ambiguous symptoms, and because later infectees didn’t bother with antigen tests. 
    • I suppose I can’t rule out the possibility that they had an unrelated disease with similar symptoms and a false positive on the Cue test. But it seems unlikely that that happened 10-28 times out a few hundred people without leaving other evidence.

A common defense of antigen tests is that they detect whether you’re contagious at that moment, not whether you will eventually become contagious. Given the existence of people who tested antigen-negative while Cue-positive and symptomatic, I can’t take that seriously.

Unfortunately Cue tests are very expensive. You need a dedicated reader, which is $250, and tests are $65 each (some discount if you sign up for a subscription). A reader can only run 1 test at a time and each test takes 30 minutes, so you need a lot for large gatherings even if people stagger their entrances. 

My contact’s best guess is that the aggressive testing reduced but did not eliminate in-office spread, but it’s hard to quantify because any given case could have been caught outside the office, and because they were trying so many interventions at once. Multiple people tested positive, took a second test right away, and got a negative result, some of whom went on to develop symptoms; we should probably assume the same chance of someone testing negative when a second test would have come back positive, and some of those would have been true positives. So even extremely aggressive testing has gaps.

Meanwhile, have I mentioned lately how good open windows and air purifiers are for covid? And other illnesses, and pollution? And that taping a HEPA filter to a box fan is a reasonable substitute for an air purifier achievable for a very small number of dollars? Have you changed your filter recently? 

PS. Before you throw your antigen tests out, note that they are more useful than Cue tests for determining if you’re over covid. Like PCR, NAAT can continue to pick up dead RNA for days, maybe weeks, after you have cleared the infection. A negative antigen test after symptoms have abated and there has been at least one positive test is still useful evidence to me. 

PPS. I went through some notes and back in September I estimated that antigen testing would catch 25-70% of presymptomatic covid cases. Omicron moves faster, maybe faster enough that 25% was reasonable for delta, but 70% looks obviously too high now. 

PPPS. Talked to another person at the office, their take is the Cue tests are oversensitive. I think this fits the data worse but feel obliged to pass it on since they were there and I wasn’t.

PPPPS (5/02): multiple people responded across platforms that they had gotten positive antigen tests. One or two of these was even presymptomatic. I acknowledge the existence proof but will not be updating until the data has a denominator. If you’re doing a large event like a conference I encourage you to give everyone both cue, antigen, and rapid PCR tests and record their results, and who eventually gets sick. If you’d like help designing this experiment in more detail please reach out (elizabeth-at-acesounderglass.com)

I Caught Covid And All I Got Was This Lousy Ambiguous Data

Tl;dr I tried to run an n of 1 study on niacin and covid, and it failed to confirm or disprove anything at all.

You may remember that back in October I published a very long post investigating a niacin-based treatment protocol for long covid. My overall conclusion was “seems promising but not a slam dunk; I expect more rigorous investigation to show nothing but we should definitely check”. 

Well recently I got covid and had run out of more productive things I was capable of doing, so decided to test the niacin theory. I learned nothing but it was a lot of effort and I deserve a blog post out of it null results are still results so I’m sharing anyway.

Background On Niacin

Niacin is a B-vitamin used in a ton of metabolic processes. If you’re really curious, I describe it in excruciating detail in the original post.

All B vitamins are water-soluble, and it is generaly believed that unless you take unbelievably stupid doses you will pee out any excess intake without noticing. It’s much harder to build up stores of water-soluble vitamins than fat vitamins, so you need a more regular supply.  Niacin is a little weird among the water-solubles in that it gives very obvious signs of overdose: called flush, the symptoms consist of itchy skin and feeling overheated. Large doses can lead to uncontrolled shaking, but why would you ever take that much, when it’s so easy to avoid?

People regularly report response patterns that sure look like their body has a store of niacin that can be depleted and refilled over time. A dose someone has been taking for weeks or months will suddenly start giving them flush, and if they don’t lower it the flush symptoms will get worse and worse. 

Some forms of niacin don’t produce flush. Open question if those offer the same benefits with no side effects, offer fewer benefits, or are completely useless.

Niacin And Long Covid

There’s an elaborate hypothesis about how covid depletes niacin (and downstream products), and this is a contributor to long covid. My full analysis is here. As of last year I hadn’t had covid (this is antibody test confirmed, I definitely didn’t have an asymptomatic case) but I did have lingering symptoms from my vaccine and not a lot else to try, so I gave the protocol a shot.

My experience was pretty consistent with the niacin-storage theory. I spent a long time at quite a high dose of the form of niacin the protocol recommends, nictonic acid. My peak dose without flush was at  least 250mg (1563% RDA) and maybe even 375mg (2345% RDA). When I hit my limit I lowered my dose until I started getting flush at the new dose, and eventually went off nicotnic acid entirely (although I restarted a B-vitamin that included 313% RDA of a different form). That ended in September or early October 2021. It made no difference in my lingering vaccine symptoms.

In early 2022 I tried nicotinic acid again. Even ¼ tablet (62.5mg, 390% RDA) gave me flush.

I Get Covid

Once I developed symptoms and had done all the more obviously useful things like getting Paxlovid, I decided it would be fun to test myself with niacin (and the rest of the supplement stack discussed in my post) and see if covid had any effect. So during my two weeks of illness and week of recovery I occasionally took nicotinic acid and recorded my results. Here’s the overall timeline:

  1. Day -2: am exposed to covid.
  2. Day 0: test positive on a cue test (a home test that uses genetic amplification).
    1. Lung capacity test: 470 (over 400 is considered health).
    2. Start Fluvamoxine and the vitamin cocktail, although I’m inconsistent with both the new and existing vitamins during the worst of the illness. Vitamin cocktail includes 313% RDA of no-flush niacin, but not nicotinic acid. 
  3. Day 1: symptomatic AF. 102.3 degree fever, awake only long enough to pee, refill my water, and make sure my O2 saturation isn’t going to kill me. I eat nothing the entire day.
    1. I monitored my O2 throughout this adventure but it never went into a dangerous zone so I’m leaving it out of the rest of the story.
  4. Day 2: start with 99 degree fever, end day with no fever. Start Paxlovid.
    1. Every day after this I am awake a little bit longer, eat a little bit more, and have a little more cognitive energy, although it takes a while to get back to normal. 
    2. Try ¼ tab nicotinic acid (62.5 mg/ 375% RDA), no flush.
    3. Lung capacity troughs at 350 (considered orange zone).
  5. Day 4: ½ tablet nictonic acid, mild flush.
  6. Day 7: lung capacity up to 450, it will continue to vary from 430-450 for the next two weeks before occasionally going higher.
  7. Day 9: ½ tablet nictonic acid, mild flush
  8. Day 10-17: ⅓ tablet nictonic acid, no flush
    1. Where by “⅓” tablet I mean “I bit off an amount of pill that was definitely >¼ and <½ and probably averaged to ~⅓ over time”
  9. Day 12: I test positive on a home antigen test
  10. Day 15: I test negative on a home antigen test (no tests in between) 
  11. Day 17: ⅓ tablet produces flush (and a second negative antigen test)
    1. This was also the first day I left my house. I had thought of myself as still prone to fatigue but ended up having a lot of energy once I got out of my house and have been pretty okay since.

Conclusions

My case of covid was about as bad as you get while still technically counting as mild. Assuming I went into it with niacin stores such that 62.5mg nicotinic acid would generate flush, it looks like covid immediately took a small bite out of them. Or it reduced my absorption of vitamins, such that the same oral dosage resulted in less niacin being taken in. There’s no way to know covid had a larger effect on niacin than other illnesses, because I don’t have any to compare it to. Or maybe the whole thing was an artifact of “not eating for two days, and then only barely, and being inconsistent with my vitamins for a week”.

Niacin as a treatment for covid? (Probably no, but I’m glad we’re checking)

Introduction

This article contains an interview with a doctor who believes NAD+ is the secret to covid’s heavy morbidity and mortality toll. The description was unusually well done for internet crackpottery. This is hard to convey rigorously, but it had a mechanistic-ness and the right level of complexity about it, and it made the right level of promises for a treatment. None of this is to say it’s definitely correct, but it had a bunch better chance of being correct than your average alt-covid-cure scribbled out in crayon. So I did some checks on it.

[Didn’t you say the risk of long covid was small? NO I SAID IT WAS TOO SMALL TO MEASURE AGAINST THE DELUGE OF CRAP THAT HAPPENS TO US EVERYDAY THAT IS NOT THE SAME]

*ahem*

This post is organized as follows:

  • Description of theory. 
  • Long section defining terms. These are all useful for understanding the claims I check later on, but depending on who you are they may not be helpful, and you may find the contextless infodump kind of a drag. Feel free to skip if it’s not useful to you personally, and know that it’s there if you need it.
  • Deep dive onto particular claims the article makes.
  • Does it work?
  • Is it safe?
  • My personal experience with the protocol 
  • Some meta

This is your reminder that my only credential is a BA in biology and I didn’t specialize in anything relevant. It is a sign of civilizational inadequacy that this post exists at all, and you should think really hard and do your own research before putting too much weight on it.

For those of you would like to skip to the take home message: science is very hard, I’m glad they’re running larger studies to follow up on all of these because that’s a reasonable thing for a rich society to do, but I’m not super hopeful about this protocol.

The Theory

As described by Dr. Ade Wentze:

There is an extremely widely used coenzyme in your body, NAD. The more active form of this compound, NAD+, is depleted by covid (converted to NADH). In people with a preexisting deficiency or difficulty rebounding after depletion, covid infection results in a persistent NAD+ deficit. This is bad in and of itself, but causes additional problems when your body tries to make up for it by requisitioning all your tryptophan to make more. Tryptophan is also a precursor for serotonin, so this leads to either low serotonin or activation of mast cells to release their serotonin stores, accompanied by histamines (which cause allergies and other issues). 

Background

There is a lot of vocabulary in that theory and in the supporting claims, which I go over here. If you’re reading for conclusions rather than deep understanding I would skip this.

NAD+

Nicotinamide adenine dinucleotide is a coenzyme that plays an essential role in hundreds of chemical reactions in your cells, including many relating to processing energy and genetic transcription.  This is a mixed blessing as a foundation for crackpot theories go: something involved in hundreds of processes across every kind of tissue in your body can cause almost any symptom, which is great because long covid has a lot of symptoms to cover. On the other hand, it can cause almost any symptom, which means it’s hard to disprove, and you should distrust things in proportion to the difficulty to disprove them. Alas, sometimes core processes are impaired and they do express that impairment in a range of unpredictable ways that vary across people, but it’s also an easy home for crackpots. 

NAD+ has two major components, one made from either tryptophan or aspartic acid (both amino acids), or by altering niacin.

Niacin

Like many vitamins, niacin aka vitamin B3 refers to a few different closely related compounds (most commonly nicotinic acid, nicotinamide, nicotinamide riboside, and inositol nicotinate, but there are others) that are almost but not quite interchangeable.

Chemical structures of niacin compounds: (a) nicotinamide; (b) nicotinic acid; (c) nicotinamide adenine dinucleotide (NAD þ ); (d) nicotinamide adenine dinucleotide phosphate (NADP þ ) (source)

Niacin is commonly prescribed for treating high cholesterol, although a metareview found it did not reduce overall mortality and may contribute to the development of type-2 diabetes. 

Severe niacin deficiency is called pellagra, and can be caused by either insufficient consumption or problems processing the vitamin. Pellagra is mostly defined as niacin deficiency but can also be caused by tryptophan deficiency, which you may remember is another path to manufacturing NAD+. Pellagra can cause diarrhea, dermatitis, dementia, and death, which are not a great match for acute or long covid. Niacin supplementation treats pellagra, often within a few days.

SIRT1

Sirtuin 1, also known as NAD-dependent deacetylase sirtuin-1, is a protein that regulates the expression of some genes in ways that haven’t yet been made clear to me but seem to be associated with aging (more SIRT1 is associated with better outcomes, although we haven’t broken down cause and effect). As indicated by its name, it’s dependent on NAD+ to operate, which means NAD+ is involved in the regulation of expression of some genes via some mechanism, which means niacin is involved in the regulation of expression of some genes via some mechanism.

SIRT1 is downregulated in cells that have high insulin resistance and inducing its expression increases insulin sensitivity, suggesting the molecule is associated with improving insulin sensitivity.

SIRT1 may be upregulated by selenium.

PARP

Another many-purposed enzyme whose activities include DNA repair, killing cells that are beyond repair. PARP requires NAD+ as a coenzyme.

Individual Claims

Groups with low NAD+ suffer more from covid

NAD+ declines with age

NAD+ does definitely decline with age but so does literally everything bad in your body, so I don’t find this very compelling.

Correlation between NAD+ levels and Age in (A) Males (B) Females (source)

Obese people have lower NAD+ levels, leading to worse outcomes

Yes, although obese people tend to do worse on a lot of metrics. However, that paper highlights that SIRT1 seems to be involved in this correlation somehow.

Diabetics have worse NAD+ levels

Yes, although diabetics also have more immune problems generally (definitely Type 2, some pop sites said the same for Type 1 and that’s believable but I didn’t quickly find a paper I liked that backed the claim).

Low selenium is associated with bad outcomes in covid

The post cites Zhang et al, which took advantage of high variations in selenium consumption in China to do a natural experiment. Variations in the population selenium levels do seem insanely correlated with the overall cure rate (defined as not dying). The study took place in February 2020 so neither data collection nor treatment was very good, but damn that is interesting.

Moreover, this study, which came out several months after the blog post was published, took advantage of the same variation and came to the same conclusion, with a much larger sample size and much more reasonable case fatality rate (1.17% in areas with no deficiency to 3.16% in severely deficient areas, P = 0.002). (Note: several authors on that paper are also named Zhang, but I assume that’s because it’s a common name in China).

Some pharma company thinks selenium is promising enough to launch a trial for it, although recruitment hasn’t started yet.

The pre-print servers are littered with natural experiments highlighting correlations that failed as interventions, but this is very strong for a correlation.

Niacin just generally seems to help lung damage

That is indeed what their citation says, however that paper’s only source looked at the effect of niacin on lung damage in hamsters deliberately induced with a chemotherapy drug, and it’s not obvious to me that that translates to damage from infection or immune reaction. There are some other scattered studies in rodents, combining niacin with other substances, none of which looked at damage from infectious disease.

The treatment for NAD+ deficiency is niacin

Their citation backs this up: niacin supplementation led patients (n=5) and controls (healthy people given the same supplementation, n=8) to increased NAD+ levels, and arguably increased strength, although with that much variation and such a small sample size I’m not convinced. Martens et al supports this with modest benefits seen in n=24 subjects.

A few minutes investigation found some other studies:

  • Dietary niacin deficiency led to NAD+ deficiency in baby rats. This paper works damn hard to hide its sample size but I think it was 10-15 per treatment group.
  • The same author exposed some rats (n=6 per treatment group) to excess oxygen and found that those with a niacin deficient diet had less NAD+ in the lungs and responded less to the damage caused by excess oxygen, but had the same wet/dry ratio as their well-fed friends (wet/dry ratio is a measure of lung health).
  • Ng et al found that in catfish liver NAD increased linearly with dietary niacin supplementation, but health returns like size and mortality dropped off between 6 and 9 mg/kg. They further found that tryptophan supplementation could not make up for a niacin deficiency (in catfish).

Plus niacin is so well established as a treatment for pellagra that no one bothers to cite anything for it, and that does seem to mediate through NAD+.

Nicotinic acid may act as a one of a kind bioenergetic “pump” of inflammatory molecules out of cells

They link to a preprint which has since been taken down, and I could not find it on my own. 

NAD+ problems have been indicated in chronic fatigue syndrome

Everything has been indicated in chronic fatigue syndrome; I’m not looking this up.

Low serotonin -> mast cell activation -> histamine release

Mast cells indeed produce serotonin, in mice. Note that that paper highlights fluoxetine as a way to reverse serotonin deficiency in mast-cell-deficient mice, and since the article was published fluoxetine has shown promise as a covid treatment. However this study says that while serotonin-producing mast cells are common, humans in particular don’t have them while healthy (although it still shows serotonin affecting mast cell movements). This appears to be an area of some controversy.

Mast cells releasing histamine in response to allergens is uncontroversial. Moreover, histamines and serotonin are stored in the same compartments (in mice). Second source (still in mice). 

Some Guy did an informal study based on this theory and it worked

Some guy (Birth name: Gez Mendinger) did indeed report this, and I have to say, for an uncredentialed dude on youtube recommending OTC supplements to treat a nebulously defined disease, this guy looks really credible, and his reasonably good analysis was quite promising. He shared his results with me, and it continued to look promising when I first dug into it with assistance from a statistician, but the deeper we drilled the less promising it looked (details). By the end, the most I could say is “yeah, worth a harder look”, but the history of things that look promising in small, poorly organized studies that wilt under large, well-organized ones is just too dismal to ignore. 

Mouse study shows low NAD+ hurts you via SIRT1

The interview also cites this mouse study featuring a direct NAD+ drip and a slightly different coronavirus. They show improved symptoms but not viral load. They don’t list the sample size anywhere I can find, judging from the low-resolution graph it looks like 7 mice in the control group and maybe 12 in the treatment group? Except for the embolism test which had many more mice.

(apologies for poor image quality, the PDF was crap)

(note: that article was up when I started this post but disappeared before I verified the SIRT1-specific part of the claim)

Quercetin increases NAD+ levels 

Yes, in rats and mice. Specifically, it speeds up the transition from NADH to NAD+

Male pattern balding and low vitamin D are both associated with poor covid outcomes and low NAD+.

The balding citation does indeed say that, but it only looked at hospitalized patients so it’s useless. Moreover, balding is associated with a testosterone derivative, and testosterone weakens the immune system. But when I went to find some cites for those, I found that within hospitalized patients, low testosterone was associated with worse outcomes. However these patients were already hospitalized, so the causality could easily go the other way.

Meanwhile I found several folk-wisdom level comments indicating a link between NAD+ and male pattern balding, but nothing rigorous.

Low vitamin D does seem to be associated with poor covid outcomes, maybe, but treatment doesn’t seem to help (at least not if you wait until patients are hospitalized). 

Chang and Kim assert that Vitamin D activates the NAD-SIRT1 pathway in fat cells in vitro, which if it held up elsewhere would be even stronger evidence for the overall theory than this claim attempts. Byers et al found that vitamin D did not protect guinea pigs against the NAD+ depleting effects of mustard gas. This is not a slam dunk.

Covid depletes NAD+ by activating PARP

Curtin et al lay out a theoretical case for using PARP-inhibitors to treat covid-caused ARDS.

Heer et al “we show that SARS-CoV-2 infection strikingly upregulates MARylating PARPs and induces the expression of genes encoding enzymes for salvage NAD synthesis from nicotinamide (NAM) and nicotinamide riboside (NR), while downregulating other NAD biosynthetic pathways” (notably, the forms not used in the protocol), “overexpression of PARP10 is sufficient to depress cellular NAD and that the activities of the transcriptionally induced enzymes PARP7, PARP10, PARP12 and PARP14 are limited by cellular NAD and can be enhanced by pharmacological activation of NAD synthesis”, “MHV induces a severe attack on host cell NAD+ and NADP+.” (MHV being used as a model)

Long covid and Pellagra share a lot of symptoms, including hyponosmia

Scattered claims pellagra causes hyponosmia but you have to look really hard, it doesn’t show up on any of the common descriptions. I checked in Spanish and didn’t find anything either.

Sen (published only last month) suggests that serotonin deficiency causes anosmia and other neuro symptoms in covid. They propose a different method for the depletion (ACE2 is a mechanism for moving serotonin into the cell), but it’s not mutually exclusive with Wentzel’s theory (that NAD+ depletion causes the body to use up tryptophan trying to produce more NAD+).

Your body hijacks tryptophan to make NAD+ at the expense of serotonin

Tryptophan can indeed be used to make NAD (albeit niacin is better) and serotonin. How your body prioritizes under a given set of circumstances is anyone’s guess.

NAD+ and the immune system

Probably at least some of long covid stems from autoimmune issues, as witnessed by the fact that it’s much more common in women and sometimes helped by steroids. The post and paper don’t make any claims on this beyond the effect of NAD+ on mast cells, which are implicated in autoimmune disorders, but out of curiosity I did some quick googling and found that NAD+ downregulate inflammation via CD4 cells (in mice) and activating SIRT1, the pathway mentioned previously (still in mice).

The Paper

Not that good. Feels associational rather than mechanistic. However Bordoni et al (published after the cited paper) found covid-19 was associated with diminished SIRT1- but Pinto et al found covid-19 upregulated SIRT1 and cite another study claiming that under conditions of energetic stress (which would imply low NAD+), SIRT1 substitutes for ACE2 (the receptor covid uses to enter the cell. Smith suggests that downregulating SIRT1 is good for fighting covid. So SIRT1, NAD+, and covid are probably related, but the first two items are very common so this isn’t damning.

Notably, this paper doesn’t explain why covid would deplete NAD+ more than other infectious diseases, which is an enormous hole.

Does it work? 

The mechanism and empirical data are definitely enough to merit more rigorous follow-up studies (which are in progress) and definitely not slam dunks. But you may need to make a decision before that’s in, so the real question is “should I take this stack if I get sick? Should my parents?”

My tentative answer is: the prescribed stack probably won’t physically hurt you (but see the next section), and it’s fairly cheap, so the limiting factor is probably “what do you have the energy to try”. This is a better thing to try than the interventions whose proof was actively made up or have been investigated and discarded, but there undoubtedly are or will be equally probable things floating around, and choosing between them will be a matter of taste..  

If you do end up giving this a shot, for covid long or acute, I invite you to preregister your complaints and intention with me (a comment here or email elizabeth@acesounderglass.com), so I can create my own little study. If you don’t feel like doing that I still encourage you to announce the intention somewhere, as a general good practice (I did so here). 

So you’re saying it’s safe then?

Anything that does anything is dangerous to you in sufficient dosages. If you’re considering an unverified supplement stack, you should carefully investigate the potential side effects of each substance and consider it in light of what you know of your own health (especially other medications you’re taking). Consider talking to a doctor, if you have a good one.

If any of you are thinking “oh niacin’s a water-soluble vitamin it must be fine”: that’s a pretty good heuristic but it doesn’t hold for niacin in particular.

My experience

As mentioned previously, I acquired lingering progressive chest congestion/inflammation from (probably) my covid vaccine. It’s always possible there was another reason but the timing and symptoms really do not match anything else. 

Since I never had covid (probably), my reaction can’t come from the infection itself, only my immune response to it. Since the theory doesn’t specify a mechanism that’s not disqualifying, but they do make it sound like it starts as a covid problem not an immune problem.

I started this supplement stack before doing any deep verification. The original blog post pattern matched to the kind of thing that was worth trying, everything on the list I either knew was generally safe or confirmed with a quick check (my doctor later confirmed my opinion on safety without endorsing the stack for any particular use), and I had a lot of client work to do. Shoemaker’s children go barefoot, and all that.  So by the time I was writing this I had been on the recommended supplement stack (and some other things besides) for 3 weeks, and was beginning to wean down. 

Overall: my chest pain got better but the timing fits better with attribution to a different intervention. The rash I got on matches very well with the supplement stack. I nonetheless was craving it after I weaned off, so probably there’s at least one thing in it I need, which hopefully isn’t the same as the thing causing the rash. 

[Alert twitter readers may have questions, since I previously was more positive on the stack. I had a major regression when I got a non-covid cold, and had to go back on the other treatment]

Interestingly, my tolerance for niacin increased and then plummeted. Originally I could take 250mg (the smallest size I could find in the right form) with only very mild flush, and that got better over time, to the point I tried 500 mg once (a mistake). But around week 3 my flush was getting worse. Lowering the dose helped, but it’s getting worse again, so I’m continuing to titrate down. This is extremely consistent with filling up NAD+ reserves over time, although very far from conclusive.

Meta

I was originally much more positive on this treatment/theory. I gave it more credit on Twitter, but that’s nothing compared to the excited messages I sent a few friends after an initial lit review. I wrote several much more positive versions of this post (and the forthcoming study analysis), but there kept being one more thing to check, until I talked my way down to what you see here. Some of my downgrade stemmed from asking better statistical questions, but some of it was just the emotional process of talking myself down from something that initially looked so promising, but ultimately had a similar amount of holes to many other things that looked equally promising and failed to pay off. This represents dozens of hours of work from me and my statistician, for the very disappointing result of “fringe treatment probably doesn’t do very much but can’t rule it out”. Reality is infinitely disappointing

Thanks to Alex Ray and my Patreon Patrons for partially funding this investigation, and Miranda Dixon-Luinenburg⁩ for copyediting.

“Eating Dirt Benefits Kids” is Basically Made Up

Sometimes people imply that epistemic spot checks are a waste of time, that it’s too easy to create false beliefs with statements that are literally true but fundamentally misleading. And sometimes they’re right.

On the other hand, sometimes you spend 4 hours and discover a tenet of modern parenting is based on absolutely nothing.

[EDIT: this definitely was a tenet among my friends, but apparently is less widespread than I thought.]

Sorry, did I say 4 hours? It was more like 90 minutes, but I spent another 2.5 hours checking my work just in case. It was unnecessary.

Intro

You are probably familiar with the notion that eating dirt is good for children’s immune systems, and you probably call that Hygiene Hypothesis, although that’s technically incorrect. 

Hygiene Hypothesis can refer to a few different things:

  1. A very specific hypothesis about the balance between specific kinds of immune cells.
  2. A broader hypothesis that exposure to nominally harmful germs provides the immune system training and challenge that ultimately reduces allergies.
    1. One particular form of this involves exposure to macroparasites, but that seems to have fallen out of favor.
  3. The hypothesis that exposure to things usually considered dirty helps populate a helpful microbiome (most often gut, but plausibly also skin, and occasionally eyeball), and that reduces allergies. This is more properly known as the Old Friends hypothesis, but everyone I know combines them.
  4. Pushback on the idea that everything children touch should be super sanitized
  5. The idea that eating dirt in particular is beneficial for children for vague allergy-related reasons.

I went into this research project very sold on the Hygiene Hypothesis (broad sense), and figured this would be a quick due diligence to demonstrate it and get some numbers. And it’s true, the backing for Hygiene and Old Friends Hypothesis seems reasonably good, although I didn’t dig into it because even if they’re true, the whole eating dirt thing doesn’t follow automatically. When I dug into that, what I found was spurious at best, and what gains there were had better explanations than dirt consumption.

This post is not exhaustive. Proving a negative is very tiring, and I felt like I did my due diligence checking the major books and articles making the claim, none of which had a leg to stand on. Counterevidence is welcome. 

Evidence

Being born via c-section instead of vaginally impoverishes a newborn’s microbiome, and applying vaginal fluid post-birth mitigates that

This has reasonable pilot studies supporting it, to the point I mentioned it to a pregnant friend.

There are reports that a mother’s previous c-sections lower a newborn’s risks even further, but I suspect that’s caused by the fact below

Having older siblings reduces allergies

Study. The explanation given is a more germ-rich environment, although that’s not proven.

Daycare reduces later allergies, with a stronger effect the earlier you enter, unless you have older siblings in which case it doesn’t matter

Study. Again, there are other explanations, but contagious diseases sure look promising.

Living with animals when very young reduces allergies

This one is a little more contentious and I didn’t focus on it.  When the animal appears seems to matter a lot.

One very popular study used to bolster Dirt Eating is a comparison of Amish and Hutterite children. Amish children get ~⅙ of the allergies Hutterite children do, which pop articles are quick to attribute to dirt “because Amish children work on farms and Hutterite children don’t.” But there are a lot of differences between the populations: dust in Amish homes have 6x the bacterial toxins of Hutterite homes, the children have much more exposure to animals, and drink unpasteurized milk. 

Limitations of Farm Studies

Even if Amish children did eat more dirt and that was why they were healthier, there’s no transfer from that to urban parks treated with pesticides and highway exhaust. They might be net positive, the contaminants might not matter that much, your park in particular might be fine, no one has proven this dirt is harmful, etc. But you should not rest your decision on the belief that that dirt has been proven beneficial, because no one has looked.

Mouse Studies

There are several very small mouse studies showing mice had fewer allergies when exposed to Amish dirt, but:

  1. They are very small.
  2. They are in mice.
  3. The studies I found never involve feeding the mice dirt. Instead, they place it in bedding, or directly their nasal passages, or gently waft it into the cage with a fan. 

So eating dirt is bad then?

I don’t know! It could easily be fine or even beneficial, depending on the dirt (but I suspect the source of dirt matters a lot). It could be good on the margin for some children and bad for others. Also, avoiding a constant battle to keep your toddler from doing something they extraordinarily want to do is its own reward. What I am asserting is merely that anyone who confidently tells you eating arbitrary dirt is definitely good is wrong, because we haven’t done the experiments to check.

I think any of [communicable diseases, animals, unpasteurized milk] have more support as anti-allergy interventions than dirt, but I hesitate to recommend them given that a high childhood disease load is already known to have significant downsides and the other two are not without risks either.

Epilogue

The frightening thing about this for me is how this became common knowledge even, perhaps especially, among my highly intelligent, relatively authority-skeptical friends, despite falling apart the moment anyone applied any scrutiny. I already thought the state of medical knowledge and the popular translation of that knowledge was poor, but somehow it still found a way to disappoint me.

My full notes are available in Roam.

This post was commissioned by Sid Sijbrandij. It was preregistered on Twitter. I am releasing it under the Creative Commons Attribution 4.0 license. Our initial agreement was that I would be paid before starting work to avoid the appearance of influence; in practice I had the time free and the paperwork was taking forever so I did the research right away and sat on the results for a week.

Thanks to Miranda Dixon-Luinenburg⁩ for copyediting.