Lessons learned from offering in-office nutritional testing

Introduction

I’ve talked previously about my concerns with nutritional deficiencies in effective altruists who go vegan for ethical reasons, especially those who don’t have a lot of contact with the broader vegan culture. No one else seemed very concerned, so I launched a tiny project to test people in this group and see if they were in fact deficient. This is a report on the latest phase of the project. 

To cut to the chase:

  • It was very easy to find lots of deficiencies, although due to a severely heterogenous sample and lack of a control group this doesn’t provide useful information about if veganism is at fault.
  • Finding these deficiencies probably leads to useful treatment, but not as much as I’d hoped.
  • There are still a lot of operational issues to work out. My guess is that the ideal would require more work (to encourage participants to act on their results) or less (by focusing on education but not providing testing). 
  • I am currently looking for a co-founder to properly investigate the impact of veganism on nutrition. 

My main question here was “is there low-hanging fruit in treating nutritional deficiencies in this group, and if so how do we pluck it?” An important part of that is “how prevalent are deficiencies?”, but I had substantially more uncertainty around “do people treat deficiencies you find?” and “does the treatment lead to improvements in anything we actually care about?” That prioritization (and budget issues) led the experimental design to focus on operational issues and outcomes, and deprioritized getting the kind of clean data that would let me compare vegan and non-vegan outcomes. Similarly this write-up is mostly focused on showing the problem exists at all and building metis of investigation and treatment, rather than estimating prevalence. 

Which is to say to to everyone planning on @ing me to complain about the sample size, heterogeneity, or mediocre statistics: you are right that this sample is not very informative about base rates of deficiencies in vegans or anyone else. If someone claimed it was, they would be committing an epistemic sin. However, this particular post is focused on “how much effort is it to get nutritional issues addressed, and is that effort worth it?”. Given that, any complaints about the terrible sampling will be considered to be offers of assistance in running the much larger study that could answer the prevalence question

Background 

(if you’ve read the previous posts, this will be review)

Last year I worked in a co-working space focused on existential risks, which is anything that might really end everything. Because x-risk is a popular topic within the effective altruism movement, many participants in the space were EAs. Another big topic within effective altruism is animal suffering, especially farmed animal suffering. This has led many EAs in the office to go vegan or at least vegetarian for ethical reasons, without making animals the focus of their lives. And when I asked them, many had put no thought into how to give up animal products in a healthy way.

Which of course would have been fine if they were eating well. My personal opinion is that most people’s optimal diet contains small amounts of animal products, but lots of people are eating suboptimally for lots of reasons and I don’t consider it my problem. But the number of fatigue and concentration issues were… I don’t actually know if they were high. I don’t know what baseline to compare them to. But it wasn’t low enough to reassure me. Neither did the way the vegans talked about nutrition, and in particular the fact that more than one person was doing keto vegan and still wasn’t investigating nutrition. 

None of this meant there was necessarily a problem, but I was suspicious. So I got a small grant to solve this with numbers. 

First I tried broad spectrum testing, which led me to identify iron deficiency as the main concern.

Then I did a short investigation into the actual costs of iron deficiency, which motivated a number of people to get tested, some of whom got in touch after the fact. 

In my last post I implied I was stopping the project, and wrote next steps as if someone else would be doing them. I ended up getting a follow on grant so pushed forward with round two on my own, which is what this post is about. 

Round 2 concrete steps

I followed the steps laid out in my previous post almost exactly.

  1. I brought in a company to draw blood from participants in the Lightcone office. This was expensive, probably 2-4x having people buy lab orders online, which is still more expensive than doctor-ordered tests with insurance.
    1. I say I brought them in – I in fact hired someone to handle the company and interface with participants. Aaron Silverbook was great, pretty much best case scenario for hiring a subcontractor, although still not as good as an invested co-founder.
  2. Based on previous results and nutritional folk wisdom my test priorities were ferritin (the best proxy for iron), B12, and Vitamin D. I threw in some other iron tests because they were very cheap.
  3. We had applicants fill out a form detailing their diet and any fatigue issues.
  4. Aaron picked 20 participants and made a schedule for testing them. Prioritization was: Fatigue issues > vegan > vegetarian > whoever happened to be in the office that day.
    1. This was my favorite part of having a contractor because scheduling involved some very intense math around prioritization and availability and from my perspective it was no work at all.
    2. Prioritizing people with fatigue issues probably made the immediate impact higher, and was useful for estimating the upper bound of possible impact, but ruined the sample for calculating base rates. It means I can’t really compare vegans vs. omnivores, because both were selected for having potentially-nutrition-caused problems. I had hoped to maybe compare vegans with fatigue to omnivores with fatigue, because the tired vegans had more nutritional issues that would be pretty suggestive, but ultimately didn’t have enough data to bother.
    3. I flat out didn’t have enough money to do a good study capable of establishing high confidence prevalence rates, even if that had been my primary goal. Putting aside sample size, good sampling requires getting representative participants, many of whom have no reason to get tested. They’re not intrinsically motivated, so you have to pay them to get tested. 
  5. The company comes into the office. There are a bunch of day-of headaches that Aaron handles beautifully and I am completely uninvolved with because he is good at his job.
  6. Results go out. Some participants received results over email, some were told to go to the company’s portal, some may never have gotten a notification at all. More on this in the Difficulties section.
  7. I ask Aaron to get results from participants to me, for analysis. This goes less well- I wasn’t sufficiently clear on what information I wanted in the form, and he’s gotten busy with other stuff and had less time to devote to my project, which has already gone longer than anticipated due to issues with the lab. Response rate is poor- we eventually get 13 people out of 20.
  8. I send out a second form asking for more information, and a bunch of emails harassing people to do both forms. Response rate continues to be poor, only 8 people this time. 
  9. 3 months after testing I followed up with the people with the worst scores to see if they had gotten treatment.

Project Difficulties

Other People’s Fault

I cannot say enough bad things about BayAreaPLS, the company we hired to come into the office and do testing.

First and most importantly, they just forgot to run the ferritin test for half the participants. They did the other tests, there was no reason to not do that one in particular, they just… didn’t. Their initial attempt to make up for this was an offer to not charge me for tests they didn’t do. I pushed back fairly hard and they agreed to some actual discounting. It’s been 2.5 months and they have yet to send me that follow-up invoice, which I guess is technically good for me but makes me feel worse overall.

Second, the results were hard to retrieve. Some participants never got results at all even after following the instructions, and I have no way to debug whose fault that is. Others didn’t bother to retrieve results because it was too hard. The happiest people were the ones who got their results over email, which is a HIPAA violation the company claims to have done by accident. Much like the lack of the second invoice, the emailed results are technically good for me but leave me more concerned about the company.

Even for people who did get results it took almost two weeks, which was not great for momentum.

Lastly the company used different deficiency thresholds for different people. They say this was based on sex and age, but they didn’t get that information from everyone (sounds like incompetence on their part), so some people got a list of thresholds. It ended up creating a moderate amount of confusion and friction, especially because I couldn’t tell what variation came from age/sex (which I mostly want to ignore) vs. the norms of a particular test (which I very much don’t). People are slow to act on results in general so the additional friction was quite costly. 

My Fault or Inherent Difficulties

Not all of these were literally caused by me, but they all fall under “my responsibility to anticipate and handle”.

I mentioned in my last post that some participants were kind of insistent on getting a lot of help from me, even after I explicitly told them something was outside my bailiwick and needed a doctor. I tried to fix that this time by hiring Aaron. That worked on that one issue, but made it harder to catch new problems. If the plan had been for people to directly show me their results and receive coaching I would have caught the missing test values much earlier. I’m not sure there is a way to have someone available enough to get all the relevant questions from participants without also having to deal with a bunch of irrelevant ones, including inappropriately persistent ones. The difference between the two just isn’t obvious with the knowledge most participants have, and emotions run so high around health stuff.

It’s hard to estimate the rate of acting on results because the people who don’t act are less likely to fill out the follow-up surveys, but my sense is it’s not good, and probably <50%. I also strongly suspect the rate of acting on results would have been higher if there had been in-person follow-up. 

The guidelines I sent participants emphasized vitamin D and ferritin because it didn’t occur to me anyone could see an anemia result on the page and not rush to treat it, but at least two people scored as anemic and, as of three months later, had not treated it. 

The lab obeyed HIPAA enough to only send results to participants, not directly to me, so I needed participants to forward them. Of 20 participants, 13 did so. Only 8 participated in the separate follow-up questionnaire (arguably my fault for asking for two separate forms, but there was good reason to ask for results quickly and do the follow-up a little later). 20 people with varying motivations for testing was never going to be strong evidence for anything systemic, but the low response rate makes it even harder to draw conclusions.

My emails to participants sometimes went to spam, possibly due to use of BCC, which was necessary to meet the privacy commitments I made in the application.

What were the total costs? 

As a ballpark:

  • ~$270 per participant (if they had done all the tests for everyone).
  • 0.5 hours per participant, including follow-up and the disruption to their schedule. If you want to be really conservative you could call this an hour to account for the disruption from transitions.
  • ~10 hours of Aaron’s time
  • ~20 hours of my time
  • $30 per deficient person for supplements (not covered by the study)

If you value everyone’s time roughly the same, that means that to break even we need to save one person 30 hours + ~$5700 (ignoring the information gained). 

If you want to complicate that math you can add any of: discount rate on time or money, exchange rate between ops time/my time/participant time, cost of unnecessary or counterproductive treatment, knowledge gained from this round can make the next round cheaper, the fact that most of my time went to a write-up that shouldn’t really be billed to participants, but a better program would have spent more time with participants.

Results and Benefits

Test Results

I should remind you here that the sample was a mix of 20 ethical EA vegans, vegetarians, people with fatigue issues, and people who happened to be in the office. Even a very large sample wouldn’t be perfectly predictive unless you had the exact same mix of participant types, and this sample was tiny. So the right way to look at these results is “is this enough to think there’s a problem?” and “did people do helpful things with the information?” not “at what exact rate does this population develop ferritin issues?”

With those caveats, here is the data from participants that reported back. I have ferritin results for 8 people and all other results for 13. 

  • 85% reported energy problems
  • 15% were honest-to-god anemic
  • 65% had low ferritin (30% clinically deficient) (none of the anemic people had ferritin tests done, so there is no overlap between this group and the anemics. There were non-anemic people without ferritin tests, so I assume this is a coincidence)
  • 60% had low vitamin D (15% clinically deficient)
  • A total of 80% had low scores in at least one of hemoglobin, ferritin, or vitamin D
  • 0% had low B12 (many were on supplements, but I haven’t correlated that with serum B12 levels because at this sample size and heterogeneity there is no point)

Some of the non-reporting was random due to the lab’s incompetence, but it’s not impossible unhealthy participants were more likely to report back. If you want to be extremely conservative and assume every missing value was A+ healthy, the results are still quite concerning: 

  • 10% anemic
  • 25% low-in-my-opinion ferritin, 10% clinically deficient (still no overlap with the anemics)
  • 35% low-in-my-opinion vitamin D, 10% clinically deficient
  • B12 is still great, good job everyone

I Lead This Horse To Water – You Won’t Believe What Happened Next

But finding results isn’t very meaningful if no one acts on them. Of the 8 people who filled out the follow-up survey, 75% changed either diet or supplements, and 1 additional person kept going with supplements they would otherwise have dropped. I assume this is overreporting because people who changed things are more likely to respond, but if you assume none of the nonresponders did anything that’s still 30% of people changing something.

Of the 5 who changed something and answered the relevant question, 40% said they thought they saw an improvement (~1 month after they received results). I don’t consider that particularly strong evidence in either direction- it can take time for deficiencies to heal, but it’s also easy to placebo yourself into seeing an improvement that isn’t there. The real test will be the six-month follow-up. 

Three months after testing I followed up with the two identified anemics to make sure they were getting treatment. Neither was, despite having health issues plausibly caused by anemia. They’ve both indicated vague plans to follow up now that I’ve pushed them on it. 

Is in-office testing worth it?

I believe this round of testing was better than not doing in-office testing, but there is a lot of room for improvement.

My absolute wild-ass-guess is that this saved between one and ten people (out of twenty participants) from anemia or a moderate iron deficiency, and this improved their life and productivity by 10% to 100% (mean around 20%). I acknowledge these are large ranges, but some problems are bigger than other problems. I’m ignoring vitamin D entirely here because I haven’t even attempted to quantify its value and its ardent fanclub has poisoned the literature.

Even in the worst case, a 5% chance at a 10% improvement is a big deal, so I think this was obviously worth it from a participant perspective. I think it’s a toss-up if it would be worth it for apparently healthy omnivores: my expected value for them is much lower, but people don’t always realize they’re operating at a deficit and catching them requires testing actually-healthy people as well.

I said above that we needed to save someone 30 hours + $6000 for the project to break even. Even one successfully treated anemic will blow that out of the water, so I don’t feel the need to do the more complicated math with discount rates and relative value of time, especially because any future round should either be less work or have a higher response rate.

Of course “break even” isn’t a very high bar. To go even further out on a limb: the median case of mild anemia easily costs someone two hours/day (source: had anemia one time).  This testing easily caught the anemia at least six months before it would otherwise have been caught (because everyone who was going to get tested on their own did so when I published my iron post).  That is, at a bare minimum, 360 hours someone otherwise wouldn’t have had. That’s a pretty great return rate for 40 hours (my time + Aaron’s time + participant time) +$6000.

Was the project overall worth it?

I would bet on yes, although a lot of information has yet to come in. 

I expect this project to have 6 lasting impacts:

  1. The treated health problems of participants.
  2. The secondary impact via participants’ work. 
  3. Public blog posts I write. My iron deficiency post, which motivated many people to get tested themselves, quite possibly more than received tests directly.
  4. Knowledge of how to do this more in the future. 
  5. Influence on Effective Altruism vegan culture.
  6. Animal suffering averted by making veganism more sustainable for participants.

#1 is what I calculated above, and think was already a sound success although not resounding.

#2 and #3 require making assessments of all individuals who received testing from the project or due to blog posts I wrote. That’s a combination of “vast amounts of missing information” and “judging individual merit” that makes it really uncomfortable to talk about in a public post. 

To be totally honest I’m on an “everything is bullshit” kick right now so deep in my soul I don’t think this paid off, but intellectually I think my standards are too high and this was a better project than average project in the space of existential risk. 

#4 and #5 depend on other people following up on this project. I absolutely believe they should, for all the other reasons but also because the return via #6 seems pretty good- health reasons are a common reason vegans go back to eating meat [It was hard to find a good source, but this shitty source says 26%. I’ll acknowledge that’s probably an overestimate, since health is the most virtuous reason to go back to eating meat.]

I can’t estimate #6 myself. I’m not familiar enough with vegan literature to sort good from bad here, and it wasn’t my main goal.

Then there are costs. The total grant was for a little less than $25k. I didn’t track my time very closely to avoid depressing myself, but my compensation is going to work out to a fraction of my normal hourly rate. If you count foregone client work you could argue the true cost was as high as $50k. 

My gut feeling is the project was straightforwardly worth it if you don’t track the foregone work. If you do, impact is dependent on having at least one of:

  • At least one of the impaired and successfully treated participants goes on to do high-impact work.
  • The iron post inspires at least 40 tests total, with a similar rate of finding and treating problems.
  • Follow-up projects exist and do good work. 
  • This work leads to more veganism, and you value that a lot. 

So can we blame veganism for the deficiencies?

This study doesn’t say anything one way or the other, which means I still think yes but you shouldn’t change your opinion based on the results. The sample is too small and skewed to compare deficiency rates in vegans and nonvegans. There were energetic omnivores with deficiencies and tired vegans with perfect scores so it’s clearly not deterministic. 

Next steps

I see three possible follow ups to this project:

Nutrition blogging

This is my default, although I don’t plan on writing many of these because there is only so much low hanging fruit and people have a very limited attention budget.  I have to be very judicious in what I suggest.

Mass testing to investigate deficiencies in effective altruism populations

Get the money to test a large enough representative sample, and run the tests with a proper control to actually estimate the cost of nutritional veganism. 

This is most useful if there are EA vegan leaders who won’t act on nutritional concerns now but would if the study demonstrated a problem. If this is you, I would love to talk to you about what you would consider sufficient to act on. 

Assuming the demand for this information is there, I still don’t think I want to run this project alone. First, it is a lot of work. Second…I know I said “assuming demand is there”, but I can’t picture a scenario where demand exists but no animal EAs consider this project worth working on. A collaborator would be both proof of investment and much better positioned than I am to get the information acted on. 

To that end, here is an ad for a co-founder. I will post it on this blog in a few days.

In office testing with real nutritional counseling

This can work, but only in a limited number of situations. You need a reason  (uninformed veganism, high fatigue rates) to suspect nutritional issues in lots of people sharing a space. There needs to be a reason people aren’t getting tested themselves that won’t also inhibit follow up (probably lack of money and existing relationship with a doctor). And even then it’s more of a hits-based model than a sure thing.  

My decision is easy because the office I was working out of closed, and in general I think most of the people in the bay area I would want to help have already been reached. The market is saturated for at least a year. There are other offices elsewhere in the world, and if you want to run this yourself I’m happy to act in an advisory capacity (especially if you share data), but it can’t really be an ongoing project in any one city. 

Conclusion

I finished most of this post planning on it being the end of my part of the project. I had hopes I would convince someone else to pick up the torch, and maybe even act as an advisor, but it seemed like the biggest problem was participant motivation, which I don’t feel equipped to solve. It was while I was writing this that I realized I wasn’t ready to let the broader issue of vegan nutrition go. I still believe the problem that offended my morals and epistemics is there and worth acting on.

But doing so is still very annoying, which is why I’m looking for someone to partner with on this. Someone who can handle the parts I’m bad at, point out where I’m wrong, and interface with the vegan EA community to get the results acted upon. If you’re interested, please reach out to elizabeth@acesounderglass.com.

Thank you to the Survival and Flourishg Fund for funding this research, and Lightcone Infrastructure for hosting the grant and testing. I inflicted this draft on a number of people but want to especially thank Gavin Bishop. Daniel Filan didn’t beta read this post but he did vegan-check my co-founder ad and suggest the title “I Lead This Horse To Water – You Won’t Believe What Happened Next”.

Product Endorsement: Apollo Neuro

Short version: This $310 vibrating bracelet dramatically improved my sleep and moderately improved my emotional regulation. The return policy is pretty liberal so if this seems at all appealing I recommend trying it, or one of the cheaper alternatives I haven’t investigated. Between now and Mothers’ Day they are $300. 

[note: the link I use here is an affiliate link that gives you a $40 discount and me a $50 Amazon gift card] 

Backstory

As a strong believer in luck-based medicine, I have a pretty liberal threshold for trying shit Facebook advertises to me. Most of it is crap, but every once in a while there is something amazing that justifies all the work and return fees. Previous purchases include resistance band clips that measure force applied, sleep-safe headphones that claimed to measure your HRV, and an infrared heat massage. But when I first looked at the Apollo Neuro’s website, it was too dumb even for me. The explanation for how it worked was a mix of absent and stupid, and the rush to provide scientific evidence was somehow worse than nothing. I only tried it because a friend raved about it. He had also thought it sounded deeply stupid and only bought it because his friend raved about it, and I can only assume she also thought it was stupid until someone raved about it to her, in a great circle of life. Now it is your turn to be told it sounds stupid but it works.

[For some people. My sample size is only three people.]

Benefits to me

My sleep improved a lot. I don’t have precise metrics for this because fitbit is stupid, but 1-3 times a week I wake up feeling drugged (positive valence) because my muscles are so relaxed. This never happened before the neuro unless I took actual drugs*. I also estimate my number of remembered wake-ups has been ~halved. 

The Neuro has about 8 hours of battery. When I started using it I always woke up with the battery drained, meaning I’d activated it repeatedly. I now wake up with it at 70-90% charge, partially because I wake up less and thus am not turning it back on, and partially because I use less intense vibration.

[*TBF I regularly take supplements for sleep, so what I really mean here is “unless I took more drugs than baseline”]

My emotional regulation improved as well. There were a number of stressful things I handled better than my baseline. Some of this is subjective, but there are a few things with obvious before-pictures. Most notably:

I have pretty bad medical anxiety, due most notably to dental malpratice leaving me with painful nerve damage, but also some other stuff. Last fall I had a doctor’s appointment on Friday, followed by friend’s child’s medical emergency on Saturday that, due to their newness to the country, I needed to be in charge for. I did it, but I was wrecked for at least a day afterwards, possibly more, and my partner had to put in a lot of emotional energy helping me recover. ~4 weeks into using the Apollo Neuro I had a dentist appointment in the afternoon, followed 8 hours later by a friend’s medical emergency requiring my attention and eventually a 3AM field trip to the ER for which I was the only available driver. I did have a little freak out once I was home, but I recovered to normal faster than I did from the incident in the fall despite that having been a less intense day with more help. 

I also found myself more decisively liking and disliking things, and noticing when things shifted. At parties the transition from “I’m enjoying this” to “I’m done” is clearer, without a lot of “maybe I should hold out and it will get better”. 

Mechanics

The website is stupid, so let me tell you how it really works. The Neuro is shaped like a watch with a large rectangular face. You can wear it around your wrist or ankle with a band, or clip it to clothes. When activated, it vibrates with oscillating intensity. There are 7 programs with varying oscillation patterns and durations: the wake-up program lasts five minutes with a short peak and shifts between off and on quickly, sleep spends longer in both phases and shifts between them much more slowly. Sleep is the longest program but also lowers intensity over time. You can configure the peak intensity but not duration or pattern, which I feel very oppressed by. 

You can change intensity, and pause and restart the last program from the watch, but to choose a program you need the smartphone app. It is impossible to use the watch without a smartphone, which is a serious quality of life issue for those of us with insomniac older relatives. 

The website talks about cumulative effects a lot, and is clearly pushing you to try for several weeks before judging it. There’s even some gamification for the first N hours. This felt to me exactly as necessary as a punch card from a heroin dealer. I loved it from the moment I put it on and found the little badges cheapening of my relationship with my device. But the cumulative effects part was true: as previously mentioned I needed less and less work from the Neuro to sleep, and lowered the intensity setting over time. When I first started most of the programs besides sleep and wake-up ran together, but at 6 weeks in I started really distinguishing the other programs and having strong preferences about which program, which changed over time. 

The website also advertises the Apollo for concentration problems, but I don’t know anyone who’s really tested that.

One thing I want to give the Apollo is that there is no subscription fee. You give them money and they give you the whole product and app. That should be standard but very much isn’t in the as-seen-on-FB crowd.

Cheaper Alternatives?

The Neuro is very expensive and seems like it can’t possibly cost that much to manufacture. For me paying them for the R+D was worth it. It would even have been worth a second one at the same inflated price, had my sleep not improved to the point I didn’t need it. 

Back when I thought I would need a second one to cover a full night of sleep I looked around for cheaper alternatives. None of them quite worked and I gave up when I no longer personally needed it, but if you’re motivated some might be worth checking out. 

The Senate works on a similar principle, but is no cheaper. Its programs are only 10 minutes, which is much too short for me. 

I tried a few sexual vibrators but even those with intermittent patterns transition from on to off much too quickly, which is how I figured out the gentle transition is so important. Presumably there are some that transition gently but I have no idea how to search for them. 

There are apps to vibrate your phone but they are mostly ad-ridden messes I couldn’t deal with. The one I managed to test had the same problem as the sexual vibrators. 

Products aimed at babies: these mostly run on external batteries, but they can be pretty cheap and many don’t require a smart phone. If you try any of these let me know because the alternative is teaching my elderly aunt to use a smart phone and I am not looking forward to it.

Conclusion

This product is very much in “immensely valuable to some people, price and quality of life issues limit its market for now”. If you have anxiety or sleep issues I seriously recommend trying it; the return fee looks to be about $20, although they’re less forthcoming than one might hope. If money is an issue or you’re just feeling curious you could also try the vibrating baby soothers. Amazon sells several and returns are usually free. 

If you do try the Apollo or any other product in the category I’d love to hear from you so I can share the information (please decide in your heart if you want to share your results before you try it, to avoid biasing the data). You can report your experience here

Thanks to my Patreon patrons for supporting this write up, and J for suggesting the Apollo to me in the first place.

Long Covid Risks: 2023 Update

Back in 2021 I wrote a post estimating the risk of long covid. Recently a client hired me to do an update, focusing on changes induced by Paxlovid and vaccination. This was a <5h project and the literature wasn’t very rich so nothing I say here is conclusive, but nothing I said last time was conclusive either so let’s enjoy this together. 

Some caveats: 

  • I spent 5 hours on this, and that includes client-specific work I’m not including here.
  • Research that met my standard was really scarce; ultimately each conclusion is based on a single study. My goal was data that includes a large population not selected for having long covid, where reporting was automated so you don’t need to worry about response bias. In practice, this means I used data from large medical systems with integrated reporting, like the American Department of Veteran Affairs, national medical systems, and HMOs. Surveys from long covid support groups were ignored with prejudice. 

Summary

Vaccination helps, a bit: Given a medically diagnosed infection (which means it was serious enough to actually get you to the doctor), up-to-date vaccination lowers the risk of long covid by about 20% (this does not include the reduction in risk of having diagnosable covid in the first place, which is substantial). 

Paxlovid helps, more: Nirmatrelvir, which is one of two drugs that make up Paxlovid, reduces long covid risk by about 30% for medically diagnosed infections (which means it was serious enough to actually get you to the doctor). An optimist might hope the other drug (which is in the same class, although most commonly used as an adjuvant) is also useful and round this to 50%.

Most symptoms are temporary: Long covid does tend to get better over time, but how quickly depends on the symptom.  At one year post-infection, the rate of heart issues is nearly indistinguishable from controls, but cognitive issues have a 50% chance of persisting. 

Calculate your absolute risk: Your absolute risk depends on your age and comorbidities. The measured risk for 70-year-old men (not controlling for comorbidities) of developing at least one serious sequelae of medically diagnosed covid n (which means it was serious enough to actually get the patient the doctor) is ~12%. If you want to norm this for your own demographic, you can get a very crude estimate by entering your demographic information in this calculator, dividing your risk of hospitalization by 3 and multiplying the total by 0.4 (which includes the 20% reduction from vaccination and the 50% reduction from Paxlovid). If you are a cis woman, multiply by 2 to account for increased risk (trans people: I have no idea, if you find good data please let me know). 

I cannot emphasize enough how crude this is. I got that 3 by making up a 70 year old man with some common comorbidities, which has a risk of hospitalization of ~36%, and noticed 36/12=3. I don’t think The Economist has been keeping up to date with the latest strains of covid or even the impact of vaccination; these proposed calculations are strictly for order-of-magnitude estimates. 

Sample calculation: a 35 year old woman with no comorbidities shows a 3.8% risk of hospitalization (with their data, which I believe is very old). 3.8%/3= 1.3%. 1.3%*0.4= 0.5%. Times 2 for being female = 1.0%. So a covid infection bad enough to require medical attention has a 1 chance in 100 of a serious persistent issue post-covid. 

Studies

Nirmatrelvir and the Risk of Post-Acute Sequelae of COVID-19 (pre-print)

This study compared people who got covid and received Nirmatrelvir (half of Paxlovid). It used data from the American Department of Veteran Affairs, which means the participants are older (average age 65), overwhelmingly male (~90%), and very white (75%). Last time I checked maleness increases the risks of short-term covid consequences but decreases the risks of long term consequences, so good luck balancing that calculation.

The distribution of medication was not random. They don’t specify beyond this, but I assume VA doctors are more likely to aggressively treat patients who are sicker or have more co-morbidities, which should lead the study to understate the impact of treatment. Additionally they were only giving nirmatrelvir, which is one of the two drugs packaged together to make Paxlovid. I’m going to be an optimistic and assume the second drug was included for good reasons, which make this study underrepresent the usefulness of Paxlovid. But they don’t give the dosage at all, so there is a wildcard.

All that said: Nirmatrelvir was quite helpful, cutting the risk of long covid (PASC) at 90 days by ~25%, which in this group translated to 2.5 percentage points. 

Survival here means “survived w/o long covid symptoms”. You might ask why that goes down over time, given some people recover between days 30 and 90. I believe the answer is that they didn’t check for symptoms’ persistence: any diagnosis of long covid issues put participants in the PASC bucket forever.

Effect by symptom:

Long COVID after breakthrough SARS-CoV-2 infection

This is another study with VA data. They compared outcomes of infection after vaccination, compared with vaccinated controls. 

Participants with infections after vaccination (aka breakthrough infections) had a 12 percentage point increase in risk of symptoms in 12 areas, compared to vaccinated people who didn’t get infected. Again, the study population is probably at higher risk than average due to age and associated comorbidities.

However, this risk is heavily concentrated among hospitalized patients:

They also compared the risks to those of infections in unvaccinated people. Vaccination clearly helped, but not by as much as one would hope.

Just for fun, here’s the long-term risks of covid relative to the flu:

Long covid outcomes at one year after mild SARS-CoV-2 infection: nationwide cohort study

This paper looked at long-term health outcomes from an Israeli HMO. It mixed vaccinated and unvaccinated participants but held infection severity constant, which is unforgivable from an absolute risk estimation standpoint but probably fine for looking at the trajectory of recovery from long covid over time. “Mild” appears to mean “did not end up in the hospital”; however the case did need to be serious enough that it made it into medical records in the first place. 

The general trend is “things get better”, with the rate of improvement varying by symptom type. Unfortunately cognitive effects are the slowest to resolve, with at best a 50% recovery rate one year out.

Thanks to anonymous patron for supporting the original research and Patreon patrons for supporting this write-up.

A different observation of Vavilov Day

At this time last year, I was in the middle of a 36 hour fast in honor of Nikolai Vavilov and his team, who starved themselves to preserve a seed bank that went on to dominate Russian agriculture.* One reason I did that was to honor the team and their sacrifices, but another was to test and develop my own ability to do hard things when necessary. It was a great experiment, I did better than I thought I would but also the costs took longer to repay than I thought, and all of that knowledge was really valuable to me.

I went back and forth about doing the fast this year. The sense of continuity and retesting myself felt valuable and I’m sad about missing out on them. But I’m currently doing hard things and my capacity to deal with that kind of pain is a limiting reagent right now. Fasting for Vavilov Day would come at the expense of an actual project that matters to me. Delaying real work for a symbolic sacrifice would not only be stupid, it would be bad symbolism. I can’t honor a sacrifice for a cause by sacrificing a cause for symbolism.

So this year I’m not fasting. I do think I’ll want another fast at some point to see how my medical miracle affected things, but it will wait.

*My one sentence here is already simplified from the story as fully known in the West, read the blog post for more. I also suspect a lot of details haven’t made it into English. Last year I looked into hiring a Russian researcher to investigate and even found someone in mid-Febuary, but they put it off one week for a root canal and then had some other excuse the next week so seems like they’re not going to come through.

Vegan Nutrition Testing Project: Interim Report

Introduction

Reducing consumption of animal products is a choice with both moral and practical consequences. Last summer I found myself in contact with many vegans who cared a lot about the moral consequences, but had put little effort into learning about or managing the practical consideration of removing animal products from their diet. I’ve suffered a lot due to bad nutrition, so this made me very concerned. With a grant from the Survival and Flourishing Fund, I launched small a pilot project to give nutritional tests to 5 vegans and near-vegans from the Lightcone Office, which they could use to choose supplements that would hopefully improve their health.

My long-term goal was for everyone to have accurate information on their personal nutritional costs of veganism and make informed choices about how to handle them, with the first line solution being supplements. My goal for the pilot was to work out practical issues in testing, narrow the confidence interval on potential impact, and improve the nutrition of the handful of people. This report is on phase 1: getting the testing done and supplements started. It is aimed at people who might want to run a similar program at scale; if you are interested in running this for yourself I recommend checking out Tuesday’s post on iron deficiency.

Tl;dr: I found rampant iron deficiencies, validating the overall concern. The procedure I used has a lot of room for improvement. 

The Experiment

I gave nutrition tests to 6 people in the Lightcone office. 

The ideal subject was completely vegan, had never put any effort or thought into their diet, and was extremely motivated to take a test and implement changes. This person does not volunteer for studies, so I ended up with 4 vegans or near-vegans who had put somewhere between 0 and a lot of thought into their diet, 1 vegetarian, and 1 extremely motivated omnivore I used to test out the process.  In addition, one hardcore vegan contributed results from private testing. I did not poll the ~vegans on their exact diets.

Unless otherwise stated the results exclude the omnivore.

I gave each of these six people a Genova Metabolomix+ test, ordered from walkinlabs.com, with the iron add-on. This test was selected for being recommended by doctors I trust (in part because they prefer urine to blood testing), having extremely easy-to-read results, being nearly comprehensive (with the unfortunate absence of vitamin D), and because I hoped urine collection at home would be easier than blood draws at a lab. Foreshadowing: I was wrong about that last part.

I also gave people the option of an add-on to determine what variant of the MTHFR gene they have. MTHFR can affect how one processes certain B vitamins, and certain variants can necessitate a more expensive form of supplements.

Several people (although not everyone) scored with undetectably low iron. I offered them follow-up blood tests, which one person accepted. An additional vegan contributed blood test results without urine results.

As of publication all subjects have received their first round of results and started supplements of their choosing. 

The original plan was to retest in 3-6 months after people began supplements, using the same urine tests.

My initial predictions

I expected the big shortages to be B12, iron, and vitamin D, the first of which has very few* natural vegan sources and the latter two of which are scarce, although not absent, in vegan sources. This makes it pretty unfortunate the original test did not include vitamin D. 

[*B12 is naturally found in some (but not all) seaweeds and algaes, in at least one kind of mushroom, and in nutritional yeast. It’s also added to many wheat products in the US, so if you eat enough wheat and aren’t going out of your way to get unfortified wheat that’s a strong source]

Relative to the mainstream I wasn’t very concerned about protein consumption. Vegan proteins are a little less abundant, a little harder to digest, and have a less ideal distribution of amino acids, but are basically fine as long as you don’t pile on additional constraints.

One reason I was concerned was that lots of people I polled were piling on additional constraints, like keto or gluten-free, and still not doing anything to manage nutrition. I expected a smattering of deficiencies from these people, and to a lesser extent from everyone, as their restrictions and tastes cut off random nutrients. These could have been in any almost nutrient.

I expected everyone to be fine on vitamin C because it is abundant in both produce and processed food (where it’s used as a preservative).

Results

(including only vegans and near-vegans)

  1. ¾  vegan testers had severe iron deficiencies in their urine tests.
    1. The one who didn’t had both a stunning dietary intake of iron, and a parent who 23andMe believes to have a genetic predisposition to excessive absorption of iron.
    2. An additional vegetarian tester was not deficient.
    3. One of these retested with a blood test and scored low normal (~30). However this person was already taking iron supplements at the time of the test. 
    4. A bonus blood-only participant tested between 13 and 20, meaning they’d be considered deficient by some standards but not others.
  2. There were no B12 deficiencies, probably because everyone was already on B12 supplements. 
  3. One tester had a lot of deficiencies, including vitamin C, to the point I suspect it’s a problem with digestion rather than diet. 
  4. Everyone had at least one amino acid deficiency, including the person eating over 100g of protein/day. I don’t know how big a deal this actually is.
  5. The urine test did not include vitamin D.  Of the 2 blood tests, both had low-normal vitamin D.
  6. Excluding the person with across-the-board deficiencies, there were scattered other deficiencies but nothing else to consistently worry about. People were mostly in their tests’ green zone, with occasional yellow and red.

What does this mean?

Only one near-vegan out of 5 had solidly good ferritin levels. As I discuss here, that’s a very big deal, potentially costing them half a standard deviation on multiple cognitive metrics. 

There’s no control group, so I can’t prove that this is a veganism problem. But I’m quite suspicious.

There were no other consistent problems, so broad-spectrum testing is probably overkill for people with no known problems. 

Retrospective on the project

What worked

I consider the core loop of the study as vindicated as can it be at this stage. 

  • Deficiencies were identified, and the primary one was one of the three I predicted.
    • And another of the three, B12, was probably absent because people treated it preemptively. Note that people were inconsistent in what they took so I can’t say definitively what they were on during testing.
  • In the counterfactual timeline the shortages were probably identified much later if at all. No one who participated had any plans for testing, including people with obvious symptoms and people whose doctors had previously recommended testing.

This will be less impressive if supplementation doesn’t turn out to fix anything, but it’s an extremely solid start.

Other things that went well:

  • Having the room in my budget for unplanned additional testing, so I could add in serum iron tests when it became obvious they were necessary.
  • Creating a shopping list with links. I was worried this was somehow taking advantage of people (since I used affiliate links), but removing a decision and several steps from the ordering process seems to have been pretty crucial. 
  • Bypassing the need for doctors’ visits to get a test. Given how long it took people to order tests I think doctors’ appointments would have killed the project entirely. 
  • The Lightcone ops team was extremely cooperative and got all of the vitamins I suggested into the office.

Difficulties + possible changes

Potential changes are framed as recommendations because I am deeply hoping to hand off this project to animal advocates, who caused the veganism in the first place. 

  1. The test ordering workaround was not as good as I had hoped
    1. I’d originally hoped to just hand participants a box, but they had to order the tests themselves.
    2. In order to get iron + genetics tests people had to call rather than order online. This is non-standard for the provider and two people had to call twice to insist on what they wanted.
    3. Tests took a long time to ship, and a long time to return results after shipping. The lab alleges this is a supply chain issue and there’s nothing to be done about it. 
    4. Those two together turned into a pretty big deal because they made it very hard to plan and people lost momentum.
    5. In combination with the results showing few problems beyond iron I recommend deemphasizing full spectrum urine tests and focusing on blood tests for iron (and vitamin D), and making those convenient, perhaps by bringing a phlebotomist to the office.
    6. Another option would be to bring in a medical practitioner, who can order tests for other people, to manage tests so the office can be stocked with them. This of course fails to solve the problem for anyone not in the office.
    7. There are home tests for vitamin D and iron specifically, but I have no idea if they’re any good.
  2. Ideal test subjects (completely vegan, never done nutritional testing or interventions, promptly puts in the effort to do these tests and act on them once I suggest it) were even thinner on the ground than anticipated.
    1. I knew there wouldn’t be many, but I didn’t think it would be so hard to get five people pretty close to that profile. 
    2. I loosened restrictions and still consistently found problems, so recommend lowering the eligibility bar for testing in future rounds, especially since that was always the plan. The strict requirements in this round were an attempt to make the signal as loud as possible.
  3. Getting everyone tested was like herding cats. Beyond the problems with the test distributor, some participants needed repeated reminders to order, one lost a test, results went missing… it was kind of a nightmare.
    1. One advantage of focusing on blood tests would be to cut down on this, especially if you bring the phlebotomist to the office.
  4. At points I was uncomfortable with the deference some participants showed me. I was as clear as I possibly could be that this was a best-effort from a knowledgeable amateur kind of thing; they were responsible for their own health and I was a nonexpert trying to provide some logistics help. I nonetheless got more than one person bringing me problems not even related to the nutrition project, and insisting I tell them what to do.
    1. Recommendation: bring in a skilled nutritionist. They can both give better advice than me and devote more time to helping people. 
  5. I initially misread the protein results (which are delivered in terms of “how deficient are you?” rather than “what’s your current level?”, making 0 the best possible score). Luckily I knew I was confused from the beginning and no one had taken any actions based on my misinterpretation. More broadly, I’m just a woman who’s had some problems and read some stuff, I expect my suggestions to be better than nothing but far from the maximum good it would be possible to do.
    1. Recommendation: bring in a skilled nutritionist
  6. I underestimated the amount of time and especially emotional labor this project would need. I was hoping to bluff my way through that until people got on supplements, at which point the improvements in health would be their own motivation. I think I always overestimated how well that would work, but it was especially wrong because all the problems with the tests drained people’s momentum.
    1. Recommendation: I still think you should bring in a skilled nutritionist
  7. Many of the participants were moving frequently and not in the office by the time their results came in (because they took so long…), so they had to buy supplements themselves. Given the option I would have selected people consistently in the office, but as mentioned I was already managing trade-offs around participants.
    1. Recommendation: ask for more money to give everyone their first month of supplements and a convenient pill planner.

Next Steps

I previously planned to give people the same urine test 3-6 months after they started supplements. That no longer seems worth it, relative to the cheaper and more convenient blood tests. 

It’s not actually clear a formal follow-up is that useful at all. I initially planned that because I expected a wide range of shortages such that literature reviews wouldn’t be helpful. But there was only one real problem, and it has a richer literature than almost any micronutrient. So I don’t think another 5 people’s worth of scattered data is going to add much information. 

So the next step for this as a project would be mass blood testing for B12, iron, and vitamin D. 

Feeling motivated?

If this has inspired you to test your own nutrition, I haven’t done anything you can’t do yourself. Both the urine and blood tests are available at walkinlabs.com, and if you have a doctor they’re quite likely to agree to testing, especially if you’re restricting meat products or fatigued. I have a draft guide of wisdom on supplementation I’ve picked up over the years here, although again, I’m not a doctor and only learned how to digest food last May, so use at your own risk. 

Thank you to the Survival and Flourishing Fund for funding this project, Lightcone for hosting, and all the participants for their precious bodily fluids.

Iron deficiencies are very bad and you should treat them

In brief

Recently I became interested in what kind of costs were inflicted by iron deficiency,  so I looked up studies until I got tired. This was not an exhaustive search, but the results are so striking that even with wide error bars I found them compelling. So compelling I wrote up a post with an algorithm for treating iron deficiency while minimizing the chance of poisoning yourself. I’ve put the algorithm and a summary of potential gains first to get your attention, but if you’re considering acting on this I strongly encourage you to continue reading to the rest of the post where I provide the evidence for my beliefs.

Tl;dr: If you are vegan or menstruate regularly, there’s a 10-50% chance you are iron deficient. Excess iron is dangerous so you shouldn’t supplement blindly, but deficiency is easy and cheap to diagnose with a common blood test. If you are deficient, iron supplementation is also easy and cheap and could give you a half standard deviation boost on multiple cognitive metrics (plus any exercise will be more effective). Due to the many uses of iron in the body, I expect moderate improvements in many areas, although how much and where will vary by person. 

Note that I’m not a doctor and even if I was there isn’t good data on this, so it’s all pretty fuzzy. The following is an algorithm for treating iron deficiency that I’ve kludged together from various doctors. I strongly believe it is a lot better than nothing on average, but individuals vary a lot and you might be unlucky. 

  1. Take a serum ferritin test. If you have a doctor they will almost certainly say yes to a request, or you can order for yourself at walkinlab.com
  2. If your results show a deficiency (<20ug/L), increase iron intake through diet or supplements such as Ferrochel, taking the default dose once per day, with a meal.
    1. The definition of deficiency can vary by study, lab and goal. I picked <20ug/L because it’s the highest level I have concrete evidence is insufficient, but personally believe people are likely to benefit from iron beyond that and am taking pills accordingly.
  3. If you experience negative effects after taking the pills, stop immediately. Give yourself a week to recover, then you can try other brands, be more careful to eat with a full meal, etc. 
  4. If you are experiencing the symptoms of iron poisoning (listed below), stop pills and see a doctor now. Iron poisoning is a very big deal, which is why step 1 of this algorithm is “get tested” not “gobble pills”.  Unfortunately several of these are pretty generic, but I’m never going to feel bad about telling people with seizures to seak medical attention:
    1. Nausea
    2. Vomiting
    3. Abdominal pain
    4. Dizziness
    5. Low blood pressure and a fast or weak pulse
    6. Headache
    7. Fever
    8. Shortness of breath and fluid in the lungs
    9. Grayish or bluish color in the skin
    10. Jaundice (yellowing of the skin due to liver damage)
    11. Seizures
    12. Black or bloody stools
  5. Retest at 8-12 weeks, ideally at the same lab as before.
  6. Continue to retest every 8-12 weeks.
  7. If you increase by 20ug from your starting value without noticing any improvements to your cognition or overall energy levels; low ferritin is probably not your bottleneck.
    1. If you believe it’s not a problem at all, quit.
    2. If you believe it is a problem but another problem is limiting your gains, stay on a maintenance dosage but don’t put more time into managing this. Verrrrry roughly, divide your current dosage by your currently monthly gains (so If you take one RDA/day and gain 10ug/month, your result is 0.1), and take that much. This hopefully keeps you from losing ground, without gaining so quickly it could become a problem.
  8. If you’re getting improvements, keep going until those taper off. I personally would exercise caution and investigate the downsides of iron once I reached 80ug/L, but I’ve never gotten close to that so it hasn’t come up. 
  9. Continue to retest and adjust until you’ve found a dose on which your values are stable and healthy.

[Note: I provided links to supplements because I found people follow through more when I do, and because it’s easy to buy worthless supplements. There are other good supplements out there and if you have a reason to prefer one, take that instead. Links are affiliate.]

Research summary

Iron’s most famous use in the body is in hemoglobin, which your blood uses to transport oxygen. Oxygen is extremely important [citation needed], so it makes sense that low hemoglobin (aka anemia) gets a lot of attention, and everyone agrees anemia is very bad. But what the studies I read found was that even among people who started with adequate hemoglobin, a low ferritin score still predicted they would benefit from supplementation. And it’s not because of a bad definition of “adequate”; people saw benefits even when their hemoglobin didn’t change. So what else does iron do? 

Iron is one of a small number of elements that can safely accept electrons in reduction-oxidation reactions. Free electrons are quite damaging, so iron’s ability to safely contain them is important. Some specific usages:

  • The enzyme catalase, which converts caustic H2O2 to harmless water.
    • Fun fact: Catalase is the least important enzyme whose name and purpose I can recall offhand. Other enzymes achieved that status by being very important (DNA polymerase), or having self explanatory names (carbohydrase), but catalase achieved this by sounding kind of similar to a song I was into the summer I took microbiology, and I made up alternate lyrics about the enzyme.
  • Multiple points in DNA synthesis and repair, including keeping DNA polymerases in their correct shapes.
  • Myoglobin: similar to hemoglobin it binds oxygen, but instead of blood it stays in muscles, holding oxygen in reserve until it is needed.
  • Regulation of many components of the immune system.
  • Proline and lysyl hydroxylases, both used to build collagen.
  • Please enjoy this list of 80 enzymes that use iron as a cofactor.

I’d say “that’s a lot” but honestly it’s not, everything in the body is like this, it was not built to be understood.

Standard tests for anemia only look at hemoglobin. Ferritin tests are considered to be a much better measurement of cellular iron levels. There’s suspicion, although not proof, that your body prioritizes hemoglobin production above other uses of iron, so it will undersupply these other uses in order to maintain hemoglobin levels. This suggests that if you have normal hemoglobin but low ferritin, additional iron will find many uses. Unfortunately, those uses and their effects are so varied I can’t really predict what any particular person will experience.

There are any number of studies showing correlations between low ferritin and low functioning, but I don’t find those very useful. The people in those studies might have any number of deficiencies for multiple reasons, or low ferritin levels could just be a proxy for poverty. In my research I stuck to actual experiments, with controls, that gave iron to subjects and checked for an improvement in function, not just test scores. Unfortunately, there were not that many of them.

The only study I liked on the cognitive effects found an absolutely enormous effect. Successful iron supplementation led to improvements averaging >0.5 standard deviations in attention, learning, and memory. I have qualms about this study and expect the results are cherrypicked, but it’s also not necessarily the full size of the effect, because they stopped after a set amount of time rather than waiting for effects to plateau.

There were multiple studies on iron supplementation and exercise. In a nutshell: everyone’s endurance improves when they exercise. Giving people with iron deficiency but not anemia (IDNA) iron supplements increases that effect. In the strongest study, people treated for iron deficiency for 6 weeks improved their 15km time by 10%, compared to 5% in the control group. Another study (which didn’t involve exercise training) showed no improvement in time to complete a given distance, but did find the treatment group used about 5% less energy while doing so. 

Iron deficiency rates vary a lot by population, but with the patterns you’d expect. Vegans are more deficient than vegetarians, who are more deficient than omnivores. People who regularly menstruate (or give blood) are more likely to be deficient. I found the baseline rate of omnivorous men in rich countries to be somewhere between 0-11%. For a female omnivore it’s 9-22% (these numbers include people already taking supplements; it’s presumably higher if you don’t). Young female vegans who were not already supplementing were at least 50% deficient, plausibly more. Data for non-supplementing male vegans was not available, but let’s ballpark it at 5-25%, based on the ratio between men and women in the general public.

People in poor countries are much more likely to be iron deficient and anemic, due to poor diet and more physical exertion. 

Caveats

I am not a doctor, my most relevant credential is a BA in a different part of biology, the fact that I couldn’t find a decent resource and had to make it myself is a sign of civilizational inadequacy.

Normally not being a doctor inhibits me from giving medical advice, but I am going to go ahead and say that iron poisoning is extremely bad and not that hard to induce with pills, don’t do that. Iron poisoning is why you need to be careful your kid only gets one multivitamin a day, and why men can’t use women’s multivitamins (which should actually be “menstruator’s multivitamins”, since the relevant issue is monthly blood loss). 

The papers are very finicky and boring and this was really important, so I’ve tried to frontload my conclusion. This is a delicate balancing act of readability and accuracy. I did my best but some trade-offs are unavoidable. 

This lit review was done with a focus on people with low iron intake, especially vegans. None of the studies I looked at filtered on dietary intake versus absorption issues. This means they probably underestimate the impact of supplementing for healthy people.

Do not take the dosages in the studies literally, especially if you don’t menstruate. The right dosage depends on the form and your personal needs. I suggest operating based on RDA percentages rather than raw chemical weights.

Details

Definitions

There are a lot of ways to measure iron and iron-related levels in the body. The two most important are hemoglobin (the protein red blood cells use to carry oxygen) and ferritin (the protein your cells use to store oxygen, but also present in blood). There are some other numbers I’m going to ignore.

Hemoglobin and ferritin are both testable via blood sample, and the tests have something called “reference ranges”, which are supposed to be the healthy range of values. Whether the ranges actually capture that is a matter of great controversy, with various people alleging the minimum is what you need to avoid hardcore deficiency diseases, but won’t get you optimum functioning, to people claiming low scores are fine and anyone who says otherwise is a psyop from Big Vitamin. And then there’s individual variation.

Hemoglobin’s reference range is 120g/L-170g/L.  Ferritin’s reference range starts between 10 and 20 ug/L, and ends at 150-200ug/L, depending on who you ask.  It’s possible to have low hemoglobin (aka anemia) without an iron deficiency or vice versa. Low hemoglobin with adequate iron typically means you’re having trouble manufacturing hemoglobin and is beyond the scope of this post. Low iron with adequate hemoglobin is more controversial. Top explanations include “the tests aren’t that good”, “you’re deficient but your body is prioritizing hemoglobin production”, “you’re about to develop anemia” and “low iron is fine, actually”.  

Impact Data

When looking at studies I used the following selection criteria:

  • Examining iron deficiency without anemia. We can assume that anemic cases will benefit more from iron, unless the anemia is unrelated to the iron deficiency.
  • No co-morbidities.
  • On adult humans (in practice this almost always means women).
  • In the developed world.
  • RCTs only, no correlational studies.

This didn’t leave a lot of studies, and I had to accept some other flaws. 

Mental

Murray-Kolb and Beard (2007)

This study was by far the best study of cognitive function, maybe the only one that tested an intervention rather than merely looking at correlations. I don’t love it. The data presentation is obviously leaving a lot of information out, I assume to dramatize results. But those results are very dramatic. 

This study allowed for mild anemia (hb < 120 but >105), but separated anemic and non-anemic subjects. The paper, uh, doesn’t mention its threshold for iron deficiency; another paper from the same authors set it at serum ferritin <=12ug/L, which is in line with the aggregated averages.

The study included a double control group that started with sufficient iron and hemoglobin. Each group (no deficiency (n=42), iron deficiency without anemia (n=73), and iron deficiency with anemia (n=34)) was split into treatment and placebo groups. 

Iron supplementation increased ferritin levels in everyone. People with iron deficiency without anemia (IDNA) increased serum ferritin (sFt in the table) 2.5x more than their placebo group; people with iron deficiency and anemia (IDA) improved ferritin levels almost 4x more than their placebos. Neither group got anywhere close to the ferritin levels of the no-deficiencies group. The treatment group was given 160 mg of ferrous iron daily.

In baseline cognitive testing, IDNA women scored about the same or slightly worse as healthy women, and IDA women scored much worse than both. This is probably an underestimate of the effect, because the study was heavily recruited from students at a single university, who can be expected to be selected for the same range of competence.

The study separately evaluated treatment-group women who had increased ferritin levels from those who didn’t. The former group had large improvements in their cognitive test results, the latter very modest ones. I think separating out non-responders is fair: if there’s a problem interfering with iron absorption that doesn’t tell you anything about the effect of increasing ferritin levels, and I am studying this mostly for the benefit of people with insufficient dietary intake.

Among ferritin responders, attention, memory, and learning increased from .5 to .75 standard deviations (although somehow that .75 is at p<0.07). That effect size is the equivalent of 7.5-12.5 IQ points or 1.1-1.6 inches in height. Of course the test could be bullshit, but it’s not out of line with anecdotes I hear. Additionally, the treatment groups did not reach the ferritin levels of the healthy group, indicating potentially more gains to be had.

Hemoglobin responders also saw more improvement than non-responders, but the effect size was smaller than with ferritin, indicating an effect of iron beyond increasing hemoglobin.

(Note that the axis has changed from performance to time required, making negatives good. Yes, I am suspicious that they presented total score for one metric and time to completion for another).

Physical

Zhu and Haas (1998)

20 women with normal hemoglobin (Hb >120 g/l) but low ferritin (serum ferritin <= 16 ug/l) were given 135 mg ferrous iron supplements for eight weeks and instructed to take with citrus juice. 17 women were given placebos as a control (random assignment, double-blinded). They were given both blood and athletic tests before and after treatment.

Treatment group hemoglobin and iron binding capacity were unchanged. Serum ferritin was up 250% for the treatment group (compared to 30% for the control). Their athletic test results did not improve any faster than the controls, however they needed less energy (2.0kj/min) and oxygen (5%) to get those same results. 

Hinton et al (2000)

22 women with normal hemoglobin (Hb >12 g/dl) but low ferritin (serum ferritin <= 16 ug/l) were given 100mg ferrous iron supplements for six weeks. 20 women were given placebos as a control (random assignment, double-blinded). They were given both blood and athletic tests before and after treatment.

Iron supplementation did not change hemoglobin or iron binding capacity levels, but did increase serum ferritin by about 50%, and transferrin saturation by 70%. Note that their ending ferritin levels (19.4) were still barely above the bottom of the reference range, indicating there was probably much more room for growth. 

The control group went marginally up on some measurements and marginally down on others, I’ve treated their changes as noise.

Both treatment and placebo groups were given 4 weeks of exercise treatment; the treatment group showed about double the athletic improvement. Endurance saw a bigger improvement than initial performance.

Brownlie et all (2004)

~20 women, with serum ferritin concentration < 16 ug/L and a hemoglobin concentration > 120 g/L were given 100mg ferrous iron/day for 6 weeks. Of that, 4 weeks also included exercise training.

Once again we see an improvement in ferritin but not hemoglobin or binding capacity.

The treatment group experienced ~30% more improvement in their trial times than the control, or 800% if they started with elevated ferritin. I’m suspicious of this posthoc subgroup analysis, but on the other hand, the bar in this graph is very big.

Hinton and Sinclair (2006)

The good news: this study has men! I did not think I was going to find any of those!

The bad news: this study has 20 people, total.

No meaningful change in hemoglobin or binding capacity, near doubling of serum ferritin in the treatment group, 15% drop in ferritin in the control group. 

Results are basically identical for the control and placebo groups.

Prevalence Data

Estimates for the prevalence of iron deficiency vary a lot by study and population. 

In the first paper I found, the estimate was 9-22% among menstruating women in the general public, and 1-2% among adult men (non-menstruating women were not included but I expect “do you lose 2-4 tablespoons of blood every month?” and “do you occasionally host demanding parasites?” to be more important than hormones or gender identification). Note that this number includes both anemic and non-anemic iron deficiency. 

Vegans are at much more risk. One German study of vegan women found a median serum ferritin level of 14 ug/L, a level that is above their reference range and LabCorp’s but below the cut-off in several of the studies cited above. They found 40% of young women fell below their threshold for deficiency (12ng/ml) and 11% of older women (presumably mostly post-menopausal) did so. Women taking iron supplements were excluded from this study.

A second German study (why are they all from Germany?) that allowed supplements and had an even gender split found rates of iron deficiency slightly lower in vegans than omnivores, but both had higher means than anyone in any of the impact studies I found. Nonetheless, 10% of vegans were iron deficient.

My own study (data forthcoming) had 3-4 male vegans and a deficiency rate of 25%-75%, depending on how you count. 

Thank you Survival and Flourishing Fund for funding this research and Lightcone Infrastructure for providing a home for it.

Thanks to Andrew Rettek for help making my normally very in-the-weeds style more accessible.

Quick look: cognitive damage from well-administered anesthesia 

Recently a client commissioned me to look at the potential cognitive impacts of general anesthesia. I was surprised to find out that it’s not obvious general anesthesia does more damage than spinal or local anesthesia, and my guess is most but not all of the damage is done by the illness or surgery themselves. 

Caveats and difficulties

I’m not a doctor. The following represents something like 5 hours of work, which obviously is not enough time to process even a fraction of the literature. I was focused on the dangers of median uses of anesthesia, where nothing goes obviously wrong and the anesthesiologist considers it a success; I didn’t even attempt to look at the rate of accidents, which can be pretty severe. My friend’s dad’s life was ruined by a fungal contaminant in a spinal injection. And of course, people die from excess general anesthesia. But for this post I only looked at damage done by routine anesthetic usage.

Like all client research, this was tailored to a particular person’s needs and budget, and shouldn’t be considered a general-purpose survey.

It’s pretty hard to tease out the difference between damage done by anesthesia, damage done by whatever necessitated the surgery, and damage done by having your body ripped open and bits moved around. Bodies hate that sort of thing. The few RCTs that exist by necessity focus on a narrow range of minimally invasive surgeries for which there exists a choice in type of anesthesia, and animal studies tended to focus on developing animals rather than adults. Even for procedures where multiple types are possible, patients tend to be pretty opinionated about what they want; one paper even announced they’d given up on reaching their sample size goal because recruiting was too hard.

Studies also often focused on cognition within a few hours of surgery (when people are still at the hospital to test). I think that’s less likely to be “damage” and more likely to be “it’s still wearing off” or “I’m sorry, I just had minor surgery and you want me to take an IQ test?”. This made me throw out a lot of studies.

Few if any of the papers attempted to control for post-operative condition or pain med usage, which seems like an enormous oversight to me.

Tl;dr

My overall take home is that:

  • Little or nothing that necessitates surgery is good for cognition and that needs to be factored into assessments.
  • Surgery itself is enormously stressful, physically and emotionally, and that stress impairs cognition, sometimes in lasting ways. This includes procedures that are not cutting new holes in you, like kidney stone treatments, although presumably it’s worse for open heart surgery.
  • Probably there are additional effects from anesthesia. At least general and spinal, maybe including local. On priors I still believe general and spinal are worse on a purely physical level.
  • Probably a lot of whatever damage there is heals in most people, although people who need surgery are already under heavy load and will be the worst at healing. 
  • There may be treatments that can prevent damage but they’re still in rodent trials right now.
  • I also believe that being awake and aware during surgery can be emotionally traumatic, and trauma is also bad for cognition, so include that in your math. 
    • But I’m not trustworthy on this, seeing as I was terrorized by a series of dentists and now can’t get myself through simple teeth cleaning without some sort of bribe, a human to guard me from the bad dentist monster, and a sedative. 

Papers

I didn’t rigorously track correlational studies, but my sense was they tended to show faster recovery from local and spinal anesthetic, relative to general, presumably because milder cases get milder anesthesia even when the procedures use the same billing code. Additionally a lot of studies were given too soon after surgery, which I don’t expect to predict long term damage

In the few studies that randomly assigned patients to spinal, local, or general anesthesia, and surveyed at least 7 days out, it’s really hard to pick a winner. 

Incidence of postoperative cognitive dysfunction after general or spinal anaesthesia for extracorporeal shock wave lithotripsy tries really hard to claim that spinal and general anesthetic are equally damaging to cognition, despite finding a 3x higher rate of cognitive issues after general anesthestic. I showed this paper to my statistician father and he gave a rant I wish I had recorded because it would make me famous in the right corner of Twitter. Hell hath no fury like a statistician forced to read a medical paper. He agreed with me that 19.6% (the rate of complications in the spinal group) was much larger than 6.8% (rate of complications in the general group), but dismissed that as merely a felony next to the war crimes against statistics they committed by using the wrong test for statistical significance.

Two meta-analyses both find a small difference in favor of spinal over general, with confidence intervals that overlap no-difference. One found spinal to be ~5% better (26 studies), the other 50% (but only 5 studies, so still overlapping with 0). The latter analysis is tiny in part because it is restricted to tests within a week of surgery. The analysis that looked also failed to find improvements from using local anesthetic.

On the other hand, animal studies of anesthesia without surgery regularly show impairment, although they can’t agree if post-anesthesia animals start off worse but catch up, or start off the same but fall behind. Also they found other medications could mediate the effects. I summarize the animal results in this spreadsheet. These are effect sizes we would clearly notice in humans so I assume they’re using much more anesthetic (although they claim it’s proportionate) or the animals, primarily rodents, are much more sensitive. Also the studies tended to be within days of anesthesia application, removing a chance to heal.

Tangent: UpToDate.com

The original commission was to investigate kidney stone treatments, and what I can say there is that the general medical site UpToDate is pretty good. Every claim I investigated checked out and I didn’t find anything at all established that they didn’t.

Thank you to Claire Zabel for commissioning the research and encouraging me to share the findings, and to my Patreon patrons for supporting the public write-up.

Follow up to medical miracle

The response to my medical miracle post has been really gratifying. Before I published I was quite afraid to talk about my emotional response to the problem, and worried that people would strong arm in the comments. The former didn’t happen and the latter did but was overwhelmed by the number of people writing to share their stories, or how the post helped them, or just to tell me I was a good writer. Some of my friends hadn’t heard about the magic pills or realized what a big deal it was, so I got some very nice messages about how happy they were for me.

However, it also became clear I missed a few things in the original post.

Conditions to make luck-based medicine work

In trying to convey the concept of luck-based medicine at all, I lost sight of traits I have that made my slot machine pulls relatively safe. Here is a non-exhaustive list of traits I’ve since recognized are prerequisites for luck based medicine:

  • I can reliably identify which things carry noticeable risks and need to be assessed more carefully. I feel like I’m YOLOing supplements, but that’s because it’s a free action to me to avoid combining respiratory depressants, and I know to monitor CYP3A4 enzyme effects. A comment on LessWrong that casually suggested throwing activated charcoal into the toolkit reminded me that not everyone does this as a free action, and the failure modes of not doing so are very bad (activated charcoal is typically given to treat poison consumption. Evidence about its efficacy is surprisingly equivocal, but to the extent it works, it’s not capable of distinguishing poison, nutrients, and medications).
    • This suggests to me that an easy lever might be a guide to obvious failure modes of supplements and medications, to lower the barrier to supplement roulette. I am not likely to have the time to do a thorough job of this myself, but if you would like to collaborate please e-mail me (elizabeth@acesounderglass.com).
  • A functioning liver. A lot of substances that would otherwise be terribly dangerous are rendered harmless by the human liver. It is a marvel. But if your liver is impaired by alcohol abuse or medical issues, this stops being true. And even a healthy liver will get overwhelmed if you pile the load high enough, so you need to incorporate liver capacity into your plans.
  • A sufficiently friendly epistemic environment. If it becomes common and known that everyone will take anything once, the bar for what gets released will become very low. I’m not convinced this can get much worse than it already has, but it is nonetheless the major reason I don’t buy the random health crap facebook advertises to me. The expected value of whatever it is probably is high enough to justify the purchase price, but I don’t want to further corrupt the system. 
  • Ability to weather small bumps. I’m self-employed and have already arranged my work to trade money for flexibility so this is not a big concern for me, but a few days off your game can be a big deal if your life is inflexible enough. Somehow I feel obliged to say this even though I’ve lost work due to side effects exactly once from a supplement (not even one I picked out; a doctor prescribed it) and at least three times from prescription medications.
  • A system for recognizing when things are helping and hurting, and phasing treatments out if they don’t justify the mental load. It’s good to get in the habit of asking what benefits you should see when, and pinning your doctor down on when they will give a medication up as useless.
    • Although again, I’ve had a bigger problem with insidious side effects from doctor-initiated prescription meds than I ever have with self-chosen supplements.
    • Probably there are other things I do without realizing how critical they are, and you should keep that in mind when deciding how to relate to my advice. 

Feel free to add your own conditions in the comments and I’ll add my favorites to this list.

Ketone Esters

Multiple people have asked for details on the ketone esters thing, and I sure hope that’s because I convinced them to try stuff rather than somehow sold ketone esters in particular as good. Answers to the common questions:

  • I use KE4, but I haven’t tried any others. I think when I originally looked it was the only one available without caffeine, but I could be wrong, or that could have changed.
  • When I first started and was doing longer intermittent fasting I’d do 10-15ml at night, 5-10 in the morning, and 5-15 before workouts (all on an empty stomach). I currently only do 5ml, before bed, to smooth out blood sugar issues whle sleeping.
    • The change is partially because I’m recovering from an injury and that does not mix with intermittent fasting, and partially because KE seems to have caused durable changes so there’s less point. I went from 3-4 sodas a day to none a few days after starting KE4 and it’s never reverted. The only caffeine I’ve had is incidentally in chocolate, and after the Bospro I’ve barely even had that.

Minimal Potato Diet

Again I am not recommending this, but if you would like to know what I’m doing:

  1. I use small potatoes- ideally the really tiny ones, but half-a-fist size at most. And I aim for a variety of color potatoes. These are out of a not particularly verified belief that skin has more vitamins than the core and that color means vitamins, or at least antioxidants. I also prefer the way the small ones cook.
  2. I cook the potatoes as soon as I receive them. If that’s not possible they might spend a few days in the fridge. When I let them age enough to get eyes they upset my stomach.
    1. A lot of people on the potato diet had to skin their potatoes to prevent feeling ill. I am curious if that would have been required if they’d used very new potatoes.
  3. I cook the potatoes by throwing them onto a cookie sheet and roasting at 350F for 45 minutes. I do this because it’s really quick and I prefer the dry texture.
    1. I cook 3 pounds a time because that is both the size of the bag they come in and about what my cookie sheet can hold.
    2. I tried gnocchi, but the additional flavor made me get tired of it faster. Also maybe my weight loss slowed around then but the potato weight loss has been weirdly punctuated so I dunno.
    3. I wish I could share a graph of just how weird the weight loss has been – same weight for 1-2 weeks, then 3 pounds in 4 days. Unfortunately, I keep changing my creatine dosage which ruins the aesthetics with a lot of water weight changes.
  4. The cooked potatoes spend at least a day in the fridge before eating, and ideally several. This is out of a slightly verified belief that the post-cooking cold converts some of the starches from digestible to indigestible, which lowers calories while doing something vaguely good for my digestive tract. But since I’m cooking much less often than eating they inevitably log a lot of fridge time anyway.
  5. Originally I ate about 100g/day, mostly in the morning but if I woke up craving something I’d start with that. For a few days now I’ve been experimenting with eating smaller amounts of potato more times per day and that’s maybe driven calorie consumption further down, but far too early to say for certain, and it’s not totally clear that would be desirable.
    1. This is based on my hypothesis that potatoes reduce calorie consumption in me by being a relatively bland food with (small amounts of) lots of different micronutrients, plus some help from the fiber. 
    2. Slime Mold Time Mold thinks it’s potassium and is testing that now. 
  6. I originally described myself as making no other changes. That was 100% true in the beginning, I will admit I now check in with my food diary calorie total and adjust a bit (including upwards, although not sure about the relative frequency). The point of the food diary is micronutrient tracking but it’s hard to avoid reacting to the calorie number once it’s there. I’m not sure that’s actually affecting things much – on days I happen to have a high count I eat much less the next few days without thinking about it. 
  7. My food diary is very clear I am not reliably hitting the RDA for most vitamins. I think you can do it on my calorie count but it would be a lot of effort and planning and I’m on vitamins anyway. Hopefully I get nutrition test results in the next month, although that will be much more a referendum on the Bospro than the potatoes. 
average nutritional intake for the last two weeks

A male friend lost 4 pounds on a 50% potato diet and then plateaued (but that could be from an injury). A female friend tried my minimal potato diet and experienced no change.  I think if that worked reliably we would already know about it.

Bonus

Shout to reader George who connected me with an offline friend who had similar symptoms with the same cure, who has done a ton of research into mechanisms and suggested some follow-ups. They’re not guaranteed to work but this feels like a rich vein to me. Thanks George and offline friend!

Luck-based medicine: My resentful story of becoming a medical miracle

You know those health books with “miracle cure” in the subtitle? The ones that always start with a preface about a particular patient who was completely hopeless until they tried the supplement/meditation technique/healing crystal that the book is based on? These people always start broken and miserable, unable to work or enjoy life, perhaps even suicidal from the sheer hopelessness of getting their body to stop betraying them. They’ve spent decades trying everything and nothing has worked until their friend makes them see the book’s author, who prescribes the same thing they always prescribe, and the patient immediately stands up and starts dancing because their problem is entirely fixed (more conservative books will say it took two sessions). You know how those are completely unbelievable, because anything that worked that well would go mainstream, so basically the book is starting you off with a shit test to make sure you don’t challenge its bullshit later?

Well 5 months ago I became one of those miraculous stories, except worse, because my doctor didn’t even do it on purpose. This finalized some already fermenting changes in how I view medical interventions and research. Namely: sometimes knowledge doesn’t work and then you have to optimize for luck.

I assure you I’m at least as unhappy about this as you are. 

Preface to the Preface

I’ve had nonspecific digestive issues since before I have memories. In pre-school my family joked that I would die as a caveman because there were so few things I would eat, and they were mostly grains. This caused a bunch of subclinical malnutrition issues that took a lot of time to manage and never got completely better. And while I couldn’t articulate this until it went away, food felt gross all the time

It’s hard to convey just how bad this was for me, because it feels like it undermines everything I did to work around it. I’ve always been functional but decidedly less healthy than my friends. I got sick more often and it hit me harder. I was slower to heal from injuries and scrapes and that limited my interest in the more athletic sort of hobbies.  I couldn’t work the same hours, and working hours traded off really sharply against energetic hobbies. I had to spend a lot of time managing food where other people can just show up and eat, which was a constant source of social stress. My genetics say I was destined to have anxiety issues, but the low level malnutrition and justified feelings of food insecurity despite apparent abundance did not help anything.

Eventually in my late 20s. I saw a nutrition-focused psychiatrist who listened to my observations (I could only eat protein with soda), immediately formed a hypothesis (I produced insufficient stomach acid), asked questions to rule it out (which I no longer remember), suggested a test (take stomach acid pills and see if they gave me heartburn), and when it came back positive (no heartburn) suggested a course of action (keep taking stomach acid pills) that showed immediate benefits in practice (indigestion removed, but only when I took the pills). My protein and produce intake increased enormously, and I felt overall much better. 

This is exactly how I want medicine to work. I gathered good data and took it to an expert who immediately formed a model, definitively tested it, and prescribed a course of action that made mechanistic sense.  If you forget that it took almost 30 years and I took those exact same symptoms to other doctors beforehand, it’s a stunning success. 

But it was not a total success. My protein intake maxed out at 50 grams/day, and that was if I made consuming protein a hobby and nothing went wrong. I was doing much better than I had been, but my nutrient tests showed I still had a lot of issues. Eventually the stomach acid pills stopped working, although that seems to be “my stomach started producing more acid and a different problem became the bottleneck”  rather than the pills ceasing to contain acid. But the problem was not solved, and more of the existing treatment did not help.

Standard Preface

I worked with a number of doctors on fixing the remaining digestive, for ~another decade. I had a lot of conversations like the following:

Me (over 20 pages of medical history and 30 minutes of conversation): I can’t digest protein or fiber, when I try it feels like something died inside me. 

Them: Oh that’s no good, you need to eat so much protein and vitamins

Me: Yes! Exactly!. That’s why I made an appointment with you, an expensive doctor I had to drive very far to get to. I’m so excited you see the problem and for the solution you’re definitely about to propose.

Them: What if you took a slab of protein and chewed it and swallowed it. But like a lot of that.

Me: Then I’d feel like something died inside me, and would still fail to absorb the nutrients which is the actual thing we want me to get from food.

Them: I can’t help you if you’re not willing to help yourself.

Or sometimes…

Me (over 20 pages of medical history and 30 minutes of conversation): I can’t digest protein or fiber, when I try it feels like something died inside me. If I make it my top priority I can get maybe 50 grams of protein a day.

Them: Oh that’s no good, you need 70 minimum, and really more like 100. Also because I’m a naturopath I’m morally obligated to tell you to give up eggs, dairy, and wheat.

Me: That’s gonna be hard seeing as those three are 90% of my protein intake and by far the easiest forms of protein to digest.
Them: What if you ate pea protein?

Me: Well that’s harder so…worse.

Them: What about hemp?

Me: That is even harder than pea protein.

Them: If you’re not going to try why are you even here?

These exchanges were incredibly draining for me, so I didn’t have them that often. Every year or two I’d get my hopes up for a new doctor, pay a shitton of money (these doctors are never covered by insurance) for several emotionally draining appointments, and then get told they couldn’t help me and this was a failure on my part.

After several years of that pattern I gave up and went back to my old PCP. She hadn’t solved the problem either, but she had solved other problems, had ideas to try for this one, and believed it was a physical rather than moral problem. Unfortunately she is very busy, and sometimes pawns me off on her assistant doctors, who are idiots. That second conversation was with one of those, although in the real conversation I was less witty, and was more like “*sob* no *sob* I told you *sob* I CAN’T”. 

I refused to see that doctor again, but this left me little leverage when they assigned me a different sub-doctor to handle a post-covid rash back in April. You know how naturopaths complain about western medicine being mechanical and reactive and not taking the time to reach a systemic understanding? Well this guy, who we will call Dr. Spray-n-pray, was determined to fight for equality by taking the same approach with unregulated supplements. He guessed I had an allergic reaction and threw 5 different antihistamines of varying legitimacy at me, with no mention of testing the hypothesis, monitoring my progress, expected changes, duration of treatment… 

And it worked.

Not on the rash; I eventually had to go to urgent care for that. But shortly after I started the pills, I found myself eating 50 grams of protein in a sitting and then going back for more the next meal. I also started chowing down on produce, and at some point realized I couldn’t remember the last time I’d had dessert. I had known I had some aversion issues with food but didn’t realize how gross I found it until the feeling went away and I could just eat without feeling contaminated. About here is when I started a food diary and found I was regularly hiting 100g of protein/day. When I crashed my scooter I ate 350 grams of protein over two days, suggesting I could do that any time I wanted but chose not to, suggesting my body was getting all the protein it felt it needed, all of the time.

I’m not sure I can convey what a big deal this is either. I would have paid several years’ salary for this cure without thinking. It is now possible for me to feel okay at an emotional level it wasn’t before. Plus, you know, I can actually get the nutrients I need to run my body and stuff.  My injuries after that scooter accident healed noticeably faster than past injuries. The fact that I haven’t caught an illness since April’s covid isn’t conclusive, since it’s summer and I haven’t done anything high risk, but it is interesting. 

[I do have covid antibody results from the December (8 months after my vaccine) and August (4 months after catching covid)  and my levels have gone way up, but that’s more likely due to the more recent and stronger immune stimulus.]

But that evidence came later. Back in May the timing of the miracle suggested that one of Dr. Spray-n-pray’s pills was responsible. This was more or less confirmed when I weaned off the various pills and the subtle grossness around food started to return. I could also feel growing sugar cravings. So it was important to figure out what the miracle pill was and get back on it immediately.

[If any of you are thinking “well it could have been a coincidence”: no it fucking couldn’t. I did not carry this around for 35 years and try everything to fix it only to have it suddenly go into remission for no reason. I’ll believe covid fixed it before I believe that.]

I had always assumed the reason doctors turned on me was that it was easier than accepting that they couldn’t solve my problem. But this one had fixed my problem! Not on purpose or anything, but I was fully prepared to pretend it was. Now we just had to figure out what had worked and why, in case it suggested any additional actions. I made a spreadsheet tracking the changes as best I could – when my diet changed (using grocery order data), when I’d started and stopped which pills. Surely my data plus his doctor ego would help us get to the bottom of this.

At the time of my follow-up appointment I had a strong guess which supplement had helped based on timing, but it didn’t make any sense. The active ingredient was Boswelia (specifically BosPro brand (affiliate link). I’m afraid to try another in case it breaks the spell). Boswelia is sometimes described by alt medicine websites as helping digestive issues, but in the same way they describe every supplement as helping digestive issues. “Helps anxiety, allergies, autoimmune disorders, inflammation, and digestion” should just be a stamp. This isn’t even necessarily illegitimate – the body is complicated and lots of things are entangled, especially with inflammation.

But I’ve tried a lot of these supplements at one point or another and there was absolutely no reason to predict this one would be different, even if I had researched it ahead of time. Examine.com is pretty positive on Boswelia but doesn’t list digestion as an issue it solves. Everything is connected to everything else in the body and it was still pretty hard for me to make a causal chain between Boswelia’s alleged mechanisms and improvements in my digestion. So I was extremely excited for Dr. Spray-n-pray to explain why it had worked.

All this was on my mind when I finally got to ask Dr. Spray-n-pray why his treatment had worked. He mumbled something about inflammation and moved on. He had zero interest in my spreadsheet or a more mechanistic understanding of what had changed. I confirmed the miracle was from BosPro when I resumed taking it and the digestive improvements returned (including the creeping feeling of grossness going away). It’s now 5 months since I started taking it and it still works but I have no idea why.

This is not how I want medicine to work, at all. A medic who clearly was not trying for a systemic understanding recommended a lot of stuff and one of them happened to fix a problem as unrelated as could be that I’d spent a decade+ searching for without success? Even knowing definitively that it works we have no idea why, and what would help or hinder it? And there’s ~0 evidence this would help other people with the same condition?

This is bullshit. But bullshit is working where logic feared to tread.

Other Evidence

This experience isn’t what got me on the path of luck-based medicine though. I was already at that point when the supplements were prescribed, which is why I took them instead of doing 5 hours of research and ignoring Dr. Spray-n-pray’s suggestions as the ravings of an idiot. There were a lot of contributors to my shift, but a few stand out.

A few years ago I ran a series of epistemic spot checks on various self-help books, and found that how helpful they were had no correlation with how rigorous or true their theoretical backing was.

Then last year I ran that ketone ester study. I and a handful of people I know get insane gains from using ketone esters – better than Ritalin with none of the side effects – but when I ran an RCT (n=8-12 depending on how you count) no one reported any benefits. 

Or take Slime Mold Time Mold’s all-potatoes-all-the-time diet study (which happened after I started on the magic pills, but is too good an example to pass up). I have an extremely long list of complaints about their hypothesis and follow up study:

  • They failed to contextualize it as a monodiet and discuss the classic monodiet problems.
  • Potatoes aren’t nutritionally complete and don’t have enough protein for people to thrive. They gestured at some of the nutritional deficiencies but I think not hard enough, and believe potatoes have more protein than reported but have not pointed to any evidence to that effect.
  • They tracked weight loss over 28 days but will not be doing a follow-up for six months. Since the default after rapid weight loss caused by an unsustainable diet is immediate regain, this is unconscionable.
  • I haven’t had time to dig into the object-level facts in the argument between SMTM and a persistent critic, but with my monkey social brain it sure does look like SMTM is blowing off well-founded criticism (given in a super aggressive manner).
  • They treat weight loss as an unalloyed good no matter how fast or what the person’s starting weight was.
    • I have not looked into the popular “weight loss not safe above 2 pounds per week” claim and it wouldn’t shock me if it were made up, but if I had an intervention with double that impact I’d spend an hour investigating the claim.
    • Weight loss beyond a certain body fat percentage is bad. You need that stuff.
  • They did warn people about solanine poisoning but I think they should be more concerned about it.
  • Analysis featured a lot of stories along the lines of “Did X on Wednesday and lost 2 pounds on Thursday” and fat loss does not work like that. Two pounds overnight is either water weight or has a lookback period longer than 24 hours.
    • I’m deeply confused about that second part, I don’t understand why or how weight-loss-that-is-definitely-not-changes-in-water-retention comes in chunks. If you have an answer I’m quite curious.

That’s a lot of epistemic sins. OTOH, their potato diet results inspired me to try the minimal potato diet, which consists of eating some potatoes every day (I started with ~100g of baby potatoes), and I’ve lost 15 pounds in 3 months. That level of weight loss with zero sacrifices buys you a lot of epistemic forgiveness, especially when my miraculous dramatic dietary improvements did fuck all to the number on the scale.

[ People already writing their “potatoes can’t possibly be the cause it must be psychosomatic” comments in their head: I see you. Your hypothesis is perfectly reasonable; in your position it would be my first reaction too. But in this particular case you’re going to need to explain why potatoes caused that magic mental shift when giving up soda, a dramatic improvement in diet and removal of dessert entirely, complete emotional reorientation to food, a mild prescription stimulant, and varying levels of exercise did nothing, and ketone esters worked better than all of those but much worse than potatoes. Comments not attempting this will be deleted or mocked as I see fit.]

If you are thinking “ah, but clearly those all did contribute and the potatoes were just the last step”: I agree that’s likely. If I’d started minimal potato diet before BosPro it either wouldn’t have worked or would have been extremely bad for me. But since it seems to work for at least some other people who didn’t have all this baggage I think we need to update in that direction.]

Or take every person who got a second opinion on their cancer and was recommended diametrically opposing treatment plans. Doctors as a class are not as epistemically virtuous as I’d like, but that’s not (always) why they propose wildly divergent treatment plans. In most cases it’s because the answer isn’t obvious, or at best has only been obvious for a few years.

And then there’s the absolute shitshow that is nutrition research. No one knows what the average optimum nutrient level is and even if we did it wouldn’t be that helpful for figuring out the optimum level for a given individual, because humans are so unbelievably variable.

I could go on here, but if you’re reading my blog you’re probably already on board with shit being extremely complicated and I don’t want to belabor the point.

Moral of the story: when intellect fails, try luck guided by intuition

Some medicine is very deterministic. Antibiotics, most of the time. That daylong IV drip when I had norovirus that probably turned the infection from deadly to a kind of annoying 36 hours. We may not know the optimum level of a given nutrient but most severe deficiency diseases can be solved by giving you the thing you’re severely deficient in. My impression is statins work pretty reliably.

But a lot of medicine just seems to be kind of random. People go through 10 antidepressants and then somehow the 11th one works great. Ketone esters increase my energy level so much I gave up soda and caffeine entirely but do nothing for most people. All those books where the cure was a miracle for someone, and it can’t just be a placebo because there’s no reason for the 35th placebo to be the one that works but nothing else makes sense.

All of which leads me to conclude that once you have exhausted the reliable part of medicine without solving your problem, looking for a mechanistic understanding or even empirical validation of potential solutions is a waste of time. The best use of energy is to try shit until you get lucky.

Not at random or anything. My guess is the world contains metis and you do better-than-chance preferentially trying things that helped one guy on a message board for your condition (even though it was shown to make no difference in real studies) or going to alt-modality practitioners (even the one with proactively stupid justifications they insist on sharing). The latter is especially true if you can find a practitioner that accepts that their treatments don’t always work and have a system to notice that and change course, but I think maybe even the really gung-ho ones sometimes have good ideas (you just have to set up your own system for deciding when to quit). Just don’t get hung up on “do we understand why this works?” or “does this work for other people?”

Also please remember that side effects and drug interactions are a thing. Anything with a real effect can hurt you. I gave a very caveated suggestion of BosPro to someone on Twitter and it caused something akin to niacin flush in them. This is the same brand that does nothing to me but makes me better at digestion and uninterested in sugar.

So I guess the full and accurate statement of my beliefs is “Try solving problems with understanding first, but accept when you’ve hit diminishing returns and consider if your energy isn’t better spent increasing your surface area to luck”.

Parting shots

Fuck you every doctor who told me my digestive problems were in my head or my fault for being a bad patient and you couldn’t help me until I solved the problem that drove me to you. You were factually incorrect and you should feel terrible.

For potential clients in particular

People sometimes approach me for medical literature reviews aimed at their specific problem. There are forms of these I will do, but those forms do not include producing a mechanistic model and high-probability treatment for someone’s persistent, sub-clinical, amorphous problem that medicine has failed to solve. There are a few reasons accepting these commisions would be wasting the clients’ money, and one of them is that by the time they come to me they have found all the low hanging deterministic fruit. The best I can do is spend a ton of time generating lists of things that might work. Sometimes I do offer that, but people tend to prefer my other offer of a referral to a researcher that’s better at individualized treatment.

The Balto/Togo theory of scientific development

Tragically I gave up on the Plate Tectonics study before answering my most important question: “Is Alfred Wegener the Balto of plate tectonics?”

Let me back up.

Balto

Balto is a famous sled dog. He got a statue in NYC for leading a team of dogs through a blizzard to deliver antibody serum to Nome, Alaska in 1925, ending a diphtheria outbreak. Later Disney made a movie about how great he was.

Except that run was a relay, and Balto only got famous because he did the last leg, which had the most press coverage but was also the easiest. The real hero was Togo, the dog who led the team through the hardest terrain and covered by far the most miles as well. Disney later made a movie about him that makes no mention of Balto for the first 90%, and then goes out of its way to talk about what a shit dog he was, that’s why he didn’t get included in any of the important teams, but Togo had had to do so many hard things they needed a backup team for the trivial last leg so Balto would have to do.

Togo’s owner died mad about the US mainland believing Balto was a hero. But since all the breeders knew who did the hard part Togo enjoyed a post-Nome level of reproductive success that Ghengis Khan could only dream about, so I feel like he was happy with his choices.

plus he did eventually get some statues

But it’s not like Togo did this alone either. He led one team in a relay, and there were 20 humans and 150 dogs that contributed to the overall run. Plus someone had to invent the serum, manufacture it, and get it to the start of the dog relay at Nenana, Alaska. So exactly how much credit should Togo get here?

The part with Wegener

I was pretty sure Alfred Wegener, popularly credited as the discoverer/inventor of continental drift and mentioned more prominently than any other scientist in discussions of plate tectonics, is a Balto.

First of all, continental drift is not plate tectonics. Continental drift is an idea that maybe some stuff happened one time. Plate tectonics is a paradigm with a mechanism that makes predictions and explains a lot of data no one knew was related until that moment.

Second, Wegener didn’t discover any of the evidence he cited, he wasn’t the first to have the idea, and it’s not even clear he did much of the synthesis of the evidence. His original paper refers to “Concerning South America and Africa, biologists and geologists are in close agreement that a Brazilian–African continent existed in the Mesozoic”

So he didn’t invent the idea, gather the data, or even really synthesize the evidence. His guess at the mechanism was wrong. But despite spending hours digging into the specific discovers and synthesizers that contributed to plate tectonics, the only name I remember is Wegener’s. Classic Balto.

On the other hand, some of the people who gathered the data used to discover plate tectonics were motivated by the concept of continental drift, and by Wegener specifically. That seems like it should count for something. My collaborator Jasen Murray thinks it counts for a lot

Jasen would go so far as to argue that shining a beacon in unknown territory that inspires explorers to look for treasure in the right place makes you the Togo, racing through fractured ice rapids social ridicule and self-doubt to do the real work of getting an idea considered at all. Showing up at the finish line to formalize a theory after there’s enough work to know it’s true is Balto work to him. This makes me profoundly uncomfortable because strongly advocating for something unproven terrifies me, but as counterargument arguments go that’s pretty weak.

One difficulty is it’s hard to distinguish “ahead of their time beacon shining” from “lucky idiot”, and even Jasen admits he doesn’t know enough to claim Wegener in particular is a Togo. But doing work that is harder to credit because it’s less legible is also very Togo-like behavior, so this proves nothing about the category. 

So I guess one of my new research questions is “how important are popularizers?” and I hate it.