Review of Examine.com’s vitamin write-ups

There are a lot of vitamins and other supplements in the world, way more than I have time to investigate. Examine.com has a pretty good reputation for its reports on vitamins and supplements. It would be extremely convenient for me if this reputation was merited. So I asked Martin Bernstoff to spot check some of their reports. 

We originally wanted a fairly thorough review of multiple Examine write-ups. Alas, Martin felt the press of grad school after two shallow reviews and had to step back. This is still enough to be useful so we wanted to share, but please keep in mind its limitations. And if you feel motivated to contribute checks of more articles, please reach out to me (elizabeth@acesounderglass.com).

My (Elizabeth’s) tentative conclusion is that it would take tens of hours to beat an Examine general write-up, but they are not complete in either their list of topics nor their investigation into individual topics. If a particular effect is important to you, you will still need to do your own research.

Photo credit DALL-E

Write-Ups

Vitamin B12

Claim: “The actual rate of deficiency [of B12] is quite variable and it isn’t fully known what it is, but elderly persons (above 65), vegetarians, or those with digestion or intestinal complications are almost always at a higher risk than otherwise healthy and omnivorous youth”

Verdict: True but not well cited. Their citation merely asserts that these groups have shortages rather than providing measurements, but Martin found a meta-analysis making the same claim for vegetarians (the only group he looked for).

Toxicology

Verdict: Very brief. Couldn’t find much on my own. Seems reasonable.

Claim: “Vitamin B12 can be measured in the blood by serum B12 concentrations, which is reproducible and reliable but may not accurately reflect bodily vitamin B12 stores (as low B12 concentrations in plasma or vitamin B12 deficiencies do not always coexist in a reliable manner[19][26][27]) with a predictive value being reported to be as low as 22%”

Verdict: True, the positive predictive value was 22%, but with a negative predictive value of 100% at the chosen threshold. But that’s only the numbers at one threshold. To know whether this is good or bad, we’d have to get numbers at different threshold (or, preferably, a ROC-AUC).

Claim: B12 supplements can improve depression

Examine reviews a handful of observational studies showing a correlation, but includes no RCTs.  This is in spite of there actually being RCTs like Koning et al. 2016 and a full meta analysis, neither of which find an effect. 

The lack of effect in RCTs is less damning than it sounds. I (Elizabeth) haven’t checked all of the studies, but the Koning study didn’t confine itself to subjects with low B12 and only tested serum B12 at baseline, not after treatment. So they have ruled out neither “low B12 can cause depression, but so do a lot of other things” nor “B12 can work but they used the wrong form”.

I still find it concerning that Examine didn’t even mention the RCTs, and I don’t have any reason to believe their correlational studies are any better. 

Interactions with pregnancy

Only one study on acute lymphoblastic leukemia. Seems a weird choice. Large meta-analyses exist for pre-term birth and low birth weight, likely much more important. Rogne et al. 2016.

Overall

They don’t seem to be saying much wrong but the write-up is not nearly as comprehensive as we had hoped. To give Examine its best shot, we decided the next vitamin should be on their best write-up. We tried asking Examine which article they are especially confident in. Unfortunately, whoever handles their public email address didn’t get the point after 3 emails, so Martin made his best guess. 

Vitamin D

Upper respiratory tract infections.

They summarize several studies but miss a very large RCT published in JAMA, the VIDARIS trial. All studies (including the VIDARIS trial) show no effect, so they might’ve considered the matter settled and stopped looking for more trials, which seems reasonable.

Claim: Vitamin D helps premenstrual syndrome

”Most studies have found a decrease in general symptoms when given to women with vitamin D deficiency, some finding notable reductions and some finding small reductions. It’s currently not known why studies differ, and more research is needed”

This summary seemed optimistic after Martin looked into the studies:

  • Abdollahi 2019:
    • No statistically significant differences between groups.
    • The authors highlight statistically significant decreases for a handful of symptoms in the Vitamin D group, but the decrease is similar in magnitude to placebo. Vitamin D and placebo both have 5 outcomes which were statistically significant.
  • Dadkhah 2016:
    • No statistically significant differences between treatment groups
  • Bahrami 2018:
    • No control group
  • Heidari 2019:
    • Marked differences between groups, but absolutely terrible reporting by the authors – they don’t even mention this difference in the abstract. This makes me (Martin) somewhat worried about the results – if they knew what they were doing, they’d focus the abstract on the difference in differences.:
  • Tartagni 2015:
    • Appears to show notable differences between groups, But terrible reporting. Tests change relative to baseline (?!), rather than differences in trends or differences in differences. 

In conclusion, only the poorest research finds effects – not a great indicator of a promising intervention. But Examine didn’t miss any obvious studies.

Claim: “There is some evidence that vitamin D may improve inflammation and clinical symptoms in COVID-19 patients, but this may not hold true with all dosing regimens. So far, a few studies have shown that high dosages for 8–14 days may work, but a single high dose isn’t likely to have the same benefit.”

The evidence Martin found seems to support their conclusions. They’re missing one relatively large, recent study (De Niet 2022). More importantly, all included studies are about hospital patients given vitamin D after admission, which are useless for determining if Vitamin D is a good preventative, especially because some forms of vitamin D take days to be turned into a useful form in the body. 

  • Murai 2021:
    • The regimen was a single, high dose at admission.
    • No statistically significant differences between groups, all the effect sizes are tiny or non-existent.
  • Sabico 2021:
    • Compares Vitamin D 5000 IU/daily to 1000 IU/daily in hospitalized patients.
    • In the Vitamin D group, they show faster
      • Time to recovery (6.2 ± 0.8 versus 9.1 ± 0.8; p = 0.039)
      • Time to restoration of taste (11.4 ± 1.0 versus 16.9 ± 1.7; p = 0.035)
        • The Kaplan-Meier Plot looks weird here, though. What happens on day 14?!
    • All symptom durations, except sore throat, were lower in the 5000 IU group:

All analyses were adjusted for age, BMI and type of D vitamin – which is a good thing, because it appears the 5000 IU group was healthier at baseline:

  • Castillo 2020:
    • Huge effect – half of the control group had to go to the ICU, whereas only one person in the intervention group did so (OR 0.02).
    • Nothing apparently wrong, but I’m still highly suspicious of the study:
      • An apparently well-done randomized pilot trial, early on, published in “The Journal of Steroid Biochemistry and Molecular Biology”. Very worrying that it isn’t published somewhere more prestigious.
      • They gave hydroxychloroquine as the “best available treatment”, even though there was no evidence of effect at the time of the study.
      • They call the study “double masked” – I hope this means double-blinded, because otherwise the study is close to worthless since their primary outcomes are based on doctor’s behavior.
      • The follow-up study is still recruiting.

Conclusion

I don’t know of a better comprehensive resource than Examine.com. It is alas still not comprehensive enough for important use cases, but still a useful shortcut for smaller problems.

Thanks to the FTX Regrant program for funding this post, and Martin for doing most of the work.

I Caught Covid And All I Got Was This Lousy Ambiguous Data

Tl;dr I tried to run an n of 1 study on niacin and covid, and it failed to confirm or disprove anything at all.

You may remember that back in October I published a very long post investigating a niacin-based treatment protocol for long covid. My overall conclusion was “seems promising but not a slam dunk; I expect more rigorous investigation to show nothing but we should definitely check”. 

Well recently I got covid and had run out of more productive things I was capable of doing, so decided to test the niacin theory. I learned nothing but it was a lot of effort and I deserve a blog post out of it null results are still results so I’m sharing anyway.

Background On Niacin

Niacin is a B-vitamin used in a ton of metabolic processes. If you’re really curious, I describe it in excruciating detail in the original post.

All B vitamins are water-soluble, and it is generaly believed that unless you take unbelievably stupid doses you will pee out any excess intake without noticing. It’s much harder to build up stores of water-soluble vitamins than fat vitamins, so you need a more regular supply.  Niacin is a little weird among the water-solubles in that it gives very obvious signs of overdose: called flush, the symptoms consist of itchy skin and feeling overheated. Large doses can lead to uncontrolled shaking, but why would you ever take that much, when it’s so easy to avoid?

People regularly report response patterns that sure look like their body has a store of niacin that can be depleted and refilled over time. A dose someone has been taking for weeks or months will suddenly start giving them flush, and if they don’t lower it the flush symptoms will get worse and worse. 

Some forms of niacin don’t produce flush. Open question if those offer the same benefits with no side effects, offer fewer benefits, or are completely useless.

Niacin And Long Covid

There’s an elaborate hypothesis about how covid depletes niacin (and downstream products), and this is a contributor to long covid. My full analysis is here. As of last year I hadn’t had covid (this is antibody test confirmed, I definitely didn’t have an asymptomatic case) but I did have lingering symptoms from my vaccine and not a lot else to try, so I gave the protocol a shot.

My experience was pretty consistent with the niacin-storage theory. I spent a long time at quite a high dose of the form of niacin the protocol recommends, nictonic acid. My peak dose without flush was at  least 250mg (1563% RDA) and maybe even 375mg (2345% RDA). When I hit my limit I lowered my dose until I started getting flush at the new dose, and eventually went off nicotnic acid entirely (although I restarted a B-vitamin that included 313% RDA of a different form). That ended in September or early October 2021. It made no difference in my lingering vaccine symptoms.

In early 2022 I tried nicotinic acid again. Even ¼ tablet (62.5mg, 390% RDA) gave me flush.

I Get Covid

Once I developed symptoms and had done all the more obviously useful things like getting Paxlovid, I decided it would be fun to test myself with niacin (and the rest of the supplement stack discussed in my post) and see if covid had any effect. So during my two weeks of illness and week of recovery I occasionally took nicotinic acid and recorded my results. Here’s the overall timeline:

  1. Day -2: am exposed to covid.
  2. Day 0: test positive on a cue test (a home test that uses genetic amplification).
    1. Lung capacity test: 470 (over 400 is considered health).
    2. Start Fluvamoxine and the vitamin cocktail, although I’m inconsistent with both the new and existing vitamins during the worst of the illness. Vitamin cocktail includes 313% RDA of no-flush niacin, but not nicotinic acid. 
  3. Day 1: symptomatic AF. 102.3 degree fever, awake only long enough to pee, refill my water, and make sure my O2 saturation isn’t going to kill me. I eat nothing the entire day.
    1. I monitored my O2 throughout this adventure but it never went into a dangerous zone so I’m leaving it out of the rest of the story.
  4. Day 2: start with 99 degree fever, end day with no fever. Start Paxlovid.
    1. Every day after this I am awake a little bit longer, eat a little bit more, and have a little more cognitive energy, although it takes a while to get back to normal. 
    2. Try ¼ tab nicotinic acid (62.5 mg/ 375% RDA), no flush.
    3. Lung capacity troughs at 350 (considered orange zone).
  5. Day 4: ½ tablet nictonic acid, mild flush.
  6. Day 7: lung capacity up to 450, it will continue to vary from 430-450 for the next two weeks before occasionally going higher.
  7. Day 9: ½ tablet nictonic acid, mild flush
  8. Day 10-17: ⅓ tablet nictonic acid, no flush
    1. Where by “⅓” tablet I mean “I bit off an amount of pill that was definitely >¼ and <½ and probably averaged to ~⅓ over time”
  9. Day 12: I test positive on a home antigen test
  10. Day 15: I test negative on a home antigen test (no tests in between) 
  11. Day 17: ⅓ tablet produces flush (and a second negative antigen test)
    1. This was also the first day I left my house. I had thought of myself as still prone to fatigue but ended up having a lot of energy once I got out of my house and have been pretty okay since.

Conclusions

My case of covid was about as bad as you get while still technically counting as mild. Assuming I went into it with niacin stores such that 62.5mg nicotinic acid would generate flush, it looks like covid immediately took a small bite out of them. Or it reduced my absorption of vitamins, such that the same oral dosage resulted in less niacin being taken in. There’s no way to know covid had a larger effect on niacin than other illnesses, because I don’t have any to compare it to. Or maybe the whole thing was an artifact of “not eating for two days, and then only barely, and being inconsistent with my vitamins for a week”.

Kencko Fruit Powder: Better Than Anything I’m Actually Going to do

UPDATE: As much as I love the concept of Kencko, if I drink them too fast they make me vomit, even when very diluted. I reluctantly withdraw my seal of approval.

 

Sometimes the modern economy really delivers.

As longtime readers know, I have two strikes against me when it comes to food: it requires both chewing and digesting. Chewing is painful for me due to nerve damage in my jaw. Digesting… well some of the problems with digestion are caused by insufficient stomach acid, but those are easily treated with a pill. I still have problems when I take the pills and no one knows why. So I eat a lot of things that require minimal chewing and are easy to digest. This set of things has very little overlap with the set of things my nutritionist wants me to eat, such as produce. I eat some fruits and veggies, especially in the summer, but not nearly enough.

Enter Kencko.

 

 

Kencko produces small packets of powderized fruits and vegetables. This requires no chewing and substantially less digestion. They taste fine. Not amazing, but fine. You could probably make them taste better by adding sugar or honey. Because they’re produced by freeze drying, they’re better nutritionally than other preserved fruit. Not as good as fresh, but, in the words of my nutritionist, “better than anything you’re actually going to do.” 

Nutritionists are hit-or-miss, so I double checked the nutrition claim myself. Based on a rushed review (primary source), I find that freeze drying has some nutritional loss, exact amount depending on the nutrient, but within the range that could conceivably counterbalanced by the increased digestibility of powder (this also means the sugar hits faster), and also the fact that I’m eating them at all. I suspect the biggest loss is the absence of probiotic flora in the sterilized powder packets.

There’s the issue of price. I was originally going to apologize for the price, chalk it up to convenience, and plead necessity for myself, but it turns out the packets are not that expensive relative to comparables. Ordered in the largest size, Kencko is $3.07/ounce. I spent 45 minutes finding prices for other freeze dried fruit powders, and that’s as good as you can do short of wholesale (spreadsheet). There are cheaper powders, but they’re inevitably something other than freeze-dried.

How about compared to actual fruit? It takes .44 lbs of fruit to produce one 20 g Kencko packet (price: $2.16-$3.30, depending on quantity ordered). According to this USDA report (chart on page 3), .5 lbs is $0.66-$0.90 cents worth of apples, $2.16 worth of blueberries, or $2.10-$2.85 worth of cherries. Note that those are advertised prices, so probably less than what you’d pay on average and certainly less than what you’d pay out of season, and for conventional produce rather than the organics Kencko uses. Kencko is definitely more expensive than in-season, on-sale produce, but not ridiculously so. Plus it never goes bad so you’re not paying for produce you throw away.

The worst thing I will say about Kencko is that their mixer bottle sucks. It mixes less well and is harder to clean than a Blender Bottle (affiliate link), buy one of those or use a spoon.

Obviously if you can just eat a vegetable you should do that. But if you find that untenable for some reason, Kencko is a reasonable way to turn money into consumed produce. This is an incredibly good trade for me and I’m really happy it exists.

[Kencko has not paid me for this post and I’m not in contact with them beyond ordering the product and following them on Twitter.]

Manganese

In summary: Manganese is an essential nutrient and too much will kill you.

On one hand we have people exposed to manganese is the air from industrial processes. These people definitely get Parkinson’s like symptoms, and in the aggregate score lower on various intellectual tasks. Monkey infants on soy formula (which has more manganese than milk based formulas) had mild neurological effects, so you can take in enough from food alone to be dangerous.

On the other hand, manganese superoxide dismutase is the only thing keeping your mitochondria from melting, and it does several other things besides. So we can’t just throw up our hands and say “no more manganese”, we have to actually figure out a safe level.

The World Health Organization has set the No Observed Adverse Effects Level at 11mg/day, based on the observation that some humans naturally get that much through their diet. As mentioned previously, we know you can get manganese poisoning through food alone, so this is not very reassuring to me.

The most comprehensive source I found on this was this thread on the soylent message boards (they cite several of the same things I did above, all correctly, so I trust their work ). According to their sources, most adults already eat enough Manganese to meet the RDA. I don’t want to put too much stock in the RDA, because neither I nor they could find out how the FDA (or the Linus Pauling Institute) set it. However even if you accept the number, if people are already getting it through diet, and too much is toxic, it doesn’t belong in a multivitamin.

This is worrisome enough to me that I’m tossing my multivitamin and focusing on getting individual missing nutrients through specific supplements (if anyone knows of a balanced B supplement, please let me know). It’s not so bad I’m tossing my protein/vitamin powder, because it has a lower volume of manganese and I don’t eat much at a time- however when it’s gone, I’ll probably switch to a straight protein powder.

 

Bone Broth

Bone broth is having its moment- paleo likes it, nut jobs who believe vaccines cause autism like it*, whoever the hell these people are like it, my nutritionist is a big fan.  The idea seems obvious- bones are full of nutrients that hard to get, especially in the typical American diet, surely drinking bones would be good.  Especially for calcium.  Everyone knows bones have calcium.

I got suspicious when I noticed that the nutritional label on my broth** reported 0% of my RDA of calcium.  I checked a few more brands, the top contender lists 2% calcium RDA and 4% iron/6 grams of protein.  Most list 0.  nutritiondata.self.com gives considerably better numbers, but no source.  Their listing contains a good deal more fat (9g, as opposed to 0 in any of the commercial broth I’ve found) and a non-zero amount of carbs.  None of the micronutrients they listed (vitamin C, a few Bs, iron, calcium, manganese) are fat soluble, but maybe there is something to preparing it at home.

Some of the websites touting bone broth list other substances that aren’t on nutritional labels but they believe are important.  I am well disposed to believe this claim.  There is no reason to believe science knows all the micronutrients we need, much less a USDA oriented towards the well being of farmers, not consumers.  The specifically mention glycosaminoglycans, a class protein/sugar hybrid found in joints.  This seems utterly plausible, but I was unable to find any numbers of this. At all.

I found one scientific paper on bone broth.  It is in Korean***.  It has some English but not enough for me to actually determine the micronutrient:protein ratio.  Beyond that you have studies about the components of bone broth and the assumption that it will be absorbed in this form.  For example this paper on collagen and rheumatoid arthritis (PDF).  Given it has 60 people and RA is a cyclic disease, their results are actually pretty good, but that still leaves it open to any number of manipulations.  The second best paper is a press relief on an informal study of chicken soup.

That leaves protein.  Everyone agrees bone broth has serious protein, but unfortunately not the most important kind.  Protein is made up of amino acids, of which there are two kinds: non-essential (which your body can manufacture) and essential (which you must take in via diet).  The RDA for protein is 0.8 g/kg body weight, of which 0.1 g should be of the essential amino acids (there are per-acid requirements but I’m not tracking 9 individual requirements), so 20% is putting  you ahead of the game, except that broth is missing two EAAs entirely.  After 20  hours of cooking (see korean) paper, 25% of the amino acids are glycine.  For comparison:

  • The pumpkin-based protein powder in front of me is 20% essential amino acids (and has way more iron than broth)
  • Whey protein is 60% EAAs.
  • Soy is 34%

I have trouble digesting protein and find bone broth stunningly easy to digest, so this is still a win for me, but it’s a slam dunk.

While traveling I’m using bone broth powder, which I’m increasingly convinced is a fancy way of saying “bone-based protein powder”.  I’m okay with bone based protein powder, although I might not have packed the pumpkin if I’d realized this.

Do you know what else is basically a protein powder?  Cricket flour.  They taste similar, cricket has a better amino acid profile (25% essential) and more trace nutrients (although I’m still tracking down how many more).  It is also cheaper, which should make !broth feel bad.

I find it more plausible than the average miracle food that bone broth has effects beyond what you’d expect from a naive read of the nutrition facts, because I expect animal bone + meat to fulfill a broader range of requirements than some berry.   I do feel better when I drink it, but a lifetime of digestive and chewing problems has given me a tendency to develop food security blankets, and broth is currently filling that role.   Simply by being a security blanket that is not jelly beans or peanut butter cups****, broth is a health food for me, personally, but I can’t really extrapolate beyond that. The current press around it appears to be almost entirely groundless.

Once again, the state of nutritional knowledge is embarrassingly bad and I would like us to shift money towards increasing it.  Also why the hell can’t I test the  nutritional content of broth I make myself?

 

*To their credit, they have an explanation that doesn’t rely on mercury, which has been extremely thoroughly disproven.  If they had presented it as a fringe thing they needed to prove, I would have entertained their hypothesis.  They presented it as fact, without any attempt to distances themselves from the atrocious denialism of the mercury-based anti-vaxxers.

**I buy it frozen on the theory that my time and not having my house continually smell of meat was worth the extra money.

***I don’t think this would be hard to determine if you read Korean, volunteers would be welcome.

****Trader Joe’s brand- I’m not an animal.

ETA: I only just learned that bone broth means “bones + connective tissue”.  Clean bones give you hardly any protein, even if there’s marrow in them.  Apparently I don’t need to pay $10/bag for store made stuff, I can use $8 worth of chicken feet and liver and eat for a week.

100% Food versus Soylent

I really, really wanted to like 100% food.  It has so much going for it- made of actual grown food, high protein variants, convenient portable single serve containers, shipping time measured in days rather than months… Unfortunately, it is inedible.

And if I’m saying inedible, it must be pretty bad.  I drink Soylent at room temperature, and for a long time didn’t bother mixing up the little protein clumps that formed (although once I realized how easy to clean my blender was, I did mix it).  As I write this I’m consuming my smoothie with a spoon because I put in too many chia seeds and it turned into a gel.  I cannot drink 100% food.  I ordered the six-variant mixer (chocolate/plain and regular/high protein/low carb), but I’ve only managed to try two (one low carb, one regular) and one of those was a mere pinch.  Extremely diluted the taste is tolerable, but still leaves this unbelievably gross oil aftertaste/film in your mouth.  I won’t drink it and you can’t make me.

The single serve containers were a disappointment too. There’s not enough room in them to really shake up the mix, so you end up with a lot of dry mix protected by a layer of wet mix.  So 100% food’s only real selling point is that it ships quickly.

I’m not 100% happy with Soylent.  I would really prefer fewer digestible carbs and more protein and fiber, but that’s addressable with some flax seeds and additional protein powder.  I would also love it if they shipped in < 3 months, and didn’t tell me they were going to ship a month before they did.  But it is better than starving to death, which is more than I can say for 100% food.