Epistemic Spot Check Changes

Previously I checked books pretty much as I went along.  Doing otherwise felt like the check was playacting; if knowing wasn’t going to affect my behavior towards the book it was rigor theater, not genuinely caring about a book’s factual accuracy.  I’m backing off on that.  It prevents a book from gathering any momentum and it’s too easy to turn into cheap shots. Instead, for the mental health books, my default is going to be read the introduction and at least one other chapter, marking what I want to verify as I go, and circle back at the end of the second chapter to fact check.  This will give the books a little time to breathe and for me to evaluate their model.  It’s also way easier for me.

The exception  will be books where I get a stuck feeling and need to look something up to continue, or where I get nerd sniped.

Epistemic Spot Check: A Guide To Better Movement (Todd Hargrove)

This is part of an ongoing series assessing where the epistemic bar should be for self-help books.


Thesis: increasing your physical capabilities is more often a matter of teaching your neurological system than it is anything to do with your body directly.  This includes things that really really look like they’re about physical constraints, like strength and flexibility.  You can treat injuries and pain and improve performance by working on the nervous system alone.  More surprising, treating these physical issues will have spillover effects, improving your mental and emotional health. A Guide To Better Movement provides both specific exercises for treating those issues and general principles that can be applied to any movement art or therapy.

The first chapter of this book failed spot checking pretty hard.  If I hadn’t had a very strong recommendation from a friend (“I didn’t take pain medication after two shoulder surgeries” strong), I would have tossed it aside.  But I’m glad I kept going, because it turned out to be quite valuable (this is what triggered that meta post on epistemic spot checking).  In accordance with the previous announcement on epistemic spot checking, I’m presenting the checks of chapter one (which failed, badly), and chapter six (which contains the best explanation of pain psychology I’ve ever seen), and a review of model quality.  I’m very eager for feedback on how this works for people.

Chapter 1: Intro (of the book)

Claim: “Although we might imagine we are lengthening muscle by stretching, it is more likely that increased range of motion is caused by changes in the nervous system’s tolerance to stretch, rather than actual length changes in muscles. ” (p. 5). 

Overstated, weak.  (PDF).  The paper’s claims to apply this up to 8 weeks, no further.  Additionally, the paper draws most (all?) of its data from two studies and it doesn’t give the sample size of either.

Claim:  “Research shows the forces required to deform mature connective tissue are probably impossible to create with hands, elbows or foam rollers.” (p. 5). 

Misleading. (Abstract).  Where by “research” the Hargrove means “mathematical model extrapolated from a single subject”.

Claim:  “in hockey players, strong adductors are far more protective against groin strain than flexible adductors, which offer no benefit” (p. 14).

Misleading. (Abstract) Sample size is small, and the study was of the relative strength of adductor to abductor, not absolute strength.

Claim: “Flexibility in the muscles of the posterior chain correlates with slower running and poor running economy.” (p. 14).

Accurate citation, weak study.  (Abstract) Sample size: 8.  Eight.  And it’s correlational.

[A number of interesting ideas whose citations are in books and thus inaccessible to me]

Claim:  “…most studies looking at measurable differences in posture between individuals find that such differences do not predict differences in chronic pain levels.”  (p. 31). 

Accurate citation.  (Abstract).  It’s a metastudy and I didn’t track down any of the 54 studies included, but the results are definitely quoted accurately.


Chapter 6: Pain

Claim: “Neuromatrix” approach to pain means the pattern of brain activity that create pain, and that pain is an output of brain activity, not an input (p93).

True, although the ability to correctly use definitions is not very impressive.

Claim: “If you think a particular stimulus will cause pain, then pain is more likely.  Cancer patients will feel more pain if they believe the pain heralds the return of cancer, rather than being a natural part of the healing process.” (p93).

Correctly cited, small sample size. (Source 1, source 2, TEDx Talk).

ClaimPsychological states associated with mood disorders (depression, anxiety, learned helplessness, etc) are associated with pain (p94).

True, (source), although it doesn’t look like the study is trying to establish causality.

ClaimMany pain-free people have the kinds of injuries doctors blame pain on (p95).

True, many sources, all with small sample sizes.  (source 1, source 2, source 3, source 4, source 5)

Claim: On taking some cure for pain, relief kicks in before the chemical has a chance to do any work (p98)

True.  His source for this was a little opaque but I’ve seen this fact validated many other places.

Claim: we know you can have pain without stimulus because you can have arm pain without an arm (p102).

True, phantom limb pain is well established.

Claim: some people feel a heart attack as arm pain because the nerves are very close to each other and the heart basically never hurts, so the brain “corrects” the signal to originating in the arm (p102).

First part: True.  Explanation: unsupported.  The explanation certainly makes sense, but he provides no citations and I can’t find any other source on it.

Claim: Inflammation lowers the firing threshold of nociceptors (aka sensitization) (p102).

True (source).

Claim: nociception is processed by the dorsal horn in the spine.  The dorsal horn can also become sensitized, firing with less stimulus than it otherwise would.  Constant activation is one of the things that increases sensitivity, which is one mechanism for chronic pain (p103).

True (source).

Claim: people with chronic pain often have poor “body maps”, meaning that their mental model of where they are in space is inaccurate and they have less resolution when assessing where a given sensation is coming from (p107).

Accurate citation (source).  This is a combination of literature review and reporting of novel results.  The novel results had a sample of five.

Claim: The hidden hand in the rubber hand illusion experiences a drop in temperature (p109).

Accurate citation, tiny sample size (source).  This paper, which is cited by the book’s citation, contains six experiments with sample sizes of fifteen or less.  I am torn between dismissing this because cool results with tiny sample sizes are usually bullshit, and accepting it because it is super cool.

Claim: “a hand that has been disowned through use of the rubber hand illusion will suffer more inflammation in response to a physical insult than a normal hand.” (p. 109).

Almost accurate citation (source).  The study was about histamine injection, not injury per se.   Insult technically covers both, but I would have preferred a more precise phrasing.  Also, sample size 34.

Claim: People with chronic back pain have trouble perceiving the outline of their back (p. 109). 

Accurate citation, sample size six (pdf).

Claim:  “Watching the movements in a mirror makes the movements less painful [for people with lower back pain].” (p. 111). Better Movement. Kindle Edition.

Accurate citation, small sample size (source).

Model Quality

Reminder: the model is that pain and exhaustion are a product of your brain processing a variety of information.  The prediction is that improving the quality of processing via the principles explained in the book can reduce pain and increase your physical capabilities.

Simplicity: Good.  This is not actually simple model, it requires a ton of explanation to a layman.  But most of its assumptions come from neurology as a whole; the leap from “more or less accepted facts about neurology” to this model is quite small.

Explanation Quality: Fantastic.  I’ve done some reading on pain psychology, much of which is consistent with Guide…, but Guide… has by far the best explanation I’ve read.

Explicit Predictions: Good, kept from greatness only by the fact that brains and bodies are both very complicated and there’s only so much even a very good model can do.

Useful Predictions: Okay. The testable prediction for the home-reader is that following the exercises in the back of the book, or going to a Feldenkrais class, will treat chronic pain, and increase flexibility and strength.  Since the book itself admits that a lot of things offer short term relief but don’t address the real problem, helping immediately doesn’t prove very much.

Acknowledging Limitations: Poor.  GTBM doesn’t have the grandiose vision of some cure-all books, and repeatedly reminds you that your brain being involved doesn’t mean your brain is in control.  But there’s no sentence along the lines of “if this doesn’t work there’s a mechanical problem and you should see a doctor.”

Measurability: poor.  This book expects you to put in a lot of time before seeing results, and does not make a specific prediction of the form they will come in.  Worse, I don’t think you can skip straight to the exercises.  If I hadn’t read the entire preceding book I wouldn’t have approached them in the correct spirit of attention and curiosity.

Hmmm, if I’d assigned a gestalt rating it would have been higher than what I now think is merited based on the subscores.  I deliberately wrote this mostly before trying the exercises, so I can’t give an effectiveness score.  If you do decide to try it, please let me know how it goes so I can further calibrate my reviews to actual effectiveness.


You might like this book if…

…you suffer from chronic pain or musculoskeletal issues, or find the mind-body connection fascinating.

This post supported by Patreon.

How to Handle Bad Examples in Texts?

How harshly should you judge when a scientific work gives a bad example of its point?

For example, I am reading Interaction Ritual Chains (Randall Collins), which focuses on emotional energy during stereotyped interactions between people (this is the scientific sense of stereotyped, meaning rigid or strongly patterned, not racist).  IRC believes that interaction is important and irreplaceable, such that people will shun much more “efficient” solutions to their official goal in order to get interpersonal interactions.  Unfortunately it uses terrible examples to illustrate this.

For example, on page 63 he claims that online shopping will never replace brick and mortar stores, because going to the store amidst other people is an energizing ritual.  At the time the book was published (2004) this was obviously untrue to me, but I can see how it wasn’t in every segment of society.  So while this is a bad prediction, it’s not a factual error.

He also claims that television has had no impact on sports attendance because people want the in person experience so much (p57).  To the extent that is true, it’s because leagues has gone to a great deal of trouble to make it not true, by imposing blackout rules such that you can’t broadcast a local game unless it has sold out.  It’s only since 1973 that the NFL had an exception for sold out games- previous to that, the only way to see a game played in your market was to be there.

How much should I reduce my confidence in Interaction Ritual Chains, given that it made this error? Or two false predictions?  Is it even fair to score them as false? In person stores still exist, although malls are having a hard time of it.  Maybe blackout rules were preemptive strikes and attendance would have stayed high without them.  But this gives me qualms about learning examples I know less about- even if the author is being accurate, if I’m drawing incorrect conclusions.

Epistemic Spot Check: The Demon Under the Microscope (Thomas Hager)


How much would it suck to be the guy who invented sulfa drugs? You dedicate your life to preventing a repeat of the horrors you saw in the war, succeed in that and so much more, and then 10 years later some idiot leaves a petri dish open and completely replaces you as the father of man’s triumph against bacteria.  Actually he left the lid off before you found your thing, but ignored the result until you hit it big because everyone knew you couldn’t fight disease with chemicals, until you proved you could.  It’s the ultimate silver medal.  The Demon Under the Microscope is the tale of that guy.

It’s by the same author (Thomas Hagen) as The Alchemy of Air.  It’s also set in the same corporation, and about field that was transforming from science to industry.  The writing style is similar.  I originally didn’t intend to fact check this book very hard because I already knew what to expect from the author (a little too invested in the subject but basically accurate), but the habit is too ingrained at this point and I couldn’t keep reading until I’d checked out the first few chapters.



Claim: “Domagk [the researcher] had the ability to see. He watched everything, noted slight variations, quietly filed it all away.”  (p. 18).

The wounds themselves he accepted as the results of war. But the infections that followed—surely science could do something to stop those. He focused on the bacteria, his personal demons, “these terrible enemies of man that murder him maliciously and treacherously without giving him a chance.” “I swore before God and myself,” he later wrote, “to counter this destructive madness.”  (p. 20).

Who knows but it’s pretty.  Someone in the same position as thousands of others (in this case a WW1 medic), caring more , and going to fix it (via sulfa drugs) is my moral aesthetic.  Of course there could be another surgeon in the same place with just as much care and potential who got blow up or gassed.  The Alchemy of Air prioritized poetry over provability, so I don’t entirely trust this, but I like it.

Claim: Cholera was a big problem for German soldiers.

This would be a weird thing to make up, but I’m a little confused.  There had been a cholera vaccine for over 20 years by that point.

Claim: Gas gangrene is bad.


Claim: Sir Almroth Wright created a typhoid vaccine that was deployed during WW1, saving may lives.  During WW1 he established a laboratory researching wound infections.

True.  He was also prescient enough to foresee the risk of drug-resistant bacteria.  Of course he also thought that bacteria were associated with but not the cause of disease, and that scurvy was caused by poorly preserved meat.  Being right is hard.

Claim: Doctors at the time thought that a dry wound was more resistant to infection; however dryness inhibited white blood cells and thus ultimately increased infections. They also thought wounds needed to be completely covered to prevent reinfection, but this created the ideal environment for anaerobic bacteria like Clostridium perfringens (which causes gas gangrene).

True. I was surprised to find ideal wound moistness still isn’t entirely settled, but the book’s description seems essentially in good faith.  Demon goes on to say that by the 1920s, doctors believed they were basically powerless and their job was to get the body’s own healing systems a pillow and some tea.  They took this so far that:

“A physician doing drug research was a physician taken away from patient care. There was an unsavory aspect to a physician’s developing a drug for money. There were ethical questions about testing drugs on patients. Developing new drug therapies smacked of a return to the discredited age of bleedings and purgings.”


To repeat: researching new treatments was considered distasteful at best and morally outrageous at worst.  And brain differentiation was once considered phrenology redux.  I just don’t think we’re very good at seeing where medicine is going (p40).

Claim: Section on Leeuwenhoek. 

True but missing time data.  Given that everything discussed so far happened in the range of 1890-1920, I would have have explicitly mentioned I was going 250 years into the past.  As it was, the only reason I noticed was that I recognized some of the names on the list of Leeuwenhoek’s contemporaries. The kindle edition may have made this worse.   But everything Hager actually says on Leeuwenhoek’s work in inventing the microscope seems accurate.

Claim: [crickets] (no page)

There’s no false statements, but I found the absence of discussion of the 1918 Spanish Flu epidemic puzzling.  Demon’s narrative is that seeing the horror of infected wounds in World War 1 drove Domangk to dedicate his life to preventing them.  Spanish Flu killed 5% of the entire world over the course of three years, and had a massive effect on troop movements and training in WW1.  From a military perspective it might have been more important.  We know now that the flu is really hard to vaccinate against, but at the time they didn’t even know it was a virus.  If you were a motivated medic looking for something to care about, Spanish Flu was a really obvious choice.  Demon mentions Spanish Flu in passing but not as an influence on Domangk, and that feels incomplete to me. Why gangrene in particular, when there were so many horrors happening at the time?

Claim: Streptococcus is the cause of everything bad.

True.  I knew it was possible to die from a scratch, but reading about everything strep causes really made me appreciate how few technological innovations are between us humans and mass die offs.  Strep causes childbed fever, St. Anthony’s Fire, meningitis, scarlet fever, pink eye, necrotizing fasciitis… Strep is the cockroach of human-infecting bacteria.  And for a while, all we had to do was take a pill and it was completely harmless.

Of course now we have MRSA (Methicillin-resistant Staphylococcus aureus) (whose natural habitat is the hospital, just like strep).  And multiply resistant gonorrhea.  And tuberculosis resistant to most known antibiotics.  The bad old days are on our heels, is what I’m saying.

One weird thing is I finished this book with the vague impression that sulfa drugs had saved a lot of lives but not actually knowing how many.  This article estimates that sulfa drugs led to a 2-3% drop in overall mortality, which translated to a 0.4-0.7 year increase in life expectancy.  That only covers up until 1943: presumably it had a bigger impact as distribution increased, or at least would have if penicillin had not taken over.

Overall Verdict

Pretty good, with some oversights.  Like Alchemy of Air the beginning is the best part, and if you find your attention flagging I’d just let it go.  I found the subject matter more innately interesting than Alchemy of Air but the writing a little less so.  Demon spends less time on the personal lives of the scientists, which was a selling point for my roommate but a disappointment for me.

This post supported by Patreon.

Epistemic Spot Check: A Brief History of the Human Race (Michael Cook)

I was worried my epistemic spot checking project was doomed before it began.  The well regarded Sapiens dismissed a link between cultural and genetic evolution, and Last Ape Standing made two explosively wrong errors in the first chapter.  Neither related to human evolution (one was about modern extreme poverty and the other about cetacean evolution), but I just couldn’t let them go.  I worried that every book was terrible if you actually fact checked it, or maybe just every book on the emergence of homonids?

And then I started A Brief History of the Human Race.  Two chapters in, I cannot find a flaw in it.  There are a few simplifications, and some broad statements he later walks back, but nothing I can point to and say “that is miseducating people.”  Meanwhile there are a lot of things I can point to and say “that is right”, even things I initially thought were wrong.  So I think A Brief History… might be the one.

Here’s a list of statements I checked and their results.  Normally I sort claims into true and untrue sections, but that proved unnecessary in this case because it is a beautiful rigorous snowflake of a book.

Claim: The holocene era has had unusual climactic stability, and is warmer than typical for the planet (page 6).
This checks out, but it’s a little weird that his graph doesn’t label the temperature axes, and it’s a graph of oxygen ratios from a single glacier rather than the wider variety of evidence available.
Cook further claims that this warmth and stability is what let farming, and thus history, start.   There’s some counter evidence to the claim of the holocene as the absolute start of agriculture. but it certainly seems legitimate to say that’s when it really took off.
Claim: there exists art “well over 20,000 years old” (page 9)
I was very surprised by this but if anything he’s underselling it; there’s claims of 50,000 year old art 
Claim: most human DNA is junk DNA that serves no purpose, and mutations have no effect (page 11):
He’s a little behind the times on junk DNA. Even when A Brief History… was published (2003) we knew it that while lots of DNA didn’t encode any proteins, much of it does seem to serve some purpose and mutations in it are significant.  At a minimum noncoding DNA definitely plays a role in the regulation of DNA transcription, the structural cohesion of the chromosome, and protecting coding DNA from degradation.  This is the closest thing to a falsehood I found in the first two chapters.  At the level of genetics this book is discussing I think it’s a forgivable simplification, although an addendum noting the real world was more complicated would have been appreciated.
Claim: humans are inbred (page 12)
Very true.
Page 13: AFAIK this explanation of multiorigin vs out of Africa human evolution hypotheses is correct, and he’s picked the correct winner.
Claim: evidence of human tool use half a million years ago (page 16)
me: there is no way there were tools that long ago.
internet: yes there is.
Claim: early hominid and chimp tool use are both culturally transmitted (page 17)
I’m pretty sure this is correct, and I appreciate the distinction he is making
Claim: it was hard for nomadic hominids to combine their tools in novel ways, because they had so little room. There are exceptions, but they might have been better off sticking with lower technology (page 20).
I am extremely happy to see this guy acknowledge that hunter gatherers had a pretty kick ass life and farming sucked.
Claim: iron is more abundant than copper because of something about supernovas (page 29).
Claim: cattle domesticated multiple times (page 30).
Claim: humans adapt to domesticated animals culturally but not genetically (page 32).
This ignores the co-domestication of humans and wolves but is basically true for everything else.
Claim: humans did not genetically adapt to farming (page 33).
me: ehhhhh….lactose tolerance was a pretty big deal.
p34: exceptions: lactose tolerance and disease resistance.
me: I accept your apology.
Claim: Maori had and then lost the technique for pottery (p36).
Not only is A Brief History… not saying wrong things, it is throwing out tangential facts at about the right rate.  I was never going to look up which molecules supernovas produced.  But now I have a general idea of the concept of “primary elements” and how this affected human history, which makes me happy.
Verdict: A Brief History of the Human Race has passed the epistemic spot check with flying colors, and is enjoyable to read.
This post supported by Patreon.