Month: June 2017

Epistemic Spot Check: A Guide To Better Movement (Todd Hargrove)

Edit 7/20/17: See comments from the author about this review.  In particular, he believes I overstated his claims, sometimes by a lot.

 

This is part of an ongoing series assessing where the epistemic bar should be for self-help books.

Introduction

Thesis: increasing your physical capabilities is more often a matter of teaching your neurological system than it is anything to do with your body directly.  This includes things that really really look like they’re about physical constraints, like strength and flexibility.  You can treat injuries and pain and improve performance by working on the nervous system alone.  More surprising, treating these physical issues will have spillover effects, improving your mental and emotional health. A Guide To Better Movement provides both specific exercises for treating those issues and general principles that can be applied to any movement art or therapy.

The first chapter of this book failed spot checking pretty hard.  If I hadn’t had a very strong recommendation from a friend (“I didn’t take pain medication after two shoulder surgeries” strong), I would have tossed it aside.  But I’m glad I kept going, because it turned out to be quite valuable (this is what triggered that meta post on epistemic spot checking).  In accordance with the previous announcement on epistemic spot checking, I’m presenting the checks of chapter one (which failed, badly), and chapter six (which contains the best explanation of pain psychology I’ve ever seen), and a review of model quality.  I’m very eager for feedback on how this works for people.

Chapter 1: Intro (of the book)

Claim: “Although we might imagine we are lengthening muscle by stretching, it is more likely that increased range of motion is caused by changes in the nervous system’s tolerance to stretch, rather than actual length changes in muscles. ” (p. 5). 

Overstated, weak.  (PDF).  The paper’s claims to apply this up to 8 weeks, no further.  Additionally, the paper draws most (all?) of its data from two studies and it doesn’t give the sample size of either.

Claim:  “Research shows the forces required to deform mature connective tissue are probably impossible to create with hands, elbows or foam rollers.” (p. 5). 

Misleading. (Abstract).  Where by “research” the Hargrove means “mathematical model extrapolated from a single subject”.

Claim:  “in hockey players, strong adductors are far more protective against groin strain than flexible adductors, which offer no benefit” (p. 14).

Misleading. (Abstract) Sample size is small, and the study was of the relative strength of adductor to abductor, not absolute strength.

Claim: “Flexibility in the muscles of the posterior chain correlates with slower running and poor running economy.” (p. 14).

Accurate citation, weak study.  (Abstract) Sample size: 8.  Eight.  And it’s correlational.

[A number of interesting ideas whose citations are in books and thus inaccessible to me]

Claim:  “…most studies looking at measurable differences in posture between individuals find that such differences do not predict differences in chronic pain levels.”  (p. 31). 

Accurate citation.  (Abstract).  It’s a metastudy and I didn’t track down any of the 54 studies included, but the results are definitely quoted accurately.

 

Chapter 6: Pain

Claim: “Neuromatrix” approach to pain means the pattern of brain activity that create pain, and that pain is an output of brain activity, not an input (p93).

True, although the ability to correctly use definitions is not very impressive.

Claim: “If you think a particular stimulus will cause pain, then pain is more likely.  Cancer patients will feel more pain if they believe the pain heralds the return of cancer, rather than being a natural part of the healing process.” (p93).

Correctly cited, small sample size. (Source 1, source 2, TEDx Talk).

Claim: Psychological states associated with mood disorders (depression, anxiety, learned helplessness, etc) are associated with pain (p94).

True, (source), although it doesn’t look like the study is trying to establish causality.

Claim: Many pain-free people have the kinds of injuries doctors blame pain on (p95).

True, many sources, all with small sample sizes.  (source 1, source 2, source 3, source 4, source 5)

Claim: On taking some cure for pain, relief kicks in before the chemical has a chance to do any work (p98)

True.  His source for this was a little opaque but I’ve seen this fact validated many other places.

Claim: we know you can have pain without stimulus because you can have arm pain without an arm (p102).

True, phantom limb pain is well established.

Claim: some people feel a heart attack as arm pain because the nerves are very close to each other and the heart basically never hurts, so the brain “corrects” the signal to originating in the arm (p102).

First part: True.  Explanation: unsupported.  The explanation certainly makes sense, but he provides no citations and I can’t find any other source on it.

Claim: Inflammation lowers the firing threshold of nociceptors (aka sensitization) (p102).

True (source).

Claim: nociception is processed by the dorsal horn in the spine.  The dorsal horn can also become sensitized, firing with less stimulus than it otherwise would.  Constant activation is one of the things that increases sensitivity, which is one mechanism for chronic pain (p103).

True (source).

Claim: people with chronic pain often have poor “body maps”, meaning that their mental model of where they are in space is inaccurate and they have less resolution when assessing where a given sensation is coming from (p107).

Accurate citation (source).  This is a combination of literature review and reporting of novel results.  The novel results had a sample of five.

Claim: The hidden hand in the rubber hand illusion experiences a drop in temperature (p109).

Accurate citation, tiny sample size (source).  This paper, which is cited by the book’s citation, contains six experiments with sample sizes of fifteen or less.  I am torn between dismissing this because cool results with tiny sample sizes are usually bullshit, and accepting it because it is super cool.

Claim: “a hand that has been disowned through use of the rubber hand illusion will suffer more inflammation in response to a physical insult than a normal hand.” (p. 109).

Almost accurate citation (source).  The study was about histamine injection, not injury per se.   Insult technically covers both, but I would have preferred a more precise phrasing.  Also, sample size 34.

Claim: People with chronic back pain have trouble perceiving the outline of their back (p. 109). 

Accurate citation, sample size six (pdf).

Claim:  “Watching the movements in a mirror makes the movements less painful [for people with lower back pain].” (p. 111). Better Movement. Kindle Edition.

Accurate citation, small sample size (source).

Model Quality

Reminder: the model is that pain and exhaustion are a product of your brain processing a variety of information.  The prediction is that improving the quality of processing via the principles explained in the book can reduce pain and increase your physical capabilities.

Simplicity: Good.  This is not actually simple model, it requires a ton of explanation to a layman.  But most of its assumptions come from neurology as a whole; the leap from “more or less accepted facts about neurology” to this model is quite small.

Explanation Quality: Fantastic.  I’ve done some reading on pain psychology, much of which is consistent with Guide…, but Guide… has by far the best explanation I’ve read.

Explicit Predictions: Good, kept from greatness only by the fact that brains and bodies are both very complicated and there’s only so much even a very good model can do.

Useful Predictions: Okay. The testable prediction for the home-reader is that following the exercises in the back of the book, or going to a Feldenkrais class, will treat chronic pain, and increase flexibility and strength.  Since the book itself admits that a lot of things offer short term relief but don’t address the real problem, helping immediately doesn’t prove very much.

Acknowledging Limitations: low. (Note: author disputes this, and it’s entirely possible he did and I forgot).  GTBM doesn’t have the grandiose vision of some cure-all books, and repeatedly reminds you that your brain being involved doesn’t mean your brain is in control.  But there’s no sentence along the lines of “if this doesn’t work there’s a mechanical problem and you should see a doctor.”

Measurability: low.  This book expects you to put in a lot of time before seeing results, and does not make a specific prediction of the form they will come in.  Worse, I don’t think you can skip straight to the exercises.  If I hadn’t read the entire preceding book I wouldn’t have approached them in the correct spirit of attention and curiosity.

Hmmm, if I’d assigned a gestalt rating it would have been higher than what I now think is merited based on the subscores.  I deliberately wrote this mostly before trying the exercises, so I can’t give an effectiveness score.  If you do decide to try it, please let me know how it goes so I can further calibrate my reviews to actual effectiveness.

 

You might like this book if…

…you suffer from chronic pain or musculoskeletal issues, or find the mind-body connection fascinating.

This post supported by Patreon.

How to Handle Bad Examples in Texts?

How harshly should you judge when a scientific work gives a bad example of its point?

For example, I am reading Interaction Ritual Chains (Randall Collins), which focuses on emotional energy during stereotyped interactions between people (this is the scientific sense of stereotyped, meaning rigid or strongly patterned, not racist).  IRC believes that interaction is important and irreplaceable, such that people will shun much more “efficient” solutions to their official goal in order to get interpersonal interactions.  Unfortunately it uses terrible examples to illustrate this.

For example, on page 63 he claims that online shopping will never replace brick and mortar stores, because going to the store amidst other people is an energizing ritual.  At the time the book was published (2004) this was obviously untrue to me, but I can see how it wasn’t in every segment of society.  So while this is a bad prediction, it’s not a factual error.

He also claims that television has had no impact on sports attendance because people want the in person experience so much (p57).  To the extent that is true, it’s because leagues has gone to a great deal of trouble to make it not true, by imposing blackout rules such that you can’t broadcast a local game unless it has sold out.  It’s only since 1973 that the NFL had an exception for sold out games- previous to that, the only way to see a game played in your market was to be there.

How much should I reduce my confidence in Interaction Ritual Chains, given that it made this error? Or two false predictions?  Is it even fair to score them as false? In person stores still exist, although malls are having a hard time of it.  Maybe blackout rules were preemptive strikes and attendance would have stayed high without them.  But this gives me qualms about learning examples I know less about- even if the author is being accurate, if I’m drawing incorrect conclusions.

Epistemic Spot Check: The Demon Under the Microscope (Thomas Hager)

Description

How much would it suck to be the guy who invented sulfa drugs? You dedicate your life to preventing a repeat of the horrors you saw in the war, succeed in that and so much more, and then 10 years later some idiot leaves a petri dish open and completely replaces you as the father of man’s triumph against bacteria.  Actually he left the lid off before you found your thing, but ignored the result until you hit it big because everyone knew you couldn’t fight disease with chemicals, until you proved you could.  It’s the ultimate silver medal.  The Demon Under the Microscope is the tale of that guy.

It’s by the same author (Thomas Hager) as The Alchemy of Air.  It’s also set in the same corporation, and about field that was transforming from science to industry.  The writing style is similar.  I originally didn’t intend to fact check this book very hard because I already knew what to expect from the author (a little too invested in the subject but basically accurate), but the habit is too ingrained at this point and I couldn’t keep reading until I’d checked out the first few chapters.

Evaluation

Claim: “Domagk [the researcher] had the ability to see. He watched everything, noted slight variations, quietly filed it all away.”  (p. 18).

The wounds themselves he accepted as the results of war. But the infections that followed—surely science could do something to stop those. He focused on the bacteria, his personal demons, “these terrible enemies of man that murder him maliciously and treacherously without giving him a chance.” “I swore before God and myself,” he later wrote, “to counter this destructive madness.”  (p. 20).

Who knows but it’s pretty.  Someone in the same position as thousands of others (in this case a WW1 medic), caring more , and going to fix it (via sulfa drugs) is my moral aesthetic.  Of course there could be another surgeon in the same place with just as much care and potential who got blow up or gassed.  The Alchemy of Air prioritized poetry over provability, so I don’t entirely trust this, but I like it.

Claim: Cholera was a big problem for German soldiers.

This would be a weird thing to make up, but I’m a little confused.  There had been a cholera vaccine for over 20 years by that point.

Claim: Gas gangrene is bad.

True.

Claim: Sir Almroth Wright created a typhoid vaccine that was deployed during WW1, saving may lives.  During WW1 he established a laboratory researching wound infections.

True.  He was also prescient enough to foresee the risk of drug-resistant bacteria.  Of course he also thought that bacteria were associated with but not the cause of disease, and that scurvy was caused by poorly preserved meat.  Being right is hard.

Claim: Doctors at the time thought that a dry wound was more resistant to infection; however dryness inhibited white blood cells and thus ultimately increased infections. They also thought wounds needed to be completely covered to prevent reinfection, but this created the ideal environment for anaerobic bacteria like Clostridium perfringens (which causes gas gangrene).

True. I was surprised to find ideal wound moistness still isn’t entirely settled, but the book’s description seems essentially in good faith.  Demon goes on to say that by the 1920s, doctors believed they were basically powerless and their job was to get the body’s own healing systems a pillow and some tea.  They took this so far that:

“A physician doing drug research was a physician taken away from patient care. There was an unsavory aspect to a physician’s developing a drug for money. There were ethical questions about testing drugs on patients. Developing new drug therapies smacked of a return to the discredited age of bleedings and purgings.”

To repeat: researching new treatments was considered distasteful at best and morally outrageous at worst.  And brain differentiation was once considered phrenology redux.  I just don’t think we’re very good at seeing where medicine is going (p40).

Claim: Section on Leeuwenhoek. 

True but missing time data.  Given that everything discussed so far happened in the range of 1890-1920, I would have have explicitly mentioned I was going 250 years into the past.  As it was, the only reason I noticed was that I recognized some of the names on the list of Leeuwenhoek’s contemporaries. The kindle edition may have made this worse.   But everything Hager actually says on Leeuwenhoek’s work in inventing the microscope seems accurate.

Claim: [crickets] (no page)

There’s no false statements, but I found the absence of discussion of the 1918 Spanish Flu epidemic puzzling.  Demon’s narrative is that seeing the horror of infected wounds in World War 1 drove Domangk to dedicate his life to preventing them.  Spanish Flu killed 5% of the entire world over the course of three years, and had a massive effect on troop movements and training in WW1.  From a military perspective it might have been more important.  We know now that the flu is really hard to vaccinate against, but at the time they didn’t even know it was a virus.  If you were a motivated medic looking for something to care about, Spanish Flu was a really obvious choice.  Demon mentions Spanish Flu in passing but not as an influence on Domangk, and that feels incomplete to me. Why gangrene in particular, when there were so many horrors happening at the time?

Claim: Streptococcus is the cause of everything bad.

True.  I knew it was possible to die from a scratch, but reading about everything strep causes really made me appreciate how few technological innovations are between us humans and mass die offs.  Strep causes childbed fever, St. Anthony’s Fire, meningitis, scarlet fever, pink eye, necrotizing fasciitis… Strep is the cockroach of human-infecting bacteria.  And for a while, all we had to do was take a pill and it was completely harmless.

Of course now we have MRSA (Methicillin-resistant Staphylococcus aureus) (whose natural habitat is the hospital, just like strep).  And multiply resistant gonorrhea.  And tuberculosis resistant to most known antibiotics.  The bad old days are on our heels, is what I’m saying.

One weird thing is I finished this book with the vague impression that sulfa drugs had saved a lot of lives but not actually knowing how many.  This article estimates that sulfa drugs led to a 2-3% drop in overall mortality, which translated to a 0.4-0.7 year increase in life expectancy.  That only covers up until 1943: presumably it had a bigger impact as distribution increased, or at least would have if penicillin had not taken over.

Overall Verdict

Pretty good, with some oversights.  Like Alchemy of Air the beginning is the best part, and if you find your attention flagging I’d just let it go.  I found the subject matter more innately interesting than Alchemy of Air but the writing a little less so.  Demon spends less time on the personal lives of the scientists, which was a selling point for my roommate but a disappointment for me.

This post supported by Patreon.

Current Projects

bookpile
Me, three days in to unemployment

No sooner did I start doing epistemic spot checks than I started designing more complicated multi-book series that needed a lot of work before they could be published. And now I have time to do them.  Here’s what I’m working on:

  • One offs on books on the history of science.
  • Reviews of a lot of self help/self improvement/self knowledge/mental health/emotional health books to determine where the scientific bar should be set.  I originally conceived this as “what percent of citations should check out”, but percent correct citations does not seem terribly correlated with usefulness, leading to my exploration of what I should evaluate instead.
  • After I read Exercise For Better Movement I launched two splinter projects: evaluating the claims of EFBM , and a comparison of movement health books.
  • A collaboration with Mark Gomer to read Interaction Ritual Chains, share insights, and have me write them up in an easier to understand format, similar to my piece on Interpretive Labor.

Goals 2 and 3 are in need of material, so please comment if you have recommendations for self-help books that claim a strong scientific backing or have been especially useful to you.

The Good News Is I’ll Have More Time For Blogging Now

I’ve been laid off from my job at Wave.com.  Wrong side of the dividing line in a reorg, I’m in very good company.  I’m disappointed, and still consider myself lucky to have worked there- a for-profit fighting poverty that let me work from my couch- for as long as I did, but I’m also kind of relieved it’s over.  Wave stopped me from quitting programing in 2015 because I believed in the mission so much, and it looks like I was correct the first time.  I love programming as a tool to help other things, but it’s not my passion in and of itself.

My passion is the things I do for this blog- research, modeling, writing.  So obviously a lot of my newfound free time will go here.  But I’d also like to look for paid opportunities to use those skills.  If you are or know of someone who needs writing or research like I do for this blog (deep scientific investigation, synthesizing difficult sources into something easy to read, effectiveness analysis, media reviews, all of these together), please reach out to me via elizabeth at this domain.   Have a thing you really want me to blog about?  Now’s a good time to ask.  I’m also available for editing or tutoring to teach people these same skills, and honestly if there’s something I didn’t list that you think I’d be good at, reach out anyway.  I’ve spent 10 years focused on programming, now is a time to explore.

I don’t care about the effective altruism label but am hoping to focus on content I find meaningful, be that in in altruism, science, history, psychology, economics…. or video games.

PS. If the thing keeping you from donating to my Patreon was that I made programmer money, it would be convenient for me if you reevaluated that decision.

Epistemic Spot Check Update

I am really enjoying this project and plan on continuing.  I have started a subproject of reading self-help-ish books that claim scientific validity, to determine what the correct standard of science is for them.  Some books have clearly failed this (looking at you, Upward Spiral).  Some have had very accurate citations and strong evidence for the problem they are solving, but their prescriptions have not actually been tested (Exercise for Mood and Anxiety). Others have wrong or weak evidence, and yet the prescriptions have been very helpful to me or people I know.  It would a great loss to throw those out, but we can’t read every self help book just to see what works.

My solution is to create a new axis for epistemic spot checks: model quality.  A high quality model has the following characteristics:

  • As simple as possible, but no simpler.
  • Explained well in the book (you should be able to teach it afterwords)
  • Makes explicit predictions that are testable on a reasonable timescale.  Ideally the predictions are novel or counterintuitive.
  • Recognizes that the technique may not work and explicitly talks about when that might happen, how to recognize it, and what to do.
  • Usefulness

How do these strike people?  Are there other criteria you would want information on? Other thoughts?