Self Help Epistemic Spot Check Results

In a word: ¯\_(ツ)_/¯

I reviewed several self-help books with a wide range of scientific backing. For posterity:

Polyvagal Theory
The Tapping Solution/EFT
Full Catastrophe Living/Mindfulness
Exercise for Mood and Anxiety
A Guide to Better Movement
There were a few others I never published because I didn’t get very far into.

As a reminder, epistemic spot checks are checking a book’s early claims for truth/scientific validity/coherent modeling, to determine whether it’s worth continuing. After a few books I concluded that scientific backing didn’t seem that predictive of a book’s helpfulness, and started focusing on modeling. But that wasn’t predictive either.

I never officially decided to quit this project, but I can no longer get excited about checking out a new book, because nothing short of trying it seems to have any predictive ability of whether or not it is helpful. This leads me to believe that most of the effects are placebo effect, not in the sense of  “imagined” as people usually use the word, but in the sense that it’s your own brain doing most of the work, and people just have to try things until something clicks for them, starting with the cheapest. I find this answer deeply unsatisfying, but what are you gonna do?

 

Epistemic Spot Check: Polyvagal Theory/Safe and Sound Protocol/Stephen Porges

I read part of the book The Polyvagal Theory and went to a two day seminar by the author, Stephen Porges. I went because I thought there was a strong possibility EFT worked by affected the vagal nerve, and thought maybe polyvagal theory could explain how. I ended up pretty disappointed.

Once I was at the seminar I was very interested in a protocol Porges developed called Safe and Sound, which purports to cure a number of things including many symptoms of autism, plus misophonia (which I have), by playing songs with certain frequencies filtered. Porges showed very impressive videos of autistic children going from non-functional to neurotypical-passing. He bragged about a 50% improvement rate. He played a sound sample and even on hotel sound system speakers, it had a very definite affect on me, relaxing many muscles. So of course I ordered it.

In a failure of order of operations I didn’t look up the results until after I’d ordered it (I really wanted my misophonia fixed, plus the demo had been so impressive). The paper tries very hard to hide this, but what actually happened was not an average 50% improvement in some patient metric, but that 50% of patients showed any improvement. Given that autism is a high variance disease and children are often receiving multiple interventions, this basically means “didn’t make anything worse, probably”.

But I’d already ordered the thing, so I decided to try it. This was kind of an ordeal, btw. Safe and Sound is available only through “trained professionals”, even though the protocol consists in its entirety of listening to some songs on an MP3 player. And I checked, there’s nothing magic about the MP3 player or headphones they send you, you could do it with any reasonably good pair you had lying around. Based on this, I have to assume the 3-digit price tag and gatekeeping are entirely about prestige, because they’re certainly not about helping people or making money (I’m sure he could make more selling the CDs without the gatekeeping).

The protocol did have an effect, in that it consistently made me very sad. It didn’t have any effect on my misophonia, even though I tried it twice. The occupational therapist tried to insist it had worked because I was blunter and more confident in my last conversation with her, but no, sweety, that was because I was more sure your system was bullshit. Then she recommended I give them more money to do other protocols, which I inexplicably declined.

I am fighting the urge to get into the science of polyvagal theory, because it is really really interesting and has a lot of explanatory power. I put off writing this for five months because I wanted to do a more scientific review. But the empirical results are not just bad, they’re bad while proponents are claiming they are good. I can’t trust someone who does that.

For bonus points, when I asked some pointed questions during the seminar, Porges blew me off. So I’m not going to give polyvagal theory any more brain space, even though it would be so cool if it was true.

 

Epistemic Spot Check: The Tapping Solution (Nick Ortner)

This is part of a series called epistemic spot checks, in which I investigate claims a book makes to see if it’s worth paying attention to, without attempting to be comprehensive about it.

Introduction

This is a weird review to write.  I went into reading The Tapping Solution with two beliefs:

  1. The scientific claims would be far less supported than the author implies.  The best case scenario was “as terrible as your average therapy research.”
  2. The book’s prescriptions work for me anyway, in the sense that they make me calmer and happier and enable me to take better actions.

This book is about EFT, which stands for emotional freedom technique. I write that in a very small font in the hopes you won’t notice how stupid it sounds.  EFT is also known as tapping, because the primary action is tapping your fingers against your face.

I originally learned about EFT in a book that went full blown magic about it: you tap your fingers on your face, it changes energy currents in your body, and the universe magically gives you what you want.  There’s no point evaluating the science in books like that; they are what they are.   The Tapping Solution markets itself as the more studious cousin of that book.  It keeps the energy channels but backs off the magic gifts claim, offering the much more defensible explanation that tapping changes something in you that lets you create better outcomes.

The basic idea of EFT is you tap out a pattern on your body, mostly your face, while repeating a statement about something with a lot of negative emotional affect for you, especially ones that activate the sympathetic nervous system (fight/flight/freeze).  Repeat until you feel better.

[There’s a lot of different techniques claiming to be The Best EFT Script and, while I suspect there are individual variations in what works best for each person, I can’t possibly care about the intra-EFT wars.  Any script you use should just be a starting point for making your own anyway.]

Why would tapping improve your mood?  I have some guesses:

  • It makes anxiety et al. boring.  There are a lot of activities where people deliberately activate their SNS (sky diving, horror movies, drugs), so there must be something fun rewarding about being activated.    Plus, lots of the things that happen to you in response to anxiety are quite pleasant.  People cuddle you and bring you ice cream.  You put off doing the stressful thing.  I don’t think many people deliberately push themselves into hysterics for the attention, but I do think these benefits bias how people handle their stress.  Tapping does not offer those kinds of rewards; after two or three rounds of tapping, you are bored.  There are times I have gone and done the stressful thing because I would rather deal with it than have to do another round of tapping.  It’s nice to have my intolerance for boredom harnessed for good.
    • I suspect this is some of how cognitive behavioral therapy works as well.  Having taught myself both, EFT is less work and yet harder to develop an immunity too, although hybrid systems do better still.
  • A sense of control lowers stress.  Having A Thing You Can Do While Stressed that you think lowers your stress level is already lowering your stress level.  You can dismiss this as a self-fulfilling prophecy, but that’s only the point if you’re actually evaluating the concept of energy meridians.  If what you want is to calm down so you can respond to comments on your code review, it doesn’t matter if it’s a placebo.
  • Something something vagus nerve.  The vagus nerve is this weird nerve that skips the spinal cord and runs all over your body, including most major organs and a lot of your face.
    • Its tasks include:
      • Parasympathetic (relaxing) stimulation of all major organs except the adrenal glands.
      • Parasympathetic stimulation of muscles around the mouth and larynx.
      • Possibly reduces systemic inflammation
      • Sympathetic (fight/flight/freeze) stimulation of blood vessels.
      • A bunch of sensory stuff around the face.
    • Activity on your face is already known to affect your body via the vagus nerve.
      • Cold water on the face slows down your heart, and this is attributed mainly to the vagus nerve.
      •  Direct electrical stimulation of the nerve is touted as a cure for all kinds of stuff.  My sense is the science on that is… optimistic, but there is a reason it is being done to the vagus nerve and not something else.
    • There’s an alternate EFT script that involves tapping only on the hands.  I have fond this to be a calming distraction at best.  Hands are also pretty innervated, so this points to the effects being due to something specifically in the face, as opposed to sensitivity in general.
    • I yawn *a lot* while tapping.  Heart problems can cause yawning via the vagus nerve. I’m obviously not damaging my heart by tapping, I mention this just to show that the vagus nerve and yawning are related.
    • So I don’t know what’s going on, but I suspect the effect of tapping is mediated via the vagus nerve.
  • It’s a framework for breaking your problem into bite sized chunks, which is the ideal size for problems to be.  EFT practices vary in how much you work off a verbal script you’re given, vs introspect on your own issues and tap on what comes up.  I predict script-style work to be at best competitive with relaxation exercises, and only introspective EFT leads to actual improvements.
  • Who knows, maybe energetic meridians are a real thing, or at least a workable metaphor for a real thing.  Lots of things sound stupid until you know how they work.
    • In particular, if you mixed up the explanations for EFT and the much more legitimate EMDR (deliberate eye movements rewiring your brain), I’m not convinced anyone could tell which one was the Officially Sanctioned Therapy and which was the crackpot treatment.
    • Mark at meditationstuff.com argues that what gets sensed as energy flow is severe awareness of your own nervous system.  He provides no compelling evidence for this, but it is interesting.
    • Many Properly Credentialed Authorities believe things that are no weirder.

 

How I evaluated this book: usually when doing these checks I evaluate any statement I find interesting.  In this case, I’m sticking to the ones for which the author explicitly claims scientific backing.  For stuff that is essentially running on placebos and metaphors, I find a calm, confident, made up explanation is better than a hedged, hesitant, literally true one, so I’m not going to investigate the obviously exaggerated claims.  But if you’re going to claim scientific validity, I am going to check.

Claim: “The amygdala is the source of emotions and long term memories, and it’s where negative experiences are encoded (p4)”.

True.  Simplified, but obviously trying to explain how the amygdala was relevant to a particular concept, not give a comprehensive overview of our friend the amygdala.  The amygdala is in fact so good at emotional memory that it can be invoked by visual cues even in people blinded by brain damage.  This confused me at first, so let me note that the amygdala is not involved in fight/flight/freeze, but the longer, cortisol-driven chronic kind of stress.

Claim: Stimulating acupoints calms down the amygdala, and this is observable in fMRI and PET machines (p5).

Misleading, either bad faith or credulous.  Both studies cited were done with acupuncture, not acupressure or tapping.  I consider that relevant evidence for EFT, but dislike that he tried to make it even stronger evidence by hiding that both studies involved needles.  The effectiveness of acupuncture appears to have large if weak support; I very quickly pulled up many more studies demonstrating the exact same thing, all of which were tiny (the largest was 18), and used fMRIs, which are suspect.

In general, studies of acupuncture have shown that it kind of works, but Official Legitimate Chinese Medicine Points don’t do any better than a random spot, so this adds more legitimacy to randomly stabbing yourself than it does to meridian points.

Claim: Other studies show that pressure works just as well for stabbing, maybe even better for anxiety (p5).

Seems legit.  I didn’t find any citation for this but I’m willing to spot him that touching works better than stabbing for anxiety.

Claim: A study demonstrated that EFT reduces cortisol levels in the saliva (p5).

True, evidence weak but better than I guessed.  The study cited is real, and with some effort I even found a full PDF.  EFT did better than both a support group and no treatment on both a symptoms assessment and cortisol levels (24% decrease vs 14%).  The differences in symptoms between EFT and the other groups are small, and some were not statistically significant.  OTOH, every one of them goes in the same direction.  I find this pretty compelling, assuming they published every trait they recorded.  As usual, small study, vulnerable to p-hacking, etc.

Claim: This John Hopkins approved doctor agrees with us (p7).

Misleading, possibly very.  The named person (David Friedman) does exist, but he’s a doctor of psychology, not psychiatry.  The level that JHU approves of him is unclear.  On his CV (PDF) he lists himself as “research associate”, “instructor”, and “faculty.”  None of these words are “professor”, which makes me think he was an adjunct and certainly didn’t have tenure.

Claim: Competing systems telling you to never think about the negative are idiotic.  True things are true (p8).  In particular The Secret is bullshit.

Seems legit.  “Make bad things approachable”  just seems like a better tactic than wishing really hard. I also enjoy watching different alt modalities fight with each other.

Claim: Meridians have been scientifically validated, they’re called Bonghan channels (p10).

False.  The official name of Bonghan channels is the primo-vascular system, and there’s minimal evidence it exists.  Given that it’s pretty hard to prove that there’s a link between them and meridians in any scientific sense.  But it’s established fact within the meridian community, so it’s at least well sourced bullshit.

 

A few more notes on The Tapping Solution.

As expected, Tapping Solution has failed the RCT test.  What about the model test?

Well, it’s a fairly vague model, and energy meridians can be used to power anything.  On the other hand it avoids my biggest complaint about heal-yourself-with-the-placebo-effect books, and also religion, certain parts of medicine, and psychology, which is that the solution to failure is often do the same thing harder.  Tapping by and large avoids that trap.  For actual physical problems you’re encouraged to see a doctor first, then tap, and if that doesn’t work see a doctor again.  If a particular tap isn’t working you’re given alternate prompts to try.  Additionally, tapping claims that often it will work so well you’ll forget you will ever upset about something, and the solution is not to hand over money to the nice man to keep the good vibes flowing, it’s to keep track of how upset you are at the beginning of the session.  That level of empiricism shouldn’t make a book stand out, but it does.  Tapping Solution, although not every book on EFT, is also pretty clear that you’re not imposing your will on the universe, you’re calming down so you can take better actions.

I don’t want to write out instructions for tapping because I believe the process of reading a book adds a lot of value over a quick run through (the same way doing yoga is better for you than waving a magic wand and becoming more flexible).  But to help you decide if even starting the book is worth your time, here are some genres of problems I think tapping is most appropriate for:

  • Somaticizations, especially back pain.
  • Emotions you find too overwhelming to deal with, especially anxiety.
  • Legit life problems that are just too big to deal with all at once and need to be broken into bite size pieces.

Model

Simplicity: very low.  “Magical energy currents” sounds simple in that you can explain it quickly, but it takes a very long time to explain what things it can’t do and why.

Explanation quality: poor.  Merdians can power anything.

Explicit predictions: okay. You have to make your own explicit predictions, but the book very much encourages you to do so.

Acknowledging limitations: mixed.

Relative to other heal-yourself-with-the-placebo-effect systems, The Tapping Solution is modest in its claims about what your mind can do.  It goes out of its way to establish that the mind-body connection is in fact a connection, it doesn’t mean your body is a hallucination you can will into whatever form you want.

GemasAugust2015.png
Lesbian space rocks whose bodies are solid holograms  are not representative study subjects.

And then on the next page there’s a story of how a woman cured her lung cancer with EFT.  So it’s not amazing on this axis.

Measurability: extremely good.  This is where EFT really shines.  They claim it’s such a good technique you will forget you ever had a problem, and encourage you to keep track so you won’t forget.

 

Empirics

I’m deliberately not giving a lot of details on how to do it yourself, because I think there might be value to going through the book beyond the technique.

I taught this technique to five people, one of whom had a good response to it. Counting myself, that’s 1/3 successes, which is not great. But it’s cheap enough and has high enough potential I still recommend trying it.

 

Epistemic Spot Check: Full Catastrophe Living (Jon Kabat-Zinn)

Full Catastrophe Living is a little weird, because between the first edition and the second a lot of science came out testing the thesis.  For this blog post, I’m reviewing the new, scienced-up edition of FCL.  However I have ordered the older edition of the book (thanks, Patreon supporters and half.com) and have dreams of reviewing that separately, with an eye towards identifying what could have predicted the experimental outcome.  E.g. if the experimental outcome is positive, was there something special about the model that we could recognize in other self-help books before rigorous science comes in?

I originally planned on fact checking two chapters, the scientific introduction and one of the explanatory chapters.  Doing the intro was exhausting and demonstrated a consistent pattern of “basically correct, from a small sample size, finding exaggerated”, so I skipped the second chapter of fact checking. I also skipped the latter two thirds of the book.

Overview

You’ve probably heard about mindfulness, but just in case: mindfulness is a meditation practice that involves being present and not holding on to thoughts, originally created within Buddhism.  Mindfulness Based Stress Reduction is a specific class created by the author of this book, Jon Kabat-Zinn.  The class has since spread across the country; he cites 720 programs in the introduction.   Full Catastrophe Living contains both a playbook for teaching the class to yourself, the science of why it works (I’m guessing this is new?), a section on stress, and followup information on how to integrate meditation into your life.

Introduction

Claim: Humans are happier when they focus on what they are doing than when they let their mind wander, which is 50% of the time.

Accurately cited, large effect size, possible confounding effects. (PDF).  The slope of the regression between mind wandering and mind not-wandering was 8.79 out of a 100 point scale, and the difference between unpleasant mind wandering and any mind not-wandering task was ~30 points.  Pleasant mind wandering was exactly as pleasant as focusing on the task at hand.  Focusing accounting for 17.7% of the between-person variation in happiness, compared to 3.2% from choice of task.

Some caveats:

  • People’s minds are more likely to wander when they’re doing something unpleasant, and when they are having trouble coping with that unpleasantness.   The study could be identifying a symptom rather than a cause.
  • The study population was extremely unrepresentative, consisting of people who chose to download an iPhone app.

Claim: Loss of telomeres is associated with stress and aging; meditation lengthens telomeres by reducing stress (location 404).

Research slightly more theoretical than is represented, but theoretical case is strong. (Source). First, let’s talk about telomeres.  Telomeres are caps on the ends of all of your chromosomes.  Because of the way DNA is copied, they will shorten a bit on every division.  There’s a special enzyme to re-lengthen them (telomerase), but leading thought right now is that stress inhibits it.  Short telomeres are associated with the diseases of aging (heart issues, type two diabetes) independent of chronological age.  This is hard to study because telomere length is a function of your entire life, not the last week, but is pretty established science at this point.

Mindfulness reduces stress, so it’s not implausible that it could lengthen telomeres and thus reduce aging.  The authors also present some evidence that negative mood reduces the activity of telomerase.  This is a very strong theoretical case, but is not quite proven.

Claim: Happiness research Dan Gilbert claims meditation is one of the keys to happiness, up there with sleep and exercise (location 461).

Confirmed that Gilbert is a happiness researcher and said the quote cited, although I can’t find where he personally researched this.

Claim: “Researchers at Massachusetts General Hospital and Harvard University have shown, using fMRI brain scanning technology, that eight weeks of MBSR training leads to thickening of a number of different regions of the brain associated with learning and memory, emotion regulation, the sense of self, and perspective taking. They also found that the amygdala, a region deep in the brain that is responsible for appraising and reacting to perceived threats, was thinner after MBSR, and that the degree of thinning was related to the degree of improvement on a perceived stress scale.” (location 502)

Accurate citation, but: small sample size (16/26), and for the first study the effect size was quite small (1%) for regions of a priori interest, and the second had quite wide error bands (source 1) (source 2).  However the book does refer to these findings as preliminary.

Claim: “They also show that functions vital to our well-being and quality of life, such as perspective taking, attention regulation, learning and memory, emotion regulation, and threat appraisal, can be positively influenced by training in MBSR.” (location 508).

Misleading.  These are really broad claims and no specific study is cited.  However, source 2 above has the following quote: “The results suggest that participation in MBSR is associated with changes in gray matter concentration in brain regions involved in learning and memory processes, emotion regulation, self-referential processing, and perspective taking.”  This is a very carefully phrased statement indicating that mindfulness is in the right ballpark for affecting these things, but is not the same as demonstrating actual change.

Claim: “Researchers at the University of Toronto, also using fMRI, found that people who had completed an MBSR program showed increases in neuronal activity in a brain network associated with embodied present-moment experience, and decreases in another brain network associated with the self as experienced across time. […]  This study also showed that MBSR could unlink these two forms of self-referencing, which usually function in tandem.” (location 508).

Accurate citation, small sample size (36) that they made particularly hard to find (source).  I can’t decipher the true size of the effect.

Claim: Relative to another health class, MSBR participants had smaller blisters in response to a lab procedure, indicating lower inflammation (location 529).

True, but only because the other class *raised* inflammation (source). Also leaves out the fact that both groups had the same cortisol levels and self-reported stress.  So this looks less like MBSR helped, and more like the control program was actively counterproductive.

For the record, this is where I got frustrated.

Claim: “people who were meditating while receiving ultraviolet light therapy for their psoriasis healed at four times the rate of those receiving the light treatment by itself without meditating.” (location 534)

Accurate citation (of his own work), small sample size (pdf).

Claim: “we found that the electrical activity in certain areas of the brain known to be involved in the expression of emotions (within the prefrontal cerebral cortex) shifted in the MBSR participants in a direction (right-sided to left-sided) that suggested that the meditators were handling emotions such as anxiety and frustration more effectively. […]

This study also found that when the people in the study in both groups were given a flu vaccine at the end of the eight weeks of training, the MBSR group mounted a significantly stronger antibody response in their immune system”

Accurate citation (of his own work), slightly misleading, small sample size.  Once again, he’s strongly implying a behavioral effect when the only evidence is that MSBR touches an area of the brain. On the other hand, the original paper gets into why they make that assumption, so either it’s correct or we just learned something cool about the brain.

Claim: MSBR reduced loneliness and a particular inflammatory protein among the elderly (location 551).

Not statistically significant. (source)  More specifically; the loneliness finding was significant but uninteresting, since the treatment was “8 weeks with a regular social activity” and the control was “not.”  The inflammation finding had p = .075.  There’s nothing magic about p < .05 and I don’t want to worship it, but it’s not a strong result.

I also researched MBSR in general, and found it to have a surprisingly large effect on depression and anxiety.

The Model

To the extent Full Catastrophe Living has a model, it’s been integrated so fully into the cultural zeitgeist that I have a hard time articulating it. It could be summarized as “do these practices and some amount of good things from this list will happen to you.” Which kills my hypothesis that having a good model is necessary to getting good results.

 

You Might Like This Book If…

I don’t know. I found it a slog and only read the first third, but the empirical evidence is very much on mindfulness’s side and I don’t know what better thing to suggest.

 

 

 

Thanks to the internet for making it possible for me to do these kinds of investigations.

Thanks to Patreon supporters for giving me money.

 

 

Epistemic Spot Check: Exercise for Mood and Anxiety (Michael W. Otto, Jasper A.J. Smits)

Introduction

Everyone knows exercise (along with diet and sleep) makes a big difference in depression and anxiety.  Depressed and anxious people are almost by definition bad at transforming information about how to improve their lives into actions with large up front costs, so this data is not as useful as it might be.  Exercise for Mood and Anxiety (Michael W. Otto, Jasper A.J. Smits) aims to close that gap by making the conventional wisdom actionable.  It does that through the following steps:

  1. Present evidence that exercise is very helpful and why, to create motivation.
  2. Walk you through setting up an environment where exercise requires relatively little will power to start.
  3. Scripts and advice to make exercise as unmiserable as possible while you are doing it.
  4. Scripts and advice to milk as much mood benefit as possible from a given amount of exercise.
  5. An idiotic chapter on weight and food.

 

Parts 3 and 4 use a lot of techniques from cognitive behavioral therapy and mindfulness, and I suspect there’s a second order benefit of learning to apply these techniques to a relatively easy thing, so you can apply them to the rest of your life later.

Epistemic Spot Checking

Claim: “a study of 55,000 adults in the United States and Canada found that people who exercised had fewer symptoms of anxiety and depression.” (Kindle Locations 103-104). 

Correctly cited, paper has no proof of causation.  (abstract) (PDF) The study does in fact say this, but it also says “Despite the fact that none of these surveys [of which this paper is a metaanalysis] was [sic] originally designed to explore this association… “.  I’m not saying you can never repurpose data, but with something like this where the real question is causality, it seems suspicious.  The authors do consider the idea that causation runs from mental health (=energy, hopefulness, executive function) -> exercise and dismiss if, for reasons I find inadequate.

Claim: “Other studies add to this list of mood benefits by indicating that exercise is also linked to less anger and cynical distrust, as well as to stronger feelings of social integration.” (Kindle Locations 104-106). 

Correctly cited, paper has no proof of causation. (Abstract).

Claim: And these benefits don’t just include reducing symptoms of distress in people who have not been formally diagnosed with depression or anxiety. The benefits of exercise also include lower rates of psychiatric disorders; there is less major depression, as well as fewer anxiety disorders in those who exercise regularly. (Kindle Locations 107-109). 

Correctly cited, paper has no proof of causation.

The dismissal of causality goes on for another three citations but I’m just going to skip to the intervention studies.  Otto gives these population studies more credence than I would but does note that the intervention studies are more informative.

Claim:  study summarized 70 studies on this topic and showed that adults who experience sad or depressed moods, but not at levels that meet criteria for a psychiatric disorder, reliably report meaningful improvements in their mood as they start exercising. (Kindle Locations 116-117).

Correctly cited, study accuracy undetermined.  (Full paper). My fear (based on spot checking a similar book you’ll see in the rejects post) is that each of these studies consists of 15 people.  All the metaanalysis in the world won’t save you if you do 100 small studies and only publish the 50 that say what you want.  The studies included go all the way back to 1969: I can’t decide if that makes them more informative or less.

Claim:  The latest estimates are that about 17% of adults experience a major depressive episode in their lifetimes and that about half who have it experience recurrent episodes over time. (Kindle Locations 124-126). 

True. (Full paper).  The same study is cited for both facts, but I can only find the 50% statistic in the paper.  The data is kind of old (started in 1981), but of course you can’t get 30-year data except by starting 30 years ago.  This paper says the lifetime prevalence of mood disorders (depression, bipolar 1 and 2, and their baby siblings) is 20%; this study puts prevalence in the US at 16.9%.

Claim: As is the case with major depressive disorder, anxiety disorders are common, affecting more than 1 in 4 (28.8%) adults in their lifetimes” (Kindle Locations 136-137).

True. (Full paper).  He cites the same paper I did for the 20% mood disorder statistic.

Claim: [Anxiety disorders] tend to be especially long-lasting when people do not receive treatment. (Kindle Locations 137-138).

True, although not particularly specific.  (Full paper)

Claim: Exercise in itself is a stressor—it requires effort, and it forces the body to adapt to the demands placed on it.  (Kindle Locations 141-142). 

True.  (Full paper).

Claim:  A study examined firefighters reaction to stress, and then gave half a 16 week exercise course.  The study group showed improvements in stress responses. (Kindle location 148)

True.  (Abstract) (PDF).  I really like this study.  The group presumably had a high baseline fitness level, so this isn’t the difference between couch potato and a walk.  And they have before and after metrics.  The study is marred only by the small sample size (53).

Claim: “stress plays a key role in both the development and the continuation of depression and anxiety disorders.” (Kindle Locations 152-153). 

Accurate citation, very complicated topic. (Abstract).

Okay, it is becoming clear I don’t have the time to check every one of these citations and you don’t have time to read it.  From here on out please assume a baseline of very dense citations, all of which accurately report the study results, if with a little more confidence than the study design merits, and I’m only going to call out things that deserve special attention on account of controversy or importance.

Claim: exercise increases serotonin just like the primary class of anti-depressants, selective serotonin update inhibitors.

True but less relevant than implied.  They’re relying on a model of how SSRIs treat depression that is fairly outdated.  SSRIs definitely increase serotonin, it’s just that there’s no evidence that’s their mechanism of action against depression except that they do it and they treat depression.  “Depression is caused by a serotonin deficiency” is a lie simplification told to patients and their families to allay fear and shame around psychiatric treatment.  This doesn’t undercut their point that exercise is good for you, but does indicate this is not a great book to learn brain chemistry from.

Claim:  Both aerobic (prolonged moderate exercise such as running, cycling, or rowing over time) and anaerobic (like weight lifting or short sprinting) exercise have been found to be effective for decreasing depression, (Kindle Locations 239-241).

True. (Study 1 PDF) (Study 2 abstract).

 

Empirical Results

The theory behind this book is very well supported; the prescriptions it makes flow naturally from the theory, but the authors present no direct evidence that they work.  I’m torn about this.  I don’t want to engage in RCT worship; having a systemic understanding of a problem is even better than evidence a particular solution worked better or worse than another solution in a different population.  On the other hand, humans are very complicated and it’s easy to identify the problem but guess the wrong solution.

I couldn’t test any of this on myself because I already enjoy exercise for a lot of reasons, so I scrounged up an unscientific sample from my wider social network to try it.

14 people filled out the pre-book survey.  3 people filled out the post-attempt survey.  None of them exercised more.

Summary

The theory sections of this book are my high water mark for scientific rigor in a self-help-psych book.  I’m currently reading a lot of those with the goal of finding out how much rigor is reasonable to expect, so that’s high praise.

The book walks the very fine line between reassuring and condescending, which is pretty unavoidable with CBT and mindfulness.

I did not like the last chapter and recommend skipping it.  It feels like they tried to stuff all the usual diet-and-exercise stuff in at the end.  Some of my problem is I think their recommendations are wrong, and some is that I believe that even if they were correct, throwing them in at the last minute undercuts the message of the book.

The first part of this is that, in America, at least in certain subcultures, any mention of weight makes the whole thing About Weight.  Too many people use health or mood as a socially acceptable way to say “you’re not hot enough”, so any mention of weight in the context of diet or exercise automatically makes weight the real topic of the conversation.  If the improvements in mood are enough of a reason to exercise, let them be enough, and the weight loss can be a pleasant surprise or not happen, and both are okay because you got what you came for.

The authors compound this problem by using Body Mass Index as a guide for goal weight.  BMI is completely unsuited for use in individuals, even more so for people who just started gaining muscle mass.  If you must talk about fat in the context of health use body fat percentage or certain circumference ratios (e.g. wrist:stomach).

The second problem is the speed with which EFMaA tries to address nutrition.  The book (correctly) treats exercise as a thing that is challenging to start despite all its benefits, and spends 10 chapters explaining why it’s worth trying and providing scripts to make it workable for you, for the sole benefit of mood, ignoring everything else you might get out of exercise.  I don’t know why the authors thought that that required an entire book but the even more complicated of nutrition for every possible benefit of nutrition could be squeezed into half a chapter.  I would be have been very excited for another book by the same authors about how to implement healthy eating, but the half assed treatment here makes me pause.

They also present a particular diet as the settled science, when there is no such thing in nutrition.  “Eat produce and fish” is fairly uncontroversial, but they recommend a lot more refined grains than many other people.  I don’t know who is correct, but it was disappointing to see a book that had been so rigorous up to that point blithely paint over controversy.

[I have emailed Michael Otto about the handling of nutrition and have yet to hear back].

Speaking of which Exercise for Mood and Anxiety mentions that both aerobic (cardio) and anaeorbic (weights) are good for mood, but every single example is cardio, with an occasional cardio + core strength.

Mixed in through the book are tales of how Olympic athletes motivate themselves.  This feels spectacularly irrelevant to me.  I don’t want to win a gold medal, I want to climb V2s and be happy.

You might find this book valuable if:

  • You want some ideas (although not conclusive proof) around how exercise helps mood.
  • You want to want to exercise, and want scripts and tools to transform that into “want to exercise right now.”
  • You find exercise unpleasant and want to get the best trade of unpleasantness-for-benefits possible.
  • You would like to treat a mood issue with exercise (whether it reaches the level of official disorder or not).
  • You want to change how you think about exercise (for improving your mood or something else).
  • You are interested in CBT or mindfulness and want to practice with the large print version before tackling them directly.
  • You think you are different than my test audience.

You probably won’t find this book valuable if:

  • You already have an exercise program you are happy with.
  • You have body image or eating disorder issues (last chapter only, and a single section of the 10th,  the rest of it is fine).
  • You want prescriptions for a particular exercise program, as opposed to general principles.
  • You want to learn the nitty gritty of how exercise affects mood.
  • You are similar to my test audience.

 

 

Post supported by Patreon.

Epistemic Spot Check Changes

Previously I checked books pretty much as I went along.  Doing otherwise felt like the check was playacting; if knowing wasn’t going to affect my behavior towards the book it was rigor theater, not genuinely caring about a book’s factual accuracy.  I’m backing off on that.  It prevents a book from gathering any momentum and it’s too easy to turn into cheap shots. Instead, for the mental health books, my default is going to be read the introduction and at least one other chapter, marking what I want to verify as I go, and circle back at the end of the second chapter to fact check.  This will give the books a little time to breathe and for me to evaluate their model.  It’s also way easier for me.

The exception  will be books where I get a stuck feeling and need to look something up to continue, or where I get nerd sniped.

Epistemic Spot Check: A Guide To Better Movement (Todd Hargrove)

Edit 7/20/17: See comments from the author about this review.  In particular, he believes I overstated his claims, sometimes by a lot.

 

This is part of an ongoing series assessing where the epistemic bar should be for self-help books.

Introduction

Thesis: increasing your physical capabilities is more often a matter of teaching your neurological system than it is anything to do with your body directly.  This includes things that really really look like they’re about physical constraints, like strength and flexibility.  You can treat injuries and pain and improve performance by working on the nervous system alone.  More surprising, treating these physical issues will have spillover effects, improving your mental and emotional health. A Guide To Better Movement provides both specific exercises for treating those issues and general principles that can be applied to any movement art or therapy.

The first chapter of this book failed spot checking pretty hard.  If I hadn’t had a very strong recommendation from a friend (“I didn’t take pain medication after two shoulder surgeries” strong), I would have tossed it aside.  But I’m glad I kept going, because it turned out to be quite valuable (this is what triggered that meta post on epistemic spot checking).  In accordance with the previous announcement on epistemic spot checking, I’m presenting the checks of chapter one (which failed, badly), and chapter six (which contains the best explanation of pain psychology I’ve ever seen), and a review of model quality.  I’m very eager for feedback on how this works for people.

Chapter 1: Intro (of the book)

Claim: “Although we might imagine we are lengthening muscle by stretching, it is more likely that increased range of motion is caused by changes in the nervous system’s tolerance to stretch, rather than actual length changes in muscles. ” (p. 5). 

Overstated, weak.  (PDF).  The paper’s claims to apply this up to 8 weeks, no further.  Additionally, the paper draws most (all?) of its data from two studies and it doesn’t give the sample size of either.

Claim:  “Research shows the forces required to deform mature connective tissue are probably impossible to create with hands, elbows or foam rollers.” (p. 5). 

Misleading. (Abstract).  Where by “research” the Hargrove means “mathematical model extrapolated from a single subject”.

Claim:  “in hockey players, strong adductors are far more protective against groin strain than flexible adductors, which offer no benefit” (p. 14).

Misleading. (Abstract) Sample size is small, and the study was of the relative strength of adductor to abductor, not absolute strength.

Claim: “Flexibility in the muscles of the posterior chain correlates with slower running and poor running economy.” (p. 14).

Accurate citation, weak study.  (Abstract) Sample size: 8.  Eight.  And it’s correlational.

[A number of interesting ideas whose citations are in books and thus inaccessible to me]

Claim:  “…most studies looking at measurable differences in posture between individuals find that such differences do not predict differences in chronic pain levels.”  (p. 31). 

Accurate citation.  (Abstract).  It’s a metastudy and I didn’t track down any of the 54 studies included, but the results are definitely quoted accurately.

 

Chapter 6: Pain

Claim: “Neuromatrix” approach to pain means the pattern of brain activity that create pain, and that pain is an output of brain activity, not an input (p93).

True, although the ability to correctly use definitions is not very impressive.

Claim: “If you think a particular stimulus will cause pain, then pain is more likely.  Cancer patients will feel more pain if they believe the pain heralds the return of cancer, rather than being a natural part of the healing process.” (p93).

Correctly cited, small sample size. (Source 1, source 2, TEDx Talk).

ClaimPsychological states associated with mood disorders (depression, anxiety, learned helplessness, etc) are associated with pain (p94).

True, (source), although it doesn’t look like the study is trying to establish causality.

ClaimMany pain-free people have the kinds of injuries doctors blame pain on (p95).

True, many sources, all with small sample sizes.  (source 1, source 2, source 3, source 4, source 5)

Claim: On taking some cure for pain, relief kicks in before the chemical has a chance to do any work (p98)

True.  His source for this was a little opaque but I’ve seen this fact validated many other places.

Claim: we know you can have pain without stimulus because you can have arm pain without an arm (p102).

True, phantom limb pain is well established.

Claim: some people feel a heart attack as arm pain because the nerves are very close to each other and the heart basically never hurts, so the brain “corrects” the signal to originating in the arm (p102).

First part: True.  Explanation: unsupported.  The explanation certainly makes sense, but he provides no citations and I can’t find any other source on it.

Claim: Inflammation lowers the firing threshold of nociceptors (aka sensitization) (p102).

True (source).

Claim: nociception is processed by the dorsal horn in the spine.  The dorsal horn can also become sensitized, firing with less stimulus than it otherwise would.  Constant activation is one of the things that increases sensitivity, which is one mechanism for chronic pain (p103).

True (source).

Claim: people with chronic pain often have poor “body maps”, meaning that their mental model of where they are in space is inaccurate and they have less resolution when assessing where a given sensation is coming from (p107).

Accurate citation (source).  This is a combination of literature review and reporting of novel results.  The novel results had a sample of five.

Claim: The hidden hand in the rubber hand illusion experiences a drop in temperature (p109).

Accurate citation, tiny sample size (source).  This paper, which is cited by the book’s citation, contains six experiments with sample sizes of fifteen or less.  I am torn between dismissing this because cool results with tiny sample sizes are usually bullshit, and accepting it because it is super cool.

Claim: “a hand that has been disowned through use of the rubber hand illusion will suffer more inflammation in response to a physical insult than a normal hand.” (p. 109).

Almost accurate citation (source).  The study was about histamine injection, not injury per se.   Insult technically covers both, but I would have preferred a more precise phrasing.  Also, sample size 34.

Claim: People with chronic back pain have trouble perceiving the outline of their back (p. 109). 

Accurate citation, sample size six (pdf).

Claim:  “Watching the movements in a mirror makes the movements less painful [for people with lower back pain].” (p. 111). Better Movement. Kindle Edition.

Accurate citation, small sample size (source).

Model Quality

Reminder: the model is that pain and exhaustion are a product of your brain processing a variety of information.  The prediction is that improving the quality of processing via the principles explained in the book can reduce pain and increase your physical capabilities.

Simplicity: Good.  This is not actually simple model, it requires a ton of explanation to a layman.  But most of its assumptions come from neurology as a whole; the leap from “more or less accepted facts about neurology” to this model is quite small.

Explanation Quality: Fantastic.  I’ve done some reading on pain psychology, much of which is consistent with Guide…, but Guide… has by far the best explanation I’ve read.

Explicit Predictions: Good, kept from greatness only by the fact that brains and bodies are both very complicated and there’s only so much even a very good model can do.

Useful Predictions: Okay. The testable prediction for the home-reader is that following the exercises in the back of the book, or going to a Feldenkrais class, will treat chronic pain, and increase flexibility and strength.  Since the book itself admits that a lot of things offer short term relief but don’t address the real problem, helping immediately doesn’t prove very much.

Acknowledging Limitations: low. (Note: author disputes this, and it’s entirely possible he did and I forgot).  GTBM doesn’t have the grandiose vision of some cure-all books, and repeatedly reminds you that your brain being involved doesn’t mean your brain is in control.  But there’s no sentence along the lines of “if this doesn’t work there’s a mechanical problem and you should see a doctor.”

Measurability: low.  This book expects you to put in a lot of time before seeing results, and does not make a specific prediction of the form they will come in.  Worse, I don’t think you can skip straight to the exercises.  If I hadn’t read the entire preceding book I wouldn’t have approached them in the correct spirit of attention and curiosity.

Hmmm, if I’d assigned a gestalt rating it would have been higher than what I now think is merited based on the subscores.  I deliberately wrote this mostly before trying the exercises, so I can’t give an effectiveness score.  If you do decide to try it, please let me know how it goes so I can further calibrate my reviews to actual effectiveness.

 

You might like this book if…

…you suffer from chronic pain or musculoskeletal issues, or find the mind-body connection fascinating.

This post supported by Patreon.