How to Handle Bad Examples in Texts?

How harshly should you judge when a scientific work gives a bad example of its point?

For example, I am reading Interaction Ritual Chains (Randall Collins), which focuses on emotional energy during stereotyped interactions between people (this is the scientific sense of stereotyped, meaning rigid or strongly patterned, not racist).  IRC believes that interaction is important and irreplaceable, such that people will shun much more “efficient” solutions to their official goal in order to get interpersonal interactions.  Unfortunately it uses terrible examples to illustrate this.

For example, on page 63 he claims that online shopping will never replace brick and mortar stores, because going to the store amidst other people is an energizing ritual.  At the time the book was published (2004) this was obviously untrue to me, but I can see how it wasn’t in every segment of society.  So while this is a bad prediction, it’s not a factual error.

He also claims that television has had no impact on sports attendance because people want the in person experience so much (p57).  To the extent that is true, it’s because leagues has gone to a great deal of trouble to make it not true, by imposing blackout rules such that you can’t broadcast a local game unless it has sold out.  It’s only since 1973 that the NFL had an exception for sold out games- previous to that, the only way to see a game played in your market was to be there.

How much should I reduce my confidence in Interaction Ritual Chains, given that it made this error? Or two false predictions?  Is it even fair to score them as false? In person stores still exist, although malls are having a hard time of it.  Maybe blackout rules were preemptive strikes and attendance would have stayed high without them.  But this gives me qualms about learning examples I know less about- even if the author is being accurate, if I’m drawing incorrect conclusions.

Epistemic Spot Check: The Demon Under the Microscope (Thomas Hager)

Description

How much would it suck to be the guy who invented sulfa drugs? You dedicate your life to preventing a repeat of the horrors you saw in the war, succeed in that and so much more, and then 10 years later some idiot leaves a petri dish open and completely replaces you as the father of man’s triumph against bacteria.  Actually he left the lid off before you found your thing, but ignored the result until you hit it big because everyone knew you couldn’t fight disease with chemicals, until you proved you could.  It’s the ultimate silver medal.  The Demon Under the Microscope is the tale of that guy.

It’s by the same author (Thomas Hagen) as The Alchemy of Air.  It’s also set in the same corporation, and about field that was transforming from science to industry.  The writing style is similar.  I originally didn’t intend to fact check this book very hard because I already knew what to expect from the author (a little too invested in the subject but basically accurate), but the habit is too ingrained at this point and I couldn’t keep reading until I’d checked out the first few chapters.

 

Evaluation

Claim: “Domagk [the researcher] had the ability to see. He watched everything, noted slight variations, quietly filed it all away.”  (p. 18).

The wounds themselves he accepted as the results of war. But the infections that followed—surely science could do something to stop those. He focused on the bacteria, his personal demons, “these terrible enemies of man that murder him maliciously and treacherously without giving him a chance.” “I swore before God and myself,” he later wrote, “to counter this destructive madness.”  (p. 20).

Who knows but it’s pretty.  Someone in the same position as thousands of others (in this case a WW1 medic), caring more , and going to fix it (via sulfa drugs) is my moral aesthetic.  Of course there could be another surgeon in the same place with just as much care and potential who got blow up or gassed.  The Alchemy of Air prioritized poetry over provability, so I don’t entirely trust this, but I like it.

Claim: Cholera was a big problem for German soldiers.

This would be a weird thing to make up, but I’m a little confused.  There had been a cholera vaccine for over 20 years by that point.

Claim: Gas gangrene is bad.

True.

Claim: Sir Almroth Wright created a typhoid vaccine that was deployed during WW1, saving may lives.  During WW1 he established a laboratory researching wound infections.

True.  He was also prescient enough to foresee the risk of drug-resistant bacteria.  Of course he also thought that bacteria were associated with but not the cause of disease, and that scurvy was caused by poorly preserved meat.  Being right is hard.

Claim: Doctors at the time thought that a dry wound was more resistant to infection; however dryness inhibited white blood cells and thus ultimately increased infections. They also thought wounds needed to be completely covered to prevent reinfection, but this created the ideal environment for anaerobic bacteria like Clostridium perfringens (which causes gas gangrene).

True. I was surprised to find ideal wound moistness still isn’t entirely settled, but the book’s description seems essentially in good faith.  Demon goes on to say that by the 1920s, doctors believed they were basically powerless and their job was to get the body’s own healing systems a pillow and some tea.  They took this so far that:

“A physician doing drug research was a physician taken away from patient care. There was an unsavory aspect to a physician’s developing a drug for money. There were ethical questions about testing drugs on patients. Developing new drug therapies smacked of a return to the discredited age of bleedings and purgings.”

 

To repeat: researching new treatments was considered distasteful at best and morally outrageous at worst.  And brain differentiation was once considered phrenology redux.  I just don’t think we’re very good at seeing where medicine is going (p40).

Claim: Section on Leeuwenhoek. 

True but missing time data.  Given that everything discussed so far happened in the range of 1890-1920, I would have have explicitly mentioned I was going 250 years into the past.  As it was, the only reason I noticed was that I recognized some of the names on the list of Leeuwenhoek’s contemporaries. The kindle edition may have made this worse.   But everything Hager actually says on Leeuwenhoek’s work in inventing the microscope seems accurate.

Claim: [crickets] (no page)

There’s no false statements, but I found the absence of discussion of the 1918 Spanish Flu epidemic puzzling.  Demon’s narrative is that seeing the horror of infected wounds in World War 1 drove Domangk to dedicate his life to preventing them.  Spanish Flu killed 5% of the entire world over the course of three years, and had a massive effect on troop movements and training in WW1.  From a military perspective it might have been more important.  We know now that the flu is really hard to vaccinate against, but at the time they didn’t even know it was a virus.  If you were a motivated medic looking for something to care about, Spanish Flu was a really obvious choice.  Demon mentions Spanish Flu in passing but not as an influence on Domangk, and that feels incomplete to me. Why gangrene in particular, when there were so many horrors happening at the time?

Claim: Streptococcus is the cause of everything bad.

True.  I knew it was possible to die from a scratch, but reading about everything strep causes really made me appreciate how few technological innovations are between us humans and mass die offs.  Strep causes childbed fever, St. Anthony’s Fire, meningitis, scarlet fever, pink eye, necrotizing fasciitis… Strep is the cockroach of human-infecting bacteria.  And for a while, all we had to do was take a pill and it was completely harmless.

Of course now we have MRSA (Methicillin-resistant Staphylococcus aureus) (whose natural habitat is the hospital, just like strep).  And multiply resistant gonorrhea.  And tuberculosis resistant to most known antibiotics.  The bad old days are on our heels, is what I’m saying.

One weird thing is I finished this book with the vague impression that sulfa drugs had saved a lot of lives but not actually knowing how many.  This article estimates that sulfa drugs led to a 2-3% drop in overall mortality, which translated to a 0.4-0.7 year increase in life expectancy.  That only covers up until 1943: presumably it had a bigger impact as distribution increased, or at least would have if penicillin had not taken over.

Overall Verdict

Pretty good, with some oversights.  Like Alchemy of Air the beginning is the best part, and if you find your attention flagging I’d just let it go.  I found the subject matter more innately interesting than Alchemy of Air but the writing a little less so.  Demon spends less time on the personal lives of the scientists, which was a selling point for my roommate but a disappointment for me.

This post supported by Patreon.

Current Projects

bookpile
Me, three days in to unemployment

No sooner did I start doing epistemic spot checks than I started designing more complicated multi-book series that needed a lot of work before they could be published. And now I have time to do them.  Here’s what I’m working on:

  • One offs on books on the history of science.
  • Reviews of a lot of self help/self improvement/self knowledge/mental health/emotional health books to determine where the scientific bar should be set.  I originally conceived this as “what percent of citations should check out”, but percent correct citations does not seem terribly correlated with usefulness, leading to my exploration of what I should evaluate instead.
  • After I read Exercise For Better Movement I launched two splinter projects: evaluating the claims of EFBM , and a comparison of movement health books.
  • A collaboration with Mark Gomer to read Interaction Ritual Chains, share insights, and have me write them up in an easier to understand format, similar to my piece on Interpretive Labor.

Goals 2 and 3 are in need of material, so please comment if you have recommendations for self-help books that claim a strong scientific backing or have been especially useful to you.

The Good News Is I’ll Have More Time For Blogging Now

I’ve been laid off from my job at Wave.com.  Wrong side of the dividing line in a reorg, I’m in very good company.  I’m disappointed, and still consider myself lucky to have worked there- a for-profit fighting poverty that let me work from my couch- for as long as I did, but I’m also kind of relieved it’s over.  Wave stopped me from quitting programing in 2015 because I believed in the mission so much, and it looks like I was correct the first time.  I love programming as a tool to help other things, but it’s not my passion in and of itself.

My passion is the things I do for this blog- research, modeling, writing.  So obviously a lot of my newfound free time will go here.  But I’d also like to look for paid opportunities to use those skills.  If you are or know of someone who needs writing or research like I do for this blog (deep scientific investigation, synthesizing difficult sources into something easy to read, effectiveness analysis, media reviews, all of these together), please reach out to me via elizabeth at this domain.   Have a thing you really want me to blog about?  Now’s a good time to ask.  I’m also available for editing or tutoring to teach people these same skills, and honestly if there’s something I didn’t list that you think I’d be good at, reach out anyway.  I’ve spent 10 years focused on programming, now is a time to explore.

I don’t care about the effective altruism label but am hoping to focus on content I find meaningful, be that in in altruism, science, history, psychology, economics…. or video games.

PS. If the thing keeping you from donating to my Patreon was that I made programmer money, it would be convenient for me if you reevaluated that decision.

Epistemic Spot Check Update

I am really enjoying this project and plan on continuing.  I have started a subproject of reading self-help-ish books that claim scientific validity, to determine what the correct standard of science is for them.  Some books have clearly failed this (looking at you, Upward Spiral).  Some have had very accurate citations and strong evidence for the problem they are solving, but their prescriptions have not actually been tested (Exercise for Mood and Anxiety). Others have wrong or weak evidence, and yet the prescriptions have been very helpful to me or people I know.  It would a great loss to throw those out, but we can’t read every self help book just to see what works.

My solution is to create a new axis for epistemic spot checks: model quality.  A high quality model has the following characteristics:

  • As simple as possible, but no simpler.
  • Explained well in the book (you should be able to teach it afterwords)
  • Makes explicit predictions that are testable on a reasonable timescale.  Ideally the predictions are novel or counterintuitive.
  • Recognizes that the technique may not work and explicitly talks about when that might happen, how to recognize it, and what to do.
  • Usefulness

How do these strike people?  Are there other criteria you would want information on? Other thoughts?