Controls and Confounds

I’ve talked a lot about controlling for confounding variables (also known as backing them out of your analysis), by which I mean, I’ve insulted several researchers for not doing it. And sometimes it is as obvious as I make it sound (controlling for smoking when studying coronary events). But sometimes it’s not. Eztra Klein has a great post about what it means to control for something.  If black and white people receive equal sentences for using crack and powder cocaine the system is, in one sense, not-racist.  But the fact that crack cocaine gets penalties so much higher than powder cocaine, and crack is more associated with black people, means that black people end up with much more jail time.  If the sentencing discrepancy is based on good reasons (e.g. yes, murder should get more jail time than jay walking), it’s not racist.  If the sentencing discrepancy exists entirely because crack is associated with black people, then it’s extremely racist, and the people who receive higher sentences for crack use are victims of racism against black people regardless of their race.  Klein takes on racism in the context of crime.  It’s true that you can almost close the criminal justice gap by controlling for things like income and type of offense, but those things are not necessarily independent of race.  It’s an excellent piece and you should read it both for it’s excellent explanation of how statistical modeling works, and its commentary on race in America.

This comes up with obesity too.  Someone went through and calculated life expectancy for each BMI* category (*moment of side eye for categorical analysis*), and found that while obesity (BMI >= 30) and being underweight (BMI < 18.5) were associated with excess deaths relative to normal weight, being overweight ( 25 <= BMI < 30), was not.  First, I find it extremely telling that their description was “not associated with excess deaths”, when the more accurate description would be “associated with noticeably fewer deaths.”  BMI-overweight people actually lived longer than BMI-normal people.  Allow me to summarize the conversation that followed.

Fat advocacy groups: see, fat isn’t unhealthy.  In fact, it’s the healthiest.
Pro-weight-loss groups: Bullshit.  The study doesn’t account for things that make you thin and then kill you, like cancer, or anorexia.
FAs: Okay, so when a confound causes death and skinniness, it’s legitimate, but if it causes death and fatness** it’s an excuse to let ourselves go?
Team skinny: What’s the alternative, let you be fat? Let you feel good about being fat? I can’t talk about this with you until you’re ready to take it seriously.

What’s the lesson here?  First, it’s to define your question very well.  Do you care if literally changing someone’s skin color improves their life, or if there is an overall system in which one color consistently comes up as the loser?  Is weight a good predictor of health outcomes, or is it a determinant of them?  Designing experiments to tease out these variables is incredibly hard.  Almost everything we know that affects weight also affects health.  The exception is liposuction, and my understanding is it is not associated with any health gains, but then, it’s barely associated with weight loss.  The situation with race, crime, and justice is even worse.

The second is that BMI is not only bullshit because it tosses a bunch of data away, but because the category boundaries are, at best, based on absolutely no data at all.

*Normally we would side eye use of BMI, but massive population studies are the one time it’s kind of okay.

**E.g. Type 2 Diabetes, sedentary lifestyle, poor diet\

3 thoughts on “Controls and Confounds

  1. I have mixed feelings about this

    > What’s the lesson here? First, it’s to define your question very well. Do you care if literally changing someone’s skin color improves their life, or if there is an overall system in which one color consistently comes up as the loser? Is weight a good predictor of health outcomes, or is it a determinant of them?

    I think its super important with social issues to ask questions about decisions you can make.

    Questions like ‘there is an overall system in which one color consistently comes up as the loser’ seem valuable for motivating people to work on change, because being able to say ‘the system is racist’ is a powerful rhetorical technique. But I think that question is distinctly bad at focusing people on appropriate solutions.

    ‘The system is racist’ brings to mind solutions like ‘try to figure out which elected officials are racist and don’t elect them’. ‘drug laws are absurdly harsh and disproportionately impact black and hispanic people’ brings to mind totally different solutions like ‘lobby for milder drug laws’.

  2. With weight, I think the question is clearer, where basically the questions you care about are ‘do interventions targeting weight loss lead to health benefits in general’ and ‘does intervention X targeting weight loss lead to health benefits’. For example ‘does telling people to have more self control wrt food’ lead to better health outcomes. Its hard not to be skeptical.

Comments are closed.