Playing with Faunalytics’ Vegan Attrition Data

I’ve been looking for data on nutrition-related health problems in vegans, and it has been hard going. By far the best I’ve found has been this 2014 survey from Faunalytics (a data science org focused on animal protection), which finds 29% of ex-vegans + vegetarians had health issues plausibly traced to veganism, but that study is still missing several factors I care about. I made up a model to see how those factors might affect the answer, and found the answer was “moderately, but not enough to change anyone’s plans”.

This is mostly intended for me to reference later, I expect most people to skip it.

I want to give Faunalytics a lot of credit here. They used a professional survey provider to poll a large number of people (11,399, although most of them wasted as they had never been veg*n), and AFAICT the survey was well designed for their goals. They were extremely forthcoming in sharing their raw data and provided some assistance in interpretation (although ultimately not enough to make it usable for me 😞 ). 

But even though they are the best I’ve found, the data is still pretty limited (~1100 ex-veg*ns), and the questions weren’t optimized for my inquiry.  I dislike that they combined vegans and vegetarians into a single category (which I label veg*nism here), but I think that was a very understandable combination of sample size issues and the fact that very few vegans don’t pass through vegetarianism. I can’t pull data out of thin air, but I can at least use guesstimate to get a range of how those factors might affect the answer.  So I made a model that linearly applied a number of factors I think are important. This is really crude, and some of the factors shouldn’t be linear, but it was a place to start. 

This is pretty informal. You could easily argue I missed important factors, or assigned the ones I made up the wrong value. I encourage you to make that argument in comments, or play with the model yourself. 

We’ll start with Faunalytics’ (page 6) raw number, which showed 29% of ex-veg*ns reported coming down with a problem plausibly attributed to their diet.  

But I care about all veg*ns, not just those who quit. In this sample, 16% of people who had ever tried veg*nism were still at it.  You could assume that set have never had health problems, or they would eventually hit the same 30%, or somewhere in between. Faunalytics showed current veg*ns tend to have been on the diet longer than the average time-to-quit of ex-veg*ns, and have better nutrition and supplementation, so “the same amount” seems much too high. I set it to 0 to 20% (log-normal distribution – so biased towards the low end).

[This still focuses on the number of self-identified vegans, not vegan-months or total animal products consumed over time, but that’s an even bigger can of worms]

How do we know diet was to blame for health issues? One easy way is to look at the percentage of people who got better when they resumed eating meat, which Faunalytics reports to be 80% improved within 3 months: I assume all of those were caused by veg*n diet. What about people still impaired after 3 months of resuming consumption of animal products? Probably some will get better later, some will never improve but the problem was still caused by veg*n diet, some have a problem veg*nism didn’t cause but will make worse (e.g. they developed a nut allergy and it made veg*nism too hard) and some have a completely unrelated problem that they are misattributing to veg*n diet. I guessed the diet-related proportion was 5-50% (of the 20% not responding in three months), with a normal distribution (so biased towards the average of that range).

Health is the most sympathetic reason to quit veg*nism, so we should expect people to overreport it. Except that the percentage of people who described themselves as quitting over health reasons (26%) was smaller than the percent who reported at least one health issue easily caused by veg*nism (29%), although higher than percent of people who developed those issues and then recovered within 3 months of resuming meat consumption (23%). I estimated the percentage of reported-health issues that were fake as 0-50%, with log-normal distribution. 

Survey organizers tend to get the results they want, and Faunalytics is an animal protection org. They outsourced the actual survey administration, which is helpful but not a total fix. I guessed that they biased results 0-50% too low, with a log-normal distribution.

I am most interested in people practicing a veg*n diet naively (meaning: they did zero research and thought into nutrition), but that 29% includes all knowledge and effort levels (although in practice current veg*ns were more likely to be eating well). I added a factor for “if naive”, and guessed the size of the impact if *everyone* was naive was between 0 and 100% (meaning, no-effect to double effect – 100% isn’t the top of the scale here), with a log-normal distribution.

People often identify as veg*n while consuming animal products, which improves their nutrition.  The first study I found said 50% of vegetarians admitted to eating meat while identifying as vegetarian – but that included people who ate meat even once, which isn’t necessarily enough to matter nutritionally. On the other hand, this is the kind of thing people are likely to underreport. I added a multiplier for that and guessed it to be an additional 10%-50%, with a log-normal distribution.

Not all issues caused by veg*nsim get diagnosed or properly traced to diet. I added a multiplier for that, +10% to 40%, with a log-normal distribution.

If you combine all my wild-ass guesses, you get…. 37% (0%-56%) of veg*ns eventually developing health issues from their diet. That’s already higher than Faunalytics’ 29%, but remember my denominator includes current veg*ns and theirs doesn’t. The comparable number, looking only at people who quit and assuming everyone who develops health issues quit over them, is 44%,which is 65% higher than Faunanalytic’s estimate. 

I’m especially interested in vegans (but not vegetarians) who enter the diet naively. I made up numbers for these as well, and ultimately got 95% (17%-100%) of naive vegans developing health issues. I know too many healthy (so far) naive vegans for that to seem right, so I changed my assumptions and got it down to 56% (3%-88%), which seems plausible if you use a strict definition of naive.

What does this mean? Primarily that I feel better about my intuition that Faunalytics is underestimating the extent of the problem, but because I already considered 30% a big deal there’s no qualitative shift in how I think about it.  If you have different intuitions, you test out them out in guesstimate.