New Water Quality x Obesity Dataset Available

Tl;dr: I created a dataset of US counties’ water contamination and obesity levels. So far I have failed to find anything really interesting with it, but maybe you will. If you are interested you can download the dataset here. Be warned every spreadsheet program will choke on it; you definitely need to be use statistical programming.

Photocredit: DALL-E and a lot of coaxing 

Many of you have read Slime Mold Time Mold’s series on the hypothesis that environmental contaminants are driving weight gain. I haven’t done a deep dive on their work, but their lit review is certainly suggestive. 

SMTM did some original analysis by looking at obesity levels by state, but this is pretty hopeless. They’re using average altitude by state as a proxy for water purity for the entire state, and then correlating that with the state’s % resident obesity. Water contamination does seem negatively correlated with its altitude, and its altitude is correlated with an end-user’s altitude, and that end user’s altitude is correlated with their average state altitude… but I think that’s too many steps removed with too much noise at each step. So the aggregation by state is basically meaningless, except for showing us Colorado is weird.

So I dug up a better data set, which had contamination levels for almost every water system in the country, accessible by zip code, and another one that had obesity prevalence by county. I combined these into a single spreadsheet and did some very basic statistical analysis on them to look for correlations.

Some caveats before we start:

  • The dataset looks reasonable to me, but I haven’t examined it exhaustively and don’t know where the holes are. 
  • Slime Mold Time Mold’s top contender for environmental contagion is lithium. While technically present in the database, litium had five entries so I ignored it. I haven’t investigated but my guess is no one tests for lithium.
  • It’s rare, but some zip codes have multiple water suppliers, and the spreadsheet treats them as two separate entities that coincidentally have the same obesity prevalence.
  • I’ve made no attempt to back out basic confounding variables like income or age.
  • “% obese” is a much worse metric than average BMI, which is itself a much worse metric than % body fat. 
  • None of those metrics would catch if a contaminant makes some people very fat while making others thin ( SMTM thinks paradoxical effects are a big deal, so this is a major gap for testing their model).
  • Correlation still does not equal causation.

The correlations (for contaminants with >10k entries):

ContaminantCorrelation# Samples
Nitrate-0.03921430
Total haloacetic acids (HAAs)0.05514666
Chloroform0.04615065
Barium (total)0.04017929
Total trihalomethanes (TTHMs)0.11721184
Copper-0.00217113
Dibromochloromethane0.08013856
Nitrate & nitrite0.03511902
Bromodichloromethane0.07914238
Lead (total)-0.00613031
Dichloroacetic acid-0.00310159

Of these, the only one that looks interesting is trihalomethanes, a chemical group that includes chloroform. Here’s the graph:

Visually this looks like the floor is rising much faster than the ceiling, but in a conversation on twitter SMTM suggested that’s an artifact of the biviariate distribution, it disappears if you look at log normal. 

Very casual googling suggests that TTHMs are definitely bad for pregnancy in sufficient quantities, and are maybe in a complicated relationship with Type 2 diabetes, but no slam dunks.

This is about as far as I’ve had time to get. My conclusions alas are not very actionable, but maybe someone else can do something interesting with the data.

Thanks to Austin Chen for zipping the two data sets together, Daniel Filan for doing additional data processing and statistical analysis, and my Patreon patrons for supporting this research.

Home Antigen Tests Aren’t Useful For Covid Screening

Epistemic status: I strongly believe this is the right conclusion given the available data. The best available data is not that good, and if better data comes out I reserve the right to change my opinion.

EDIT (4/27): In a development I consider deeply frustrating but probably ultimately good, the same office is now getting much more useful information from antigen tests. They aren’t tracking with same rigor so I can’t comapre results, but they are now beating the bar of “literally ever noticing covid”.

In an attempt to avoid covid without being miserable, many of my friends are hosting group events but requiring attendees to take a home covid test beforehand. Based on data from a medium-sized office, I believe testing for covid with the tests people are using, to be security theater and provide no decrease in riskAntigen tests don’t work for covid screening. There is a more expensive home test available that provides some value, and rapid PCR may still be viable.

It’s important to distinguish between test types here: antigen tests look for viral proteins, and genetic amplification tests amplify viral RNA until it reaches detectable levels. The latter are much more sensitive. Most home tests are antigen tests, with the exception of Cue, which uses NAAT (a type of genetic amplification). An office in the bay area used aggressive testing with both Cue and antigen tests to control covid in the office and kept meticulous notes, which they were kind enough to share with me. Here are the aggregated numbers: 

  • The office requested daily Cue tests from workers. I don’t know how many people this ultimately included, probably low hundreds? I expect compliance was >95% but not perfect.
    • The results are from January when the dominant strain was Omicron classic, but no one got strain tested.
  • 39 people had at least one positive Cue test, all of which were either asymptomatic or ambiguously symptomatic (e.g. symptoms could be explained by allergies) at the time, and 27 of which had recent negative cue tests (often but not always the day before, sometimes the same day)
  • Of these, 10 definitely went on to develop symptoms, 7 definitely did not, and 18 were ambiguous (and a few were missing data).
  • 33 people with positives were retested with cue tests, of which 9 were positive. 
  • Of those 24 who tested positive and then negative, 4 tested positive on tests 3 or 4.
  • Of the 20 people with a single positive test followed by multiple negative retests, 6 went on to develop symptoms.
  • 0 people tested positive on antigen tests. There was not a single positive antigen test across this group. They not only didn’t catch covid as early as Cue did, they did not catch any cases at all, including at least 2 people who took the tests while experiencing definitive systems.
    • Antigen tests were a mix of Binax and QuickVue.
    • Early cases took multiple antigen tests over several days, later cases stopped bothering entirely.
    • The “negative test while symptomatic” count is artificially low because I excluded people with ambiguous symptoms, and because later infectees didn’t bother with antigen tests. 
    • I suppose I can’t rule out the possibility that they had an unrelated disease with similar symptoms and a false positive on the Cue test. But it seems unlikely that that happened 10-28 times out a few hundred people without leaving other evidence.

A common defense of antigen tests is that they detect whether you’re contagious at that moment, not whether you will eventually become contagious. Given the existence of people who tested antigen-negative while Cue-positive and symptomatic, I can’t take that seriously.

Unfortunately Cue tests are very expensive. You need a dedicated reader, which is $250, and tests are $65 each (some discount if you sign up for a subscription). A reader can only run 1 test at a time and each test takes 30 minutes, so you need a lot for large gatherings even if people stagger their entrances. 

My contact’s best guess is that the aggressive testing reduced but did not eliminate in-office spread, but it’s hard to quantify because any given case could have been caught outside the office, and because they were trying so many interventions at once. Multiple people tested positive, took a second test right away, and got a negative result, some of whom went on to develop symptoms; we should probably assume the same chance of someone testing negative when a second test would have come back positive, and some of those would have been true positives. So even extremely aggressive testing has gaps.

Meanwhile, have I mentioned lately how good open windows and air purifiers are for covid? And other illnesses, and pollution? And that taping a HEPA filter to a box fan is a reasonable substitute for an air purifier achievable for a very small number of dollars? Have you changed your filter recently? 

PS. Before you throw your antigen tests out, note that they are more useful than Cue tests for determining if you’re over covid. Like PCR, NAAT can continue to pick up dead RNA for days, maybe weeks, after you have cleared the infection. A negative antigen test after symptoms have abated and there has been at least one positive test is still useful evidence to me. 

PPS. I went through some notes and back in September I estimated that antigen testing would catch 25-70% of presymptomatic covid cases. Omicron moves faster, maybe faster enough that 25% was reasonable for delta, but 70% looks obviously too high now. 

PPPS. Talked to another person at the office, their take is the Cue tests are oversensitive. I think this fits the data worse but feel obliged to pass it on since they were there and I wasn’t.

PPPPS (5/02): multiple people responded across platforms that they had gotten positive antigen tests. One or two of these was even presymptomatic. I acknowledge the existence proof but will not be updating until the data has a denominator. If you’re doing a large event like a conference I encourage you to give everyone both cue, antigen, and rapid PCR tests and record their results, and who eventually gets sick. If you’d like help designing this experiment in more detail please reach out (elizabeth-at-acesounderglass.com)

I Caught Covid And All I Got Was This Lousy Ambiguous Data

Tl;dr I tried to run an n of 1 study on niacin and covid, and it failed to confirm or disprove anything at all.

You may remember that back in October I published a very long post investigating a niacin-based treatment protocol for long covid. My overall conclusion was “seems promising but not a slam dunk; I expect more rigorous investigation to show nothing but we should definitely check”. 

Well recently I got covid and had run out of more productive things I was capable of doing, so decided to test the niacin theory. I learned nothing but it was a lot of effort and I deserve a blog post out of it null results are still results so I’m sharing anyway.

Background On Niacin

Niacin is a B-vitamin used in a ton of metabolic processes. If you’re really curious, I describe it in excruciating detail in the original post.

All B vitamins are water-soluble, and it is generaly believed that unless you take unbelievably stupid doses you will pee out any excess intake without noticing. It’s much harder to build up stores of water-soluble vitamins than fat vitamins, so you need a more regular supply.  Niacin is a little weird among the water-solubles in that it gives very obvious signs of overdose: called flush, the symptoms consist of itchy skin and feeling overheated. Large doses can lead to uncontrolled shaking, but why would you ever take that much, when it’s so easy to avoid?

People regularly report response patterns that sure look like their body has a store of niacin that can be depleted and refilled over time. A dose someone has been taking for weeks or months will suddenly start giving them flush, and if they don’t lower it the flush symptoms will get worse and worse. 

Some forms of niacin don’t produce flush. Open question if those offer the same benefits with no side effects, offer fewer benefits, or are completely useless.

Niacin And Long Covid

There’s an elaborate hypothesis about how covid depletes niacin (and downstream products), and this is a contributor to long covid. My full analysis is here. As of last year I hadn’t had covid (this is antibody test confirmed, I definitely didn’t have an asymptomatic case) but I did have lingering symptoms from my vaccine and not a lot else to try, so I gave the protocol a shot.

My experience was pretty consistent with the niacin-storage theory. I spent a long time at quite a high dose of the form of niacin the protocol recommends, nictonic acid. My peak dose without flush was at  least 250mg (1563% RDA) and maybe even 375mg (2345% RDA). When I hit my limit I lowered my dose until I started getting flush at the new dose, and eventually went off nicotnic acid entirely (although I restarted a B-vitamin that included 313% RDA of a different form). That ended in September or early October 2021. It made no difference in my lingering vaccine symptoms.

In early 2022 I tried nicotinic acid again. Even ¼ tablet (62.5mg, 390% RDA) gave me flush.

I Get Covid

Once I developed symptoms and had done all the more obviously useful things like getting Paxlovid, I decided it would be fun to test myself with niacin (and the rest of the supplement stack discussed in my post) and see if covid had any effect. So during my two weeks of illness and week of recovery I occasionally took nicotinic acid and recorded my results. Here’s the overall timeline:

  1. Day -2: am exposed to covid.
  2. Day 0: test positive on a cue test (a home test that uses genetic amplification).
    1. Lung capacity test: 470 (over 400 is considered health).
    2. Start Fluvamoxine and the vitamin cocktail, although I’m inconsistent with both the new and existing vitamins during the worst of the illness. Vitamin cocktail includes 313% RDA of no-flush niacin, but not nicotinic acid. 
  3. Day 1: symptomatic AF. 102.3 degree fever, awake only long enough to pee, refill my water, and make sure my O2 saturation isn’t going to kill me. I eat nothing the entire day.
    1. I monitored my O2 throughout this adventure but it never went into a dangerous zone so I’m leaving it out of the rest of the story.
  4. Day 2: start with 99 degree fever, end day with no fever. Start Paxlovid.
    1. Every day after this I am awake a little bit longer, eat a little bit more, and have a little more cognitive energy, although it takes a while to get back to normal. 
    2. Try ¼ tab nicotinic acid (62.5 mg/ 375% RDA), no flush.
    3. Lung capacity troughs at 350 (considered orange zone).
  5. Day 4: ½ tablet nictonic acid, mild flush.
  6. Day 7: lung capacity up to 450, it will continue to vary from 430-450 for the next two weeks before occasionally going higher.
  7. Day 9: ½ tablet nictonic acid, mild flush
  8. Day 10-17: ⅓ tablet nictonic acid, no flush
    1. Where by “⅓” tablet I mean “I bit off an amount of pill that was definitely >¼ and <½ and probably averaged to ~⅓ over time”
  9. Day 12: I test positive on a home antigen test
  10. Day 15: I test negative on a home antigen test (no tests in between) 
  11. Day 17: ⅓ tablet produces flush (and a second negative antigen test)
    1. This was also the first day I left my house. I had thought of myself as still prone to fatigue but ended up having a lot of energy once I got out of my house and have been pretty okay since.

Conclusions

My case of covid was about as bad as you get while still technically counting as mild. Assuming I went into it with niacin stores such that 62.5mg nicotinic acid would generate flush, it looks like covid immediately took a small bite out of them. Or it reduced my absorption of vitamins, such that the same oral dosage resulted in less niacin being taken in. There’s no way to know covid had a larger effect on niacin than other illnesses, because I don’t have any to compare it to. Or maybe the whole thing was an artifact of “not eating for two days, and then only barely, and being inconsistent with my vitamins for a week”.

Bazant: An alternate covid calculator

Most of what I see people use Microcovid.org for now is estimating risk for large gatherings, which it was not designed for and thus doesn’t handle very well. I spent a few hours going through every covid calculator I could find and this calculator from the Bazant lab at MIT, while less user-friendly than Microcovid and having some flaws of its own, is tailored made for calculating risks for groups indoors, and I think it is worth a shot. 

[Note: I’ll be discussing the advanced version of the calculator here; I found the basic version too limited]

The Bazant calculator comes out of physics lab with a very detailed model of how covid particles hang and decay in the air, and how this is affected by ventilation and filtration. I haven’t checked their model, but I never checked Microcovid’s model either. The Bazant calculator lets you very finely adjust the parameters of a room: dimensions, mechanical ventilation, air filtration, etc. It combines those with more familiar parameters like vaccination and mask usage and feeds them into the model in this paper to produce an estimate of how long N people can be in a room before they accumulate a per-person level of risk between 0 and 1 (1 = person is definitely getting covid = 1,000,000 microcovids per person; .1 = 10% chance someone gets sick = 100,000 microcovids per person). It also produces an estimate of how much CO2 should accumulate over that time, letting you use a CO2 monitor to check its work and notice if risk is accumulating more rapidly than expected.

Reasons/scenarios to use the Bazant calculator over Microcovid:

  • You have a large group and want to set % immunized or effective mask usage for the group as a whole, instead of configuring everyone’s vaccinations and masks individually.
  • You want to incorporate the mechanics of the room and ventilation in really excruciating detail. 
  • You want to set your own estimate for prevalence based on beliefs about your subpopulation.
  • You want a live check on your work, in the form of the CO2 estimates.

Reasons to use Microcovid instead:

  • Your scenario is outside – Bazant calculator doesn’t handle this at all.
  • You don’t want to have an opinion on infection prevalence, immunization, or mask usage.
  • Your masks are better than surgical masks (Bazant doesn’t handle N95 or similar. Also, it rates surgical masks as 90% effective, which seems very high to me).
  • Your per-person risk tolerance is < 10,000 microcovids (Bazant calculator can’t bet set at a lower risk tolerance, although you can do math on their results to approximate this).
  • You’re still using a bubble model, or tracking accumulated risk rather than planning for an event.

Scenarios neither handle well

  • Correlated risk. You might be fine with 10% of your attendees getting sick, but not a 10% chance of all of the attendees getting sick at once.
  • Differences in risk from low-dose vs. high-dose exposures.

I’m not currently planning any big events, but if someone else is, please give this a try and let us know if it is useful. 

My Tiny Study Finds No Effect For Ketone Esters as a Nootropic

Several months ago I announced an RCT  I was running for a client to test ketone esters as a nootropic. After many more months than planned (pro-tip: do not create a study that depends on people with ADHD following a finicky multi-step process, and if you must, make your ops perfect) I failed to find an effect of ketone esters on productivity. It was a small study and a lot went wrong so this failure isn’t conclusive, but since I expected to find an enormous effect size, I’m still pretty disappointed. 

I originally intended to write up detailed results and a post-mortem on experimental operations. I’m unfortunately pressed for time right now, and can’t work up the motivation to detail exactly how little effect ketone esters had. If you’re considering running your own experiment and want the details, please reach out here or over email; I’m happy to share if it will be useful, but I didn’t want to delay publishing over details that probably don’t matter. However, you can see the experimental protocol in detail here. If you check, the hash of that file matches the hash in the recruitment doc.

The overall outcome: if you compare the average productivity for control and treatment days per person, and then average the per person difference together, that average is -0.2 (scale of 1-10), meaning treatment made people slightly worse on average. If you include only people who completed more than 10 of the planned 14 observations (4 out of 7), the average improvement is 0.3 (same 1-10 scale). The largest improvement for any one person was 1.1. I didn’t bother to check for statistical significance because even if it eked out a p-value, the real-world significance is too small.  

This result is confusing, because ketone esters obviously work really well for me, and a few other people I know. That could be a placebo, but the effects are so strong and consistent that that seems unlikely. It’s possible there was an experimental error – our operations were really not what they needed to be for a study of anyone, much less people specifically selected for having difficulties with organization – but even if I squint at the data I can’t see the kinds of patterns that might hint at that. Or the design could have been flawed; I was trading off ideal circumstances and ease of use for volume of participants. 

Given these results, I intend to keep taking ketone esters when they feel helpful to me, but can’t recommend them as in any way vindicated for mass use as a nootropic. 

Epistemic Legibility

Tl;dr: being easy to argue with is a virtue, separate from being correct.

Introduction

Regular readers of my blog know of my epistemic spot check series, where I take claims (evidential or logical) from a work of nonfiction and check to see if they’re well supported. It’s not a total check of correctness: the goal is to rule out things that are obviously wrong/badly formed before investing much time in a work, and to build up my familiarity with its subject. 

Before I did epistemic spot checks, I defined an easy-to-read book as, roughly, imparting an understanding of its claims with as little work from me as possible. After epistemic spot checks, I started defining easy to read as “easy to epistemic spot check”. It should be as easy as possible (but no easier) to identify what claims are load-bearing to a work’s conclusions, and figure out how to check them. This is separate from correctness: things can be extremely legibly wrong. The difference is that when something is legibly wrong someone can tell you why, often quite simply. Illegible things just sit there at an unknown level of correctness, giving the audience no way to engage.

There will be more detailed examples later, but real quick: “The English GDP in 1700 was $890324890. I base this on $TECHNIQUE interpretation of tax records, as recorded in $REFERENCE” is very legible (although probably wrong, since I generated the number by banging on my keyboard). “Historically, England was rich” is not. “Historically, England was richer than France” is somewhere in-between. 

“It was easy to apply this blog post format I made up to this book” is not a good name, so I’ve taken to calling the collection of traits that make things easy to check “epistemic legibility”, in the James C. Scott sense of the word legible. Legible works are (comparatively) easy to understand, they require less external context, their explanations scale instead of needing to be tailored for each person. They’re easier to productively disagree with, easier to partially agree with instead of forcing a yes or no, and overall easier to integrate into your own models.

[Like everything in life, epistemic legibility is a spectrum, but I’ll talk about it mostly as a binary for readability’s sake]

When people talk about “legible” in the Scott sense they often mean it as a criticism, because pushing processes to be more legible cuts out illegible sources of value. One of the reasons I chose the term here is that I want to be very clear about the costs of legibility and the harms of demanding it in excess. But I also think epistemic legibility leads people to learn more correct things faster and is typically underprovided in discussion.

If I hear an epistemically legible argument, I have a lot of options. I can point out places I think the author missed data that impacts their conclusion, or made an illogical leap. I can notice when I know of evidence supporting their conclusions that they didn’t mention. I can see implications of their conclusions that they didn’t spell out. I can synthesize with other things I know, that the author didn’t include.

If I hear an illegible argument, I have very few options. Perhaps the best case scenario is that it unlocks something I already knew subconsciously but was unable to articulate, or needed permission to admit. This is a huge service! But if I disagree with the argument, or even just find it suspicious, my options are kind of crap. I write a response of equally low legibility, which is unlikely to improve understanding for anyone. Or I could write up a legible case for why I disagree, but that is much more work than responding to a legible original, and often more work than went into the argument I’m responding to, because it’s not obvious what I’m arguing against.  I need to argue against many more things to be considered comprehensive. If you believe Y because of X, I can debate X. If you believe Y because …:shrug:… I have to imagine every possible reason you could do so, counter all of them, and then still leave myself open to something I didn’t think of. Which is exhausting.

I could also ask questions, but the more legible an argument is, the easier it is to know what questions matter and the most productive way to ask them. 

I could walk away, and I am in fact much more likely to do that with an illegible argument. But that ends up creating a tax on legibility because it makes one easier to argue with, which is the opposite of what I want.

Not everything should be infinitely legible. But I do think more legibility would be good on most margins, that choices of the level of legibility should be made more deliberately, and that we should treat highly legible and illegible works more differently than we currently do. I’d also like a common understanding of legibility so that we can talk about its pluses and minuses, in general or for a particular piece.

This is pretty abstract and the details matter a lot, so I’d like to give some better examples of what I’m gesturing at. In order to reinforce the point that legibility and correctness are orthogonal; this will be a four quadrant model. 

True and Legible

Picking examples for this category was hard. No work is perfectly true and perfectly legible, in the sense of being absolutely impossible to draw an inaccurate conclusion from and having no possible improvements to legibility, because reality is very complicated and communication has space constraints. Every example I considered, I could see a reason someone might object to it. And the things that are great at legibility are often boring. But it needs an example so…

Acoup

Bret Devereaux over at Acoup consistently writes very interesting history essays that I found both easy to check and mostly true (although with some room for interpretation, and not everyone agrees). Additionally, a friend of mine who is into textiles tells me his textile posts were extremely accurate. So Devereaux does quite well on truth and legibility, despite bringing a fair amount of emotion and strong opinions to his work. 

As an example, here is a paragraph from a post arguing against descriptions of Sparta as a highly equal society.

But the final word on if we should consider the helots fully non-free is in their sanctity of person: they had none, at all, whatsoever. Every year, in autumn by ritual, the five Spartan magistrates known as the ephors (next week) declared war between Sparta and the helots – Sparta essentially declares war on part of itself – so that any spartiate might kill any helot without legal or religious repercussions (Plut. Lyc. 28.4; note also Hdt. 4.146.2). Isocrates – admittedly a decidedly anti-Spartan voice – notes that it was a religious, if not legal, infraction to kill slaves everywhere in Greece except Sparta (Isoc. 12.181). As a matter of Athenian law, killing a slave was still murder (the same is true in Roman law). One assumes these rules were often ignored by slave-holders of course – we know that many such laws in the American South were routinely flouted. Slavery is, after all, a brutal and inhuman institution by its very nature. The absence of any taboo – legal or religious – against the killing of helots marks the institution as uncommonly brutal not merely by Greek standards, but by world-historical standards.

Here we have some facts on the ground (Spartiates could kill their slaves, killing slaves was murder in most contemporaneous societies), sources for some but not all of them (those parentheticals are highly readable if you’re a classicist, and workable if you’re not), the inference he drew from them (Spartans treated their slaves unusually badly), and the conclusions he drew from that (Sparta was not only inequitable, it was unusually inequitable even for its time and place).

Notably, the entire post relies heavily on the belief that slavery is bad, which Devereaux does not bother to justify. That’s a good choice because it would be a complete waste of time for modern audiences – but it also makes this post completely unsuitable for arguing with anyone who disagreed. If for some reason you needed to debate the ethics of slavery, you need work that makes a legible case for that claim in particular, not work that takes it as an axiom.

Exercise for Mood and Anxiety

A few years ago I ESCed Exercise for Mood and Anxiety, a book that aims to educate people on how exercise can help their mental health and then give them the tools to do so. It did really well at the former: the logic was compelling and the foundational evidence was well cited and mostly true (although exercise science always has wide error bars). But out of 14 people who agreed to read the book and attempt to exercise more, only three reported back to me and none of them reported an increase in exercise. So EfMaA is true and epistemically legible, but nonetheless not very useful. 

True but Epistemically Illegible

You Have About Five Words is a poetic essay from Ray Arnold. The final ~paragraph is as follows:

If you want to coordinate thousands of people…

You have about five words.

This has ramifications on how complicated a coordinated effort you can attempt.

What if you need all that nuance and to coordinate thousands of people? What would it look like if the world was filled with complicated problems that required lots of people to solve?

I guess it’d look like this one.

I think the steelman of its core claim, that humans are bad at remembering long nuanced writing and the more people you are communicating with, the more you need to simplify your writing, is obviously true. This is good, because Ray isn’t doing crap to convince me of it. He cites no evidence and gives no explanation of his logic. If I thought nuance increased with the number of readers I would have nothing to say other than “no you’re wrong” or write my own post from scratch, because he gives no hooks to refute. If someone tried to argue that you get ten words rather than five, I would think they were missing the point. If I thought he had the direction right but got the magnitude of the effect wrong enough that it mattered (and he was a stranger rather than a friend), I would not know where to start the discussion.

[Ray gets a few cooperation points back by explicitly labeling this as poetry, which normally I would be extremely happy about, but it weakened its usefulness as an example for this post so right this second I’m annoyed about it.]

False but Epistemically Legible

Mindset

I think Carol Dweck’s Mindset and associated work is very wrong, and I can produce large volumes on specific points of disagreement. This is a sign of a work that is very epistemically legible: I know what her cruxes are, so I can say where I disagree. For all the shit I’ve talked about Carol Dweck over the years, I appreciate that she made it so extraordinarily easy to do so, because she was so clear on where her beliefs came from. 

For example, here’s a quote from Mindset

All children were told that they had performed well on this problem set: “Wow, you did very well on these problems. You got [number of problems] right. That’s a really high score!” No matter what their actual score, all children were told that they had solved at least 80% of the problems that they answered.

Some children were praised for their ability after the initial positive feedback: “You must be smart at these problems.” Some children were praised for their effort after the initial positive feedback: “You must have worked hard at these problems.” The remaining children were in the control condition and received no additional feedback.

And here’s Scott Alexander’s criticism

This is a nothing intervention, the tiniest ghost of an intervention. The experiment had previously involved all sorts of complicated directions and tasks, I get the impression they were in the lab for at least a half hour, and the experimental intervention is changing three short words in the middle of a sentence.

And what happened? The children in the intelligence praise condition were much more likely to say at the end of the experiment that they thought intelligence was more important than effort (p < 0.001) than the children in the effort condition. When given the choice, 67% of the effort-condition children chose to set challenging learning-oriented goals, compared to only 8% (!) of the intelligence-condition. After a further trial in which the children were rigged to fail, children in the effort condition were much more likely to attribute their failure to not trying hard enough, and those in the intelligence condition to not being smart enough (p < 0.001). Children in the intelligence condition were much less likely to persevere on a difficult task than children in the effort condition (3.2 vs. 4.5 minutes, p < 0.001), enjoyed the activity less (p < 0.001) and did worse on future non-impossible problem sets (p…you get the picture). This was repeated in a bunch of subsequent studies by the same team among white students, black students, Hispanic students…you probably still get the picture.

Scott could make those criticisms because Dweck described her experiment in detail. If she’d said “we encouraged some kids and discouraged others”, there would be a lot more ambiguity.

Meanwhile, I want to criticize her for lying to children. Messing up children’s feedback system creates the dependencies on adult authorities that lead to problems later in life. This is extremely bad even if it produces short-term improvements (which it doesn’t). But I can only do this with confidence because she specified the intervention.

The Fate of Rome

This one is more overconfident than false. The Fate of Rome laid out very clearly how they were using new tools for recovering meteorological data to determine the weather 2000 years ago, and using that to analyze the Roman empire. Using this new data, it concludes that the peak of Rome was at least partially caused by a prolonged period of unusually good farming weather in the Mediterranean, and that the collapse started or was worsened when the weather began to regress to the mean.

I looked into the archeometeorology techniques and determined that they, in my judgement, had wider confidence intervals than the book indicated, which undercut the causality claims. I wish the book had been more cautious with its evidence, but I really appreciate that they laid out their reasoning so clearly, which made it really easy to look up points I might disagree with them on.

False and Epistemically Illegible

Public Health and Airborne Pathogen Transmission

I don’t know exactly what the CDC’s or WHO’s current stance is on breathing-based transmission of covid, and I don’t care, because they were so wrong for so long in such illegible ways. 

When covid started, the CDC and WHO’s story was that it couldn’t be “airborne”, because the viral particle was > 5 microns.  That phrasing was already anti-legible for material aimed at the general public, because airborne has a noticeably different definition in virology (”can persist in the air indefinitely”) than it does for popular use (”I can catch this through breathing”). But worse than that, they never provided any justification for the claim. This was reasonable for posters, but not everything was so space constrained, and when I looked in February 2021 I could not figure out where the belief that airborne transmission was rare was coming from. Some researcher eventually spent dozens to hundreds of hours on this and determined the 5 micron number probably came from studies of tuberculosis, which for various reasons needs to get deeper in the lungs than most pathogens and thus has stronger size constraints. If the CDC had pointed to their sources from the start we could have determined the 5 micron limit was bullshit much more easily (the fact that many relevant people accepted it without that proof is a separate issue).

When I wrote up the Carol Dweck example, it was easy. I’m really confident in what Carol Dweck believed at the time of writing Mindset, so it’s really easy to describe why I disagree. Writing this section on the CDC was harder, because I cannot remember exactly what the CDC said and when they said it; a lot of the message lived in implications; their statements from early 2020 are now memory holed and while I’m sure I could find them on archive.org, it’s not really going to quiet the nagging fear that someone in the comments is going to pull up a different thing they said somewhere else that doesn’t say exactly what I claimed they said, or that I view as of a piece with what I cited but both statements are fuzzy enough that it would be a lot of work to explain why I think the differences are immaterial….

That fear and difficulty in describing someone’s beliefs is the hallmark of epistemic illegibility. The wider the confidence interval on what someone is claiming, the more work I have to do to question it.

And More…

The above was an unusually legible case of illegibility. Mostly illegible and false arguments don’t feel like that. They just feel frustrating and bad and like the other person is wrong but it’s too much work to demonstrate how. This is inconveniently similar to the feeling when the other person is right but you don’t want to admit it. I’m going to gesture some more at illegibility here, but it’s inherently an illegible concept so there will be genuinely legible (to someone) works that resemble these points, and illegible works that don’t.

Marks of probable illegibility:

  • The person counters every objection raised, but the counters aren’t logically consistent with each other. 
  • You can’t nail down exactly what the person actually believes. This doesn’t mean they’re uncertain – saying “I think this effect is somewhere between 0.1x and 10000x” is very legible, and sometimes the best you can do given the data. It’s more that they imply a narrow confidence band, but the value that band surrounds moves depending on the subargument. Or they agree they’re being vague but they move forward in the argument as if they were specific. 
  • You feel like you understand the argument and excitedly tell your friends. When they ask obvious questions you have no answer or explanation. 

A good example of illegibly bad arguments that are specifically trying to ape legibility are a certain subset of alt-medicine advertisements. They start out very specific, with things like “there are 9804538905 neurons in your brain carrying 38923098 neurotransmitters”, with rigorous citations demonstrating those numbers. Then they introduce their treatment in a way that very strongly implies it works with those 38923098 transmitters but not, like, what it does to them or why we would expect that to have a particular effect. Then they wrap it up with some vague claims about wellness, so you’re left with the feeling you’ll definitely feel better if you take their pill, but if you complain about any particular problem it did not fix they have plausible deniability.

[Unfortunately the FDA’s rules around labeling encourage this illegibility even for products that have good arguments and evidence for efficacy on specific problems, so the fact that a product does this isn’t conclusive evidence it’s useless.]

Bonus Example: Against The Grain

The concept of epistemic legibility was in large part inspired by my first attempt at James C. Scott’s Against the Grain (if that name seems familiar: Scott also coined “legibility” in the sense in which I am using it), whose thesis is that key properties of grains (as opposed to other domesticates) enabled early states. For complicated reasons I read more of AtG without epistemic checking than I usually would, and then checks were delayed indefinitely, and then covid hit, and then my freelancing business really took off… the point is, when I read Against the Grain in late 2019, it felt like it was going to be the easiest epistemic spot check I’d ever done. Scott was so cooperative in labeling his sources, claims, and logical conclusions. But when I finally sat down to check his work, I found serious illegibilities.

I did the spot check over Christmas this year (which required restarting the book). It was maybe 95% as good as I remembered, which is extremely high. At chapter 4 (which is halfway through the book, due to the preface and introduction), I felt kinda overloaded and started to spot check some claims (mostly factual – the logical ones all seemed to check out as I read them). A little resentfully, I checked this graph.

This should have been completely unnecessary, Scott is a decent writer and scientist who was not going to screw up basic dates. I even split the claims section of the draft into two sections, “Boring” and “Interesting”, because I obviously wasn’t going to come up with anything checking names and dates and I wanted that part to be easy to skip.

I worked from the bottom. At first, it was a little more useful than I expected – a major new interpretation of the data came out the same year the book was published, so Scott’s timing on anatomically modern humans was out of date, but not in a way that reflected poorly on him.

Finally I worked my way up to “first walled, territorial state”. Not thinking super hard, I googled “first walled city”, and got a date 3000 years before the one Scott cites. Not a big deal, he specified state, not walls. What I can google to find that out? “Earliest state”, obviously, and the first google hit does match Scott’s timing, but… what made something a state, and how can we assess those traits from archeological records? I checked, and nowhere in the preface, introduction, or first three chapters was “state” defined. No work can define every term it uses, but this is a pretty important one for a book whose full title is Against the Grain: A Deep History of the Earliest States

You might wonder if “state” had a widespread definition such that it didn’t need to be defined. I think this is not the case for a few reasons. First, Against The Grain is aimed at a mainstream audience, and that requires defining terms even if they’re commonly known by experts. Second, even if a reader knew the common definition of what made a state, how you determine whether something was a state or merely a city from archeology records is crucial for understanding the inner gears of the book’s thesis. Third, when Scott finally gives a definition, it’s not the same as the one on wikipedia.

[longer explanation] Among these characteristics, I propose to privilege those that point to territoriality and a specialized state apparatus: walls, tax collection, and officials.

Against the Grain

States are minimally defined by anthropologist David S. Sandeford as socially stratified and bureaucratically governed societies with at least four levels of settlement hierarchy (e.g., a large capital, cities, villages, and hamlets)

Wikipedia (as of 2021-12-26)

These aren’t incompatible, but they’re very far from isomorphic. I expect that even though there’s a fairly well accepted definition of state in the relevant field(s), there are disputed edges that matter very much for this exact discussion, in which Scott views himself as pushing back against the commonly accepted narrative. 

To be fair, the definition of state was not that relevant to chapters 1-3, which focus on pre-state farming. Unless, you know, your definition of “state” differs sufficiently from his. 

Against The Grain was indeed very legible in other ways, but loses basically all of its accrued legibility points and more for not making even a cursory definition of a crucial term in the introduction, and for doing an insufficient job halfway through the book.

This doesn’t mean the book is useless, but it does mean it was going to be more work to extract value from than I felt like putting in on this particular topic.

Why is this Important?

First of all, it’s costing me time.

I work really hard to believe true things and disbelieve false things, and people who argue illegibly make that harder, especially when people I respect treat arguments as more proven than their level of legibility allows them to be. I expect having a handle with which to say “no I don’t have a concise argument about why this work is wrong, and that’s a fact about the work” to be very useful.

More generally, I think there’s a range of acceptable legibility levels for a given goal, but we should react differently based on which legibility level the author chose, and that arguments will be more productive if everyone involved agrees on both the legibility level and on the proper response to a given legibility level. One rule I have is that it’s fine to declare something a butterfly idea and thus off limits to sharp criticism, but that inherently limits the calls to action you can make based on that idea. 

Eventually I hope people will develop some general consensus around the rights and responsibilities of a given level of legibility, and that this will make arguments easier and more productive. Establishing those rules is well beyond the scope of this post. 

Legibility vs Inferential Distance

You can’t explain everything to everyone all of the time. Some people are not going to have the background knowledge to understand a particular essay of yours. In cases like this, legibility is defined as “the reader walks away with the understanding that they didn’t understand your argument”. Illegibility in this case is when they erroneously think they understand your argument. In programming terms, it’s the difference between a failed function call returning a useful error message (legible), versus failing silently (illegible).  

A particularly dangerous way this can occur is when you’re using terms of art (meaning: words or phrases that have very specific meanings within a field) that are also common English words. You don’t want someone thinking you’re dismissing a medical miracle because you called it statistically insignificant, or invalidating the concept of thought work because it doesn’t apply force to move an object.

Cruelly, misunderstanding becomes more likely the more similar the technical definition is to the English definition. I watched a friend use the term “common knowledge” to mean “everyone knows that everyone knows, and everyone knows that everyone knows… and that metaknoweldge enables actions that wouldn’t be possible if it was merely true that everyone knew and thought they were the only one, and those additional possible actions are extremely relevant to our current conversation” to another friend who thought “common knowledge” meant “knowledge that is common”, and had I not intervened the ensuing conversation would have been useless at best.

Costs of Legibility

The obvious ones are time and mental effort, and those should not be discounted. Given a finite amount of time, legibility on one work trades off against another piece being produced at all, and that may be the wrong call.

A second is that legibility can make things really dry. Legibility often means precision, and precision is boring, especially relative to work optimized to be emotionally activating. 

Beyond that, legibility is not always desirable. For example, unilateral legibility in an adversarial environment makes you vulnerable, as you’re giving people the keys to the kingdom of “effective lies to tell you”. 

Lastly, premature epistemic legibility kills butterfly ideas, which are beautiful and precious and need to be defended until they can evolve combat skills.

How to be Legible

This could easily be multiple posts, I’m including a how-to section here more to help convey the concept of epistemic legibility than write a comprehensive guide to how to do it. The list is not a complete list, and items on it can be faked. I think a lot of legibility is downstream of something harder to describe. Nonetheless, here are a few ways to make yourself more legible, when that is your goal.

  • Make it clear what you actually believe.
    • Watch out for implicit quantitative estimates (“probably”, “a lot”, “not very much”) and make them explicit, even if you have a very wide confidence interval. The goals here are twofold: the first is to make your thought process explicit to you. The second is to avoid confusion – people can mean different things by “many”, and I’ve seen some very long arguments suddenly resolve when both sides gave actual numbers.
  • Make clear the evidence you are basing your beliefs on.
    • This need not mean “scientific fact” or “RCT”. It could be “I experienced this a bunch in my life” or “gut feeling” or “someone I really trust told me so”. Those are all valid reasons to believe things. You just need to label them.
  • Make that evidence easy to verify.
    • More accessible sources are better.
      • Try to avoid paywalls and $900 books with no digital versions.
      • If it’s a large work, use page numbers or timestamps to the specific claim, removing the burden to read an entire book to check your work (but if your claim rests on a large part of the work, better to say that than artificially constrict your evidence)
    • One difficulty is when the evidence is in a pattern, and no one has rigorously collated the data that would let you demonstrate it. You can gather the data yourself, but if it takes a lot of time it may not be worth it. 
    • In times past, when I wanted to refer to a belief I had in a blog post but didn’t have a citation for it, I would google the belief and link to the first article that came up. I regret this. Just because an article agrees with me doesn’t mean it’s good, or that its reasoning is my reasoning. So one, I might be passing on a bad argument. Two, I know that, so if someone discredits the linked article it doesn’t necessarily change my mind, or even create in me a feeling of obligation to investigate. I now view it as more honest to say “I believe this but only vaguely remember the reasons why”, and if it ends up being a point of contention I can hash it out later.
  • Make clear the logical steps between the evidence and your final conclusion.
  • Use examples. Like, so many more examples than you think. Almost everything could benefit from more examples, especially if you make it clear when they’re skippable so people who have grokked the concept can move on.
    • It’s helpful to make clear when an example is evidence vs when it’s a clarification of your beliefs. The difference is if you’d change your mind if the point was proven false: if yes, it’s evidence. If you’d say “okay fine, but there are a million other cases where the principle holds”, it’s an example.  One of the mistakes I made with early epistemic spot checks was putting too much emphasis on disproving examples that weren’t actually evidence.
  • Decide on an audience and tailor your vocabulary to them. 
    • All fields have words that mean something different in the field than in general conversation, like “work”, “airborne”, and “significant”. If you’re writing within the field, using those terms helps with legibility by conveying a specific idea very quickly. If you’re communicating outside the field, using such terms without definition hinders legibility, as laypeople misapply their general knowledge of the English language to your term of art and predictably get it wrong. You can help on the margins by defining the term in your text, but I consider some uses of this iffy.
      • The closer the technical definition of a term is to its common usage, the more likely this is to be a problem because it makes it much easier for the reader to think they understand your meaning when they don’t.
    • At first I wanted to yell at people who use terms of art in work aimed at the general population, but sometimes it’s unintentional, and sometimes it’s a domain expert who’s bad at public speaking and has been unexpectedly thrust onto a larger stage, and we could use more of the latter, so I don’t want to punish people too much here. But if you’re, say, a journalist who writes a general populace book but uses an academic term of art in a way that will predictably be misinterpreted, you have no such excuse and will go to legibility jail. 
    • A skill really good interviewers bring to the table is recognizing terms of art that are liable to confuse people and prompting domain experts to explain them.
  • Write things down, or at least write down your sources. I realize this is partially generational and Gen Z is more likely to find audio/video more accessible than written work, and accessibility is part of legibility. But if you’re relying on a large evidence base it’s very disruptive to include it in audio and very illegible to leave it out entirely, so write it down.
  • Follow all the rules of normal readability – grammar, paragraph breaks, no run-on sentences, etc.

A related but distinct skill is making your own thought process legible. John Wentworth describes that here.

Synthesis

“This isn’t very epistemically legible to me” is a valid description (when true), and a valid reason not to engage. It is not automatically a criticism.

“This idea is in its butterfly stage”, “I’m prioritizing other virtues” or “this wasn’t aimed at you” are all valid defenses against accusations of illegibility as a criticism (when true), but do not render the idea more legible.

“This call to action isn’t sufficiently epistemically legible to the people it’s aimed at” is an extremely valid criticism (when true), and we should be making it more often.

I apologize to Carol Dweck for 70% of the vigor of my criticism of her work; she deserves more credit than I gave her for making it so easy to do that. I still think she’s wrong, though.

Epilogue: Developing a Standard for Legibility

As mentioned above, I think the major value add from the concept of legibility is that it lets us talk about whether a given work is sufficiently legible for its goal. To do this, we need to have some common standards for how much legibility a given goal demands. My thoughts on this are much less developed and by definition common standards need to be developed by the community that holds them, not imposed by a random blogger, so I’ll save my ideas for a different post. 

Epilogue 2: Epistemic Cooperation

Epistemic legibility is part of a broader set of skills/traits I want to call epistemic cooperation. Unfortunately, legibility is the only one I have a really firm handle on right now (to the point I originally conflated the concepts, until a few conversations highlighted the distinction- thanks friends!). I think epistemic cooperation, in the sense of “makes it easy for us to work together to figure out the truth” is a useful concept in its own right, and hope to write more about it as I get additional handles. In the meantime, there are a few things I want to highlight as increasing or signalling cooperation in general but not legibility in particular:

  • Highlight ways your evidence is weak, related things you don’t believe, etc.
  • Volunteer biases you might have.
  • Provide reasons people might disagree with you.
  • Don’t emotionally charge an argument beyond what’s inherent in the topic, but don’t suppress emotion below what’s inherent in the topic either.
  • Don’t tie up brain space with data that doesn’t matter.

Thanks to Ray Arnold, John Salvatier, John Wentworth, and Matthew Graves for discussion on this post. 

Butterfly Ideas

Or “How I got my hyperanalytical friends to chill out and vibe on ideas for 5 minutes before testing them to destruction”

Sometimes talking with my friends is like intellectual combat, which is great. I am glad I have such strong cognitive warriors on my side. But not all ideas are ready for intellectual combat. If I don’t get my friend on board with this, some of them will crush an idea before it gets a chance to develop, which feels awful and can kill off promising avenues of investigation. It’s like showing a beautiful, fragile butterfly to your friend to demonstrate the power of flight, only to have them grab it and crush it in their hands, then point to the mangled corpse as proof butterflies not only don’t fly, but can’t fly, look how busted their wings are.

You know who you are

When I’m stuck in a conversation like that, it has been really helpful to explicitly label things as butterfly ideas. This has two purposes. First, it’s a shorthand for labeling what I want (nurturance and encouragement). Second, it explicitly labels the idea as not ready for prime time in ways that make it less threatening to my friends. They can support the exploration of my idea without worrying that support of exploration conveys agreement, or agreement conveys a commitment to act.

This is important because very few ideas start out ready for the rigors of combat. If they’re not given a sheltered period, they will die before they become useful. This cuts us off from a lot of goodness in the world. Examples:

  • A start-up I used to work for had a keyword that meant “I have a vague worried feeling I want to discuss without justifying”. This let people bring up concerns before they had an ironclad case for them and made statements that could otherwise have felt like intense criticism feel more like information sharing (they’re not asserting this will definitely fail, they’re asserting they have a feeling that might lead to some questions). This in turn meant that problems got brought up and addressed earlier, including problems in the classes “this is definitely gonna fail and we need to make major changes” and  “this excellent idea but Bob is missing the information that would help him understand why”.
    • This keyword was “FUD (fear, uncertainty, doubt)”. It is used in exactly the opposite way in cryptocurrency circles, where it means “you are trying to increase our anxiety with unfounded concerns, and that’s bad”. Words are tricky.
  • Power Buys You Distance From The Crime started out as a much less defensible seed of an idea with a much worse explanation. I know that had I talked about it in public it would have caused a bunch of unproductive yelling that made it harder to think because I did and it did (but later, when it was ready, intellectual combat with John Wentworth improved the idea further).
  • The entire genre of “Here’s a cool new emotional tool I’m exploring”
  • The entire genre of “I’m having a feeling about a thing and I don’t know why yet”

I’ve been on the butterfly crushing end of this myself- I’m thinking of a particular case last year where my friend brought up an idea that, if true, would require costly action on my part. I started arguing with the idea, they snapped at me to stop ruining their dreams. I chilled out, we had a long discussion about their goals, how they interpreted some evidence, and why they thought a particular action might further said goals, etc. 

A week later all of my objections to the specific idea were substantiated and we agreed not to do the thing- but thanks to the conversation we had in the meantime, I have a better understanding of them and what kinds of things would be appealing to them in the future. That was really valuable to me and I wouldn’t have learned all that if I’d crushed the butterfly in the beginning.

Notably, checking out that idea was fairly expensive, and only worth it because this was an extremely close friend (which both made the knowledge of them more valuable, and increased the payoff to helping them if they’d been right). If they had been any less close, I would have said “good luck with that” and gone about my day, and that would have been a perfectly virtuous reaction. 

I almost never discuss butterfly ideas on the public internet, or even 1:many channels. Even when people don’t actively antagonize them, the environment of Facebook or even large group chats means that people often read with half their brain and respond to a simplified version of what I said. For a class of ideas that live and die by context and nuance and pre-verbal intuitions, this is crushing. So what I write in public ends up being on the very defensible end of the things I think. This is a little bit of a shame, because the returns to finding new friends to study your particular butterflies with is so high, but ce la vie. 

This can play out a few ways in practice. Sometimes someone will say “this is a butterfly idea” before they start talking. Sometimes when someone is being inappropriately aggressive towards an idea the other person will snap “will you please stop crushing my butterflies!” and the other will get it. Sometimes someone will overstep, read the other’s facial expression, and say “oh, that was a butterfly, wasn’t it?”. All of these are marked improvements over what came before, and have led to more productive discussions with less emotional pain on both sides.

A Quick Look At 20% Time

I was approached by a client to research the concept of 20% time for engineers, and they graciously agreed to let me share my results. Because this work was tailored to the needs of a specific client, it may have gaps or assumptions that make it a bad 101 post, but in the expectation that it is more useful than not publishing at all, I would like to share it (with client permission). 

Side project time, popularized as 20% time at Google, is a policy that allows employees to spend a set percentage of their time on a project of their choice, rather than one directed by management. In practice this can mean a lot of different things, ranging from “spend 20% of your time on whatever you want” to “sure, spend all the free time you want generating more IP for us, as long as your main project is completely unaffected” (often referred to as 120% time) to “theoretically you’re free to do whatever, but we’ve imposed so many restrictions that this means nothing”. I did a 4-hour survey to get a sense of what implementations were available and how they felt for workers.

A frustration here is that almost all of what I could find via Google searches were puff-pieces, anti-puff-pieces, and employees complaining on social media (and one academic article). The single best article I found came not through a Google search, but because I played D&D with the author 15 years ago and she saw me talking about this on Facebook. She can’t be the only one writing about 20% time in a thoughtful way and I’m mad that that writing has been crowded out by work that is, at best, repetitive, and at worst actively misleading.

There are enough anecdotal reports that I believe 20% time exists and is used to good effect by some employees at some companies (including Google) some of the time. The dearth of easily findable information on specific implementations, managerial approaches, trade-offs, etc, makes me downgrade my estimate of how often that happens, vs 20% time being a legible signal of an underlying attitude towards autonomy, or a dubious recruitment tool. I see a real market gap for someone to explain how to do 20% time well at companies of different sizes and product types.

But in the meantime, here’s the summary I gave my client. Reminder: this was originally intended for a high-context conversation with someone who was paying me by the hour, and as such is choppier, less nuanced, and has different emphases than ideal for a public blog post.  

My full notes are available here.

  • To the extent it’s measured, utilization appears to be low, so the policy doesn’t cost very much.
    • In 2015, a Google HR exec estimated utilization at 10% (meaning it took 2% of all employees’ time). 
    • In 2009, 12 months after Atlassian introduced 20% time, recorded utilization was at 5% (meaning employees were measured to spend 1.1% of their time on it) and estimated actual utilization was <=15% (Notably, nobody complains that Atlassian 20% is fake, and I confirmed with a recently departed employee that it was still around as of 2020).
  • Interaction with management and evaluation is key. A good compromise is to let people spend up to N hours on a project, and require a check-in with management beyond that. 
    • Googlers consistently (although not universally) complained on social media that even when 20% time was officially approved, you’d be a fool to use it if you wanted a promotion or raises. 
    • However a manager at a less famous company indicated this hadn’t been a problem for them, and that people who approached perf the way everyone does at Google would be doomed anyway. So it looks like you can get out of this with culture.
    • An approval process is the kiss of death for a feeling of autonomy, but letting employees work on garbage for 6 months and then holding it against them at review time hurts too. 
    • Atlassian requires no approval to start, 3 uninvolved colleagues to vouch for a project to go beyond 5 days, and founder approval at 10 days. This seems to be working okay for them (but see the “costs” section below).
  • Costs of 20% time:
    • Time cost appears to be quite low (<5% of employee time, some of which couldn’t have been spent on core work anyway)
    • Morale effects can backfire: Sometimes devs make tools or projects that are genuinely useful, but not useful enough to justify expanding or sometimes even maintaining them. This leads to telling developers they must give up on a project they value and enjoyed (bad for their morale) or an abundance of tools that developers value but are too buggy to really rely on (bad for other people’s morale). This was specifically called out as a problem at Atlassian.
    • Employees on small teams are less likely to feel able to take 20% time, because they see the burden of core work shifting to their co-workers. But being on a small team already increases autonomy, so that may not matter.
  • Benefits of 20% time:
    • New products. This appears to work well for companies that make the kind of products software developers are naturally interested in, but not otherwise.
    • The gain in autonomy generally causes the improvements in morale and thus productivity that you’d expect (unless it backfires), but no one has quantified them.
    • Builds slack into the dev pipeline, such that emergencies can be handled without affecting customers.
    • Lets employees try out new teams before jumping ship entirely.
    • Builds cross-team connections that pay off in a number of ways, including testing new teams.
    • Gives developers a valve to overrule bug fixes and feature requests that their boss rejected from the official roadmap.
  • There are many things to do with 20% time besides new products.
    • Small internal tools, QOL improvements, etc (but see “costs”).
    • Learning, which can mean classes, playing with new tools, etc.
    • Decreasing technical debt.
    • Non-technical projects, e.g. charity drives.
  • Other notes:
    • One person suggested 20% time worked better at Google when it hired dramatically overqualified weirdos to work on mundane tech, and as they started hiring people more suited to the task with less burning desire to be working on something else, utilization and results decreased. 
    • 20% or even 120% time has outsized returns for industries that have very high capital costs but minimal marginal costs, such that employees couldn’t do them at home. This was a big deal at 3M (a chemical company) and, for the right kind of nerd, big data.

Thanks to the anonymous client for commissioning this research and allowing me to share it, and my Patreon patrons for funding my writing it up for public consumption.

Vavilov Day Starts Tomorrow

Content note: discussion of fasting.

Three weeks ago, I announced a plan to fast from the 25th to the 27th, in honor of Nikolai Vavilov and the staff of his botany institute, several of whom starved to death in the service of ending famine (and were partially successful, although far from the sole contributors). The goal was to test/improve my own ability to do hard things in the service of worthy projects. 

I had wanted to put much more research in the original post than I did, but decided it was more important to get the announcement out quickly and I should save something for the day-of post anyway. Since then, a lot has happened. Over three weeks I had 3 or 4 urgent demands around the size of “my furnace is maybe poison and my landlord is being difficult about it”. Everything is fine now, but it was a lot of effort to get it that way. I also had some emergency work drop in my lap for an extremely worthy project. I’m glad I got the opportunity to contribute and I’d make the same decision again but it ate up all of the slack I had left. And then my cell phone broke.

The immediate impact of this is there’s I’m not writing the highly researched post on Vavilov I wanted to. The internet is full of articles of the quality I could produce in the time I have available, there’s no reason to add to them.

But the more important impact is that I said I wanted to test my ability to do hard things, and then I did that, before the fast even started. My capacity was not as high as I wanted but more than I feared, and my capacity to respond to my limits gracefully instead of failing explosively exceeded my hopes.    

So in a lot of ways the purpose of the fast has already been served. I thought about letting myself out of it, but there are a few dimensions this month hasn’t tested and I still want to play with those. However in light of the fact that I am starting from a place of much lower slack and much higher time value than anticipated, I will be removing some of the rules, such as “I have to work a normal workday” and “I have to do at least one physical activity”. Those rules were for someone who didn’t expend all her reserves doing intense cognitive work on no notice while angry people made horrible noises banging on her furnace for three days straight. As of writing this (Monday night) I haven’t made up my mind on relaxing the calorie restriction to allow for ketone esters, which for me are a small source of calories that greatly reduce the cognitive and emotional costs of fasting. 

Tomorrow (the 26th) is the 69th anniversary of Nikolai Vavilov’s death. The day after is the 68th anniversary of the end of the siege of Leningrad, which meant the institute staff no longer needed to starve themselves to protect their seed bank. I will be fasting from 10PM tonight (the 25th) to 10AM on the 27th, but no promises on doing more than that. And if that high-value project needs more no-notice immediate-turnaround work from me and the ketone esters aren’t enough, I don’t even promise to keep fasting. Because this was never about pain for pain’s sake, it was about testing and increasing my ability to follow through on my own principles, and one of those principles is “don’t pointlessly incapacitate yourself when high impact time-sensitive work is waiting”.  

“…it was hard to wake up, it was hard to get on your feet and put on your clothes in the morning, but no, it was not hard to protect the seeds once you had your wits about you. Saving those seeds for future generations and helping the world recover after war was more important than a single person’s comfort.”

unknown Vavilov Institute scientist

Nudging My Way Out Of The Intellectual Mosh Pit

As part of my ongoing effort to improve my cost/benefit ratio on social media, I’m nudging myself away from intellectual mosh pit platforms like Facebook and Twitter, and towards blog posts, articles, videos essays, etc. Really longform consumption (e.g. books) remains about the same, my limitations on that are mostly my insistence on fact-checking nonfiction and very narrow tastes in fiction, this post is about changing what I reach for when I’m bored in a line. Towards that goal I have made a few changes, which I list here roughly in ascending order of how much effort they were:

  • Put all of my screens in greyscale. If you only try one thing on this list, make it this one. It takes five seconds to test once you have instructions, and the relief for me was immediate and has lasted days so far. Every time I have to turn color on to look at graphs I resent it. 
  • Discovered the existence of Focus Mode for android, which allows you to use the internet but disables notifications.
    • All of the Focus Mode instructions require several clicks into a deep menu. You can access it more easily by enabling the relevant fast-access card, using the instructions for older Android phones above. 
  • Moved all my short-OODA-loop apps off my phone home in favor of long-OODA-loop apps.
    • The newly defavored apps include obvious candidates like Twitter, but also all messaging apps and Chrome itself.
    • The new stars include my article aggregator, as well as very long-form content like Kindle, Audible, and PodCatcher.
    • The home screen continues to hold non-content-consumption apps I want to access quickly, like Maps, Calendar, etc. 
  • Resumed use of a read-it-later tool, which lets me save cool articles I see on social media to be consumed when I’m in a better head space (I can’t switch between reading microblogs and regular blogs quickly – different headspace ).
  • Switched to an RSS reader that lets me read things out of order without marking earlier articles as Read.
  • Combined my RSS feeds, email newsletters, and saved articles in a single service (that lets me read in any order) so I can choose from all my essay-length options at once. This was a stupid amount of effort and yak shaving and it’s not pretty but I got it working. Most of this post will be about that.

How to Combine All Article-Length Content

The following instructions work with both Inoreader and Feedly. I eventually chose Inoreader but it was a close call and your mileage may vary.

  1. Import your existing RSS feeds to Inoreader.
    1. Feedly instructions.
  2. You have two options for saving arbitrary content to your new aggregator. 
    1. Inoreader has a built-in feature to do this with a Chrome plugin, but there’s no way to see saved articles and RSS articles in the same list. So if that’s important for you…
    2. Sign up for Pocket, the leading read-it-later app, and configure Pocket to put your saved articles on an RSS Feed, then add that RSS feed to Inoreader per normal.
      1. Install the save to Pocket extension in your browser to make it easy to add to the feed as you come across things you want to read at some point (note: not available for Android, so I have to manually copy the URL and open the Pocket app).
        1. All articles in the saved-to-Pocket feed will show in Inoreader as having the same author (“my content feed”) and they won’t have the body of the article, only the title and header image. I really care about having all of my articles in the exact same place, but if you don’t, just using save-to-Inoreader will save you several steps.
  3. You also have two options for newsletters.
    1. Set up forwarding using Inoreader’s built-in system.
      1. This is a Pro feature, so it’s $6/month. 
        1. Feedly version ($12/month)
      2. You will either need to set up a new email for every subscription (which Inoreader supports, although you’ll need to upgrade past 20 subscriptions) or they will all be listed as having the same author.
    2. Or use a newsletter-to-RSS conversion tool like kill-the-newsletter.com, and add the resulting RSS feed to Feedly. Unless substack lets you configure separate emails for each subscription this will still require you to set up forwarding.

Tada! With a mere 20 minutes of work and a small monthly fee you have a system that combines all of your article-length-content in one place. Inoreader and Feedly both support Youtube channels and podcasts as well, although I haven’t tested those out.

Rejected Options

Pocket

Pocket bafflingly doesn’t support RSS input. You can hack it with IFTTT or Zapier, but each RSS feed counts as its own applet so you reach the $10/month plan very quickly. This is slightly cheaper than the Feedly pro plan but more than Inoreader, plus paying Inoreader gets rid of ads.

Pocket allows you to forward individual emails to it, but because Gmail requires forwarding address confirmation you can’t automate forwarding to pocket from Gmail. You could fix this with a newsletter-to-RSS converter and then IFTTT/Zapier, but that’s a lot of work.

Feedly

I originally settled on Feedly before making one final sweep and choosing Inoreader instead. The driving concern was that Feedly required me to be on tier 3 ($12) rather than Inoreader’s tier 2 ($6). Inoreader’s browser plugin was also better, letting you subscribe while on a blog’s homepage, where Feedly requires you to not only go to its own page to add feeds, but track down the actual feed URL rather than figuring it out from the blog’s homepage URL (which is surprisingly hard because RSS is out of favor and most readers can guess, so the RSS feed is rarely displayed prominently).

I do like the Feedly Android app a little more; Inoreader has not adjusted to Pixel’s lack of a back button and eats the replacement gesture, but I liked the webapp more so stuck with Inoreader. 

Feedly Pro+ (required to get newsletter forwarding) boasts an AI assistant I assume is terrible. Inoreader has easily accessible filters and prioritization rules that I haven’t tested. Between the two of these I predict I get more value out of Inoreader, but I could be wrong.

Lastly, Feedly promised me a trial and immediately charged me for an annual subscription, so they can die in a fire. 

Everyone Else

This is a spreadsheet where I went through every read-it-later service I could find, looking for RSS native support. None of them had it. It is possible there is another RSS reader with better bookmarking or newsletter support, but I am exhausted and Inoreader is working so I stopped looking.