Introduction

Two months ago I did an epistemic spot check on Kyle Harper’s The Fate of Rome. At the time I found only a minor flaw- stating that Roman ships weren’t surpassed until the 14th century, when China did it in the 13th century. I did not consider this fatal by any means.

Recently I decided to reread The Fate of Rome (affiliate link). This was driven by a few things. Primarily, I found myself resistant to reading more Roman history, which typically means I’m holding things in my short-term memory and will not be allowed to put new things into my brain until the existing things have been put in long term storage. But it did not hurt at all that I had just gotten access to a new exobrain, Roam, a workflowy/wiki hybrid, and yes, for purposes of this post that is an extremely unfortunate name.

This post is going to wear many hats: a second check of The Fate of Rome, a log of my work improving the epistemic spot check process, and a discussion of how Roam has affected my work. These will not be equally interesting to all people but I couldn’t write it any other way. That said, let us begin.

Process

Previously, I’d “taken notes” by highlighting passages and occasionally writing notes in the Kindle file, and then never reading them because Amazon’s anti-consumer choices made them a pain to access. Worse, I used highlights as an excuse not to take information into my brain- it was a pointer to process something later, not a reminder of something I had already processed.

When I took notes in Roam, I took notes. My initial workflow was to create a page for the book I was reading, and on it list claims from the book, each of which got their own page (I would eventually change that and leave them as bullet points on the source page). You can see the eventual result here: typically I recorded multiple claims per source-page, mostly rephrased into my own words, and always thought through instead of saved for thinking about later. (For comparison: notes from Fall of Rome round 1).

A few changes started about this time:

• I stopped being able to read without taking notes on my laptop, meaning I could no longer use my Kindle. I don’t think I got worse at reading on Kindle, it just became obvious how bad that always was.
• Despite having to use a multi-purpose device, I was more focused and harder to distract, probably by an order of magnitude.
• I couldn’t work on the project passed ~9PM. I don’t think I was ever doing my best work past 9, it just became obvious in contrast to the better work I could now do.
• I wanted to put a timestamp on every claim, so I noticed when it was unclear what time period a statement referred to.
• “How do we know that?” questions moved from something I pushed myself to think about during second read-throughs to popping into my head unbidden. There were just natural “How do we know that?” shaped holes in my notes.
• It became much more obvious when a bunch of paragraphs said nothing, or said nothing I valued, because even when I tried I couldn’t distill them into my notes.
• Reading books felt like play in a way it never had before, even though it was always something I enjoyed doing.
• I got more proactive about housecleaning. No, I wasn’t using Roam as a GTD system, it was purely research notes. And yet, I had more activation energy and more willingness to do multi-step chores. I have logs from Toggl to demonstrate this correlation, if not causation. Even assuming it’s causal I’d be shocked if it were common, so you probably shouldn’t incorporate it into your expected value of trying Roam.

At this stage the workflow is nothing I couldn’t have done in google docs, but I didn’t. I have all kinds of justifications about how knowing what I could do with Roam changed how I approached the work, but when I started that was theoretical so I’m not confident that’s what was going on. Nonetheless, I did it in Roam where I didn’t in Docs. 

So I had a Source page and a bunch of Claim pages. I started to do what I used to do in google docs or even a wordpress draft: select a claim and look for things confirming or denying it. This meant putting evidence on the Claims pages. But that didn’t feel right- why should some sources get their own page when others sat on the pages of claims from other sources? So I let claims motivate my choice of sources to look up, but every source got its own page with its claims listed on it. When I felt I knew enough I would create a Synthesis page representing what I really thought, with links to all the relevant claims (Roam lets you link to bullet points, not just pages) and a slider bar stating how firmly I believed it. This supported something I already wanted conceptually, which was shifting from [evaluating claims for truth and then judging the trustworthiness of the book] to [collating data from multiple sources of unknown reliability to inform my opinion of the world]. When this happened it became obvious Claims didn’t need their own pages and could live happily as bullet points on their associated Source page.

Once I had a Synthesis I would back-propagate a Credence to the claim that inspired the thread. Ideally I would have back propagated to all relevant claims, but that was more effort than it was worth. I put credences right in the claim so they would automatically show up when linked to, giving me a quick visual on how credible the book’s claims were when I investigated. The visual isn’t perfect because claims can have wildly different weights, but it is a start.

[Due to a bug, slider bars can be changed even by people given only read-access, so I also put the Credence in text]

Results

It turns out that The Fate of Rome was a near-ideal book about which to start asking “how do we know this?” (or maybe I’ll do more books and find out it’s average, but it definitely rewarded the behavior), because it is working with cutting edge science to prove its points, meaning it’s doing a lot of interpretation.

The Fate of Rome makes two big claims: Rome’s peak coincides with a period of unusually favorable and stable weather in the Mediterranean (from 200 BC to 150 AD), and Rome was a constant disease fest punctuated by peaks of even more illness.  What I would like to do right now is link you to my Fate of Rome Roam page, tell you to look at the links at the bottom, filter for Synthesis, and just browse through my work. It’s better prepared than I could ever do linearly, and lets you choose which parts are important to you. But I suspect there’s a learning curve to Roam so I will write things out the tedious linear way.

The Fate of Rome lists many sources of data on ancient climate. Here is a list of what I consider the 5 strongest, and the time period they supposedly applied. If you were reading this on Roam, you would have page numbers so you could verify my interpretation:

• Cosmogenic radionuclides in ice cores say that 360BC – 690 AD had unusually stable solar activity
• (Source unknown) says no major volcanic eruptions between “late republic” (end of the BCs) and “age of Justinian” (530s)
• Ratio of Oxygen18 to Oxygen16 in stalagmites points to warmth during “early Imperial Rome”
• The Tiber River flooded regularly (source unknown) during peak Imperial Rome
• Radiocarbon-dated sediments say the Dead Sea was at a peak from 200 BC to 200 AD

I have three complaints here: he doesn’t share the resolution of each method, two of the data points are unsourced (although one points to a paper where I could have looked it up), and these time periods don’t match up particularly well. For the first: I tried to find the resolution for ice cores at a depth of 2000 years, and was unable to come to a definitive answer, but I did find a suggestion that they’re extremely sensitive to the assumptions in your model, which makes me nervous. The third thing seems even more concerning: if anything it seems like the good times should have rolled through the collapse of the western empire, not ended at 150AD like Fate suggests. When you add in the innate political nature of any claims about changing climate, I’m inclined to view Fate’s climate claims as speculative, although not impossible. 

Another question Fate raises is the baseline health of the Romans. I think Fate is correct that it was terrible, and that’s an update for me. Turns out communal baths are not a source of hygiene before chlorine. Harper claims the disease and parasite load was worse than the people on the same land before or after. I initially thought this seemed reasonable for “before” but unreasonable for “after”- medieval peasants had shockingly terrible diets and disease risks. But if anything the evidence supports the opposite of what I thought– you have to go pretty far back to find people much taller than the Romans, but height jumps just as the (western) empire falls. There are other explanations for this, around exactly which skeletons get found, but basically all the sources I found agreed that the Roman disease load was high.

I’m not without qualms though. A prime piece of evidence he uses to demonstrate a high disease load is dental caries (cavities) versus Linear Enamel Hypoplasia, a defect in the growth of a tooth. Medieval peasants had more caries than Romans but less LEH. Harper’s interpretation is that medieval peasants had worse diets than Romans (because the caries indicate high carb content) but less disease (LEH can be caused by both poor nutrition and disease, and a better diet is indicated by the lack of caries). Martin Bernstorff, a friendly medical student who I met on Roam Slack, helped me out on this one. Based on a half hour of his research, an equally plausible explanation is that medieval peasants had the same disease load but more calcium. This doesn’t mean Rome wasn’t terrible- medieval European peasants had it shockingly bad. But it is not clear cut evidence of Rome being worse.

A sub-claim is that the Antonine Plague (165AD-180AD) was caused by Smallpox. Harper is careful to say that retrospective diagnosis is difficult without biochemical evidence and there’s not actually a lot riding on this conclusion: he’s not doing epidemiological modeling dependent on properties of smallpox in particular, for example. But he does sound very confident, and I wanted to see if that was justified. Martin took a look at this one too, and concluded there was a 95% chance Harper was correct, assuming the Roman doctor’s notes were accurate. The remaining 5% covers the chance of a related pox virus with a lower mortality rate.

Overall I still like The Fate of Rome, but I have much less trust in it than I did after my first spot check, when its only sin was briefly forgetting China existed. It its fight with The Fall of Rome, it has lost ground.

 

More Process

My first try at Fate took an unrecorded number of hours to read, and ~two hours to spot check (this is shorter than usual, because of the amplification experiment) Call it < 10 hours, not counting the time to write it up. This round took 17 hours of combined reading and investigation into claims  (plus 1.5 hours of Martin’s time), and so far three hours to write it up. This isn’t an apples-to-apples comparison, but that’s not *that much* additional time, for the increase in depth and understanding I got. I credit Roam with speeding things up enormously.

Since this is partially a love letter to Roam, I want to add a few things: 

• Over the years I’ve tried workflowy, calculist, and google docs. I did not go looking for other tools in this space and don’t intend to because I am Roam’s exact use case, so even if it’s not the best now I expect it grow towards me.
• It’s just into beta and it shows: I probably file a bug or feature request per day. It’s never anything that renders Roam unusable, just things take longer than they should. 
• Roam’s CEO, Conor White-Sullivan, has encouraged me to share my experience but has not given me anything for this post except a good product and the hope that it will continue to exist if enough people use it. 

 

As always, tremendous thanks to my Patreon patrons for their support. I would additionally like to thank Martin Bernstorff for his research (check out his new blog) and Edo Arad for comments on a draft.