Month: August 2019

Epistemic Spot Check: The Fate of Rome (Kyle Harper)

Introduction

Epistemic spot checks are a series in which I select claims from the first few chapters of a book and investigate them for accuracy, to determine if a book is worth my time. This month’s subject is The Fate of Rome, by Kyle Harper, which advocates for the view that Rome was done in by climate change and infectious diseases (which were exacerbated by climate change).

This check is a little different than the others, because it arose from a collaboration with some folks in the forecasting space. Instead of just reading and evaluating claims myself, I took claims from the book and made them into questions on a prediction market, for which several people made predictions of what my answer would be before I gave it. In some but not all cases I read their justifications (although not numeric estimates) before making my final judgement.

I expect we’ll publish a post-mortem on that entire process at some point, but for now I just want to publish the actual spot check. Because of the forecasting crossover, this spot check will differ from those that came before in the following ways:

  1. Claims are formatted as questions answerable with a probability. If a claim lacks a question mark, the implicit question is “what is the probability this is true?”.
  2. Questions have a range of specificity, to allow us to test what kind of ambiguities we can get away with (answer: less than I used).
  3. Some of my answers include research from the forecasters, not just my own.
  4. Due to timing issues, I finished the book and a second on the topic before I did the research for spot check.
  5. Due to our procedure for choosing questions, I didn’t investigate all the claims I would have liked to.

 

Claims

Original Claim: “Very little of Roman wealth was due to new technological discoveries, as opposed to diffusion of existing tech to new places, capital accumulation, and trade.”
Question: What percentage of Rome’s gains came from technological gains, as opposed to diffusion of technical advantages, capital accumulation, and trade?

1%-30% log distribution

Data:

  • The Fall of Rome talks extensively about how trade degraded when the Romans left and how that lowered the standard of living.
  • https://brilliantmaps.com/roman-empire-gdp/ shows huge differences in GDP by region, implying there was a big opportunity to grow GDP through trade and diffusion of existing tech. That means potential growth just from catch up growth was > 50%.
  • Wikipedia doesn’t even show growth in GDP per capita (with extremely wide error bars) from 14AD to 150AD.
  • Rome did have construction and military tech (https://en.wikipedia.org/wiki/Roman_technology)
  • It also seems likely that expansion created a kind of Dutch disease, in which capable, ambitious people were drawn to fighting and/or politics, and not discovering new tech.
  • One potential place where Roman technology could have contributed greatly to the economy was lowering disease via sanitation infrastructure. According to Fate of Rome and my own research, this didn’t happen; sanitation was not end to end and therefor you had all the problems inherent in city living.

Original Claim: “The blunt force of infectious disease was, by far, the overwhelming determinant of a mortality regime that weighed heavily on Roman demography”
Question: Even during the Republic and successful periods of the empire, disease burden was very high in cities.

60%-90% normal distribution

The wide spread and lack of inclusion of 100% in the confidence interval stem from the lack of precision in the question. What distinguishes “high” from “very high”, and are we counting diseases of malnutrition or just infectious ones? I expected to knock this one out in two minutes, but ended up feeling the current estimates of disease mortality lack the necessary precision to answer it.

Data:

 

Original Claim: “The main source of population growth in the Roman Empire was not a decline in mortality but, rather, elevated levels of fertility”
Question: When Imperial Rome’s population was growing, it was due to a decline in death rates, rather than elevated fertility.

80-100%, c – log distribution

“Elizabeth, that rephrase doesn’t look much like that original claim” you might be saying quietly to yourself. You are correct- I misread the claim in the book, at least twice, and didn’t catch it until this write-up. This isn’t as bad as it seems. The claims are not quite opposite, because my rephrase was trying to explain variation in growth within Rome, and the book was trying to explain absolute levels, or possibly the difference relative to today.

Back when he was doing biology, Richard Dawkins had a great answer to the common question “how much is X due to genetics, as opposed to environment?”. He said asking that is like asking how much of a rectangle’s area is due to its length, as opposed to its width. It’s a nonsensical question. But you assign proportionate responsibility for the change in area between two rectangles.

Fate‘s original claim was much like asking how much of a trait is due to genetics. This is bad and it should feel bad, but it’s a very common mistake, and I give Fate a lot of credit for providing the underlying facts such that I could translate it into the “what causes differences between things” question without even noticing.

Since weak framing wasn’t a systemic problem in the book and it presented the underlying facts well enough for me to form my own, correct, model, I’m not docking Fate very harshly on this one.

Original Claim: “The size of Roman merchant ships was not exceeded until the 15th century, and the grain ships were not surpassed until the 19th.”
Question: “The size of Roman merchant ships was not exceeded until the 15th century, and the grain ships were not surpassed until the 19th.”

0-10% log distribution.

This is true within the Mediterranean, but if  you check Chinese ships it’s obvious it’s off by at least 100 years, possibly more.

Original Claim: too diffuse to quote.
Question: The Roman Empire suffered greatly from intense epidemics, more so than did the Republic or 700-1000 AD Europe.

90-100% c – log distribution

https://en.wikipedia.org/wiki/List_of_epidemics shows a pretty clear presence of epidemics in the relevant period and absence in the others.

 

Original Claim: too diffuse to quote.
Question: Starvation was not a big concern in Imperial Rome’s prime.

80-100% c – log distribution

https://en.wikipedia.org/wiki/List_of_famines shows Roman famine in 441 BC (the Republic) and isolated famines from 370 on, but pretty much validates that during the prime empire, mass starvation was not a threat.

Conclusion:

My fact checking found two flaws:

  1. An inaccuracy in when ships that exceeded the size of Roman trade ships were built, and/or forgetting China was a thing. The inaccuracy does not invalidate the author’s point, which is that the Romans had better shipping technology than the cultures that followed them.
  2. Bad but extremely common framing for the relative effects of disease mortality vs. birth rates.

These is well within tolerances for things a book might get wrong. I’m happy I read this book, and would read another by the same author (with perhaps more care when it refers to happenings outside of Europe), but they are not jumping to the of my list.

Is The Fate of Rome correct in its thesis that Rome was brought down by climate change and disease? I don’t know. It certainly seems plausible, but is clearly advocating for a position rather than trying to present all the relevant facts. There are obvious political implications to Fate even if it doesn’t spell them out, so I would want to read at least one of the 80 million other books on the Fall of Rome before I developed an opinion. I’m told some people think it had to do with something military, which Fate barely deigns to mention. In the future I hope to be a good enough prediction-maker to put a range on this anyways, however wide it must be, but for now I’m succumbing to the siren song of “but you could just get more data”.

[Many thanks to my Patreon patrons and Parallel Forecast for financial support for this post]

PS. This book is the first step of an ongoing experiment with epistemic spot checks and prediction markets. If you would like to participate in or support these experiments, please e-mail me at elizabeth-at-this-domain-name. The next round is planned to start Saturday August 24th.

Epistemic Spot Check Scaling Experiment

Tl;dr: would you like to exchange money for an extra-rigorous epistemic spot check or automating me out of a hobby? I have an opportunity for you.

Most of you reading this probably know the epistemic spot check series on this blog, in which I somewhat-arbitrarily check claims early in a book to calibrate my trust level in said book.

I’ve been approached by a pre-public prediction market org to see if we can scale ESCs using a forecasting tournament. As conceived of right now, I would extract claims from a book and put them in the tournament, where anyone could bet on how I would eventually rule on the claim. I then check a subset of the claims (the others resolve as “ambiguous”) and money is distributed to the winners. In order to standardize things, this will be done with more rigor and consistency than is usually seen in epistemic spot checks.

We currently have prize money to distribute to the winners, but not to cover my time. We’re looking for $1,000-$2,000 depending on the book and any particular requests you have. If you’re feeling generous, more prize money would not hurt either.

If you’re at all interested, e-mail me at elizabeth – at – this- domain and we can chat.

Power Buys You Distance From The Crime

2077158526_98393523f5_o

Introduction

Taxes are typically meant to be proportional to money (or negative externalities, but that’s not what I’m focusing on). But one thing money buys you is flexibility, which can be used to avoid taxes. Because of this, taxes aimed at the wealthy tend to end up hitting the well-off-or-rich-but-not-truly-wealthy harder, and tax cuts aimed at the poor end up helping the middle class. Examples (feel free to stop reading these when you get the idea, this is just the analogy section of the essay):

  • Computer programmers typically have the option to work remotely in a low-tax state; teachers need to be where the classroom is. 
  • Estate taxes tend to hit families with single large assets (like a business) harder than those with diverse investments (who can simply sell assets to pay for taxes), who are hit harder than those with enough wealth to create trust funds.
  • Executives can choose to receive stock (which is taxed more favorably) instead of cash to the exact percentage they desire. Well paid employees are offered stock, but the amount will not be tailored to their needs. Lower level employees either are not offered this, or are not in a position to take advantage of it.
  • The legal distinction between a business (whose expenses are tax deductible) and a hobby (deductions not allowed) is based on whether the activity nets you income (there are complications and you can sometimes prove a money loser is a business, but this is a good rule of thumb). Small business owners (e.g. lawyers) can fold their occasionally-revenue-generating hobby (e.g. photography) into their real business, enabling tax deductions for their hobby.
  • IRAs, 401ks, HSAs, and FSAs all lock your money up for a time or purpose, in exchange for lower or delayed taxes. You can only take advantage of them if you’re sure you won’t need the money for another purpose sooner.
  • More examples here.

Note that most of these are perfectly legal and the rest are borderline. But we’re still not getting the result we want, of taxes being proportional to income.

When we assess moral blame for a situation, we typically want it to be roughly in proportion to much power a person has to change said situation. But just like money can be used to evade taxes, power can be used to avoid blame. This results in a distorted blame-distribution apparatus which assigns the least blame to the person most able to change the situation. Allow me a few examples to demonstrate this.

 

Examples 1 + 2: Corporate Malfeasance

Amazon.com provides a valuable service by letting any idiot sell a book, with minimal overhead. One of the costs of this complete lack of verification is that people will sell things that wouldn’t pass verification, such as counterfeits, at great cost to publishers and authors. Amazon could never sell counterfeits directly: they’re a large company that’s easy to sue. But by setting themselves up as a platform on which other people sell, they enable themselves to profit from counterfeits.

Or take slavery. No company goes “I’m going to go out and enslave people today” (especially not publicly), but not paying people is sometimes cheaper than paying them, so financial pressure will push towards slavery. Public pressure pushes in the opposite direction, so companies try not to visibly use slave labor. But they can’t control what their subcontractors do, and especially not what their subcontractors’ subcontractors’ subcontractors do, and sometimes this results in workers being unpaid and physically blocked from leaving.

Who’s at fault for the subcontractor(^3)’s slave labor? One obvious answer is “the person locking them in during the fire” or “the parent who gives their kid piecework”, and certainly it couldn’t happen without them. But if we say “Nike’s lack of knowledge makes them not responsible”, we give them an incentive to subcontract without asking follow up questions. The executive is probably benefiting more from the system of slave labor than the factory owner is from his little domain, and has more power to change what is happening. If the small factory owner pays fair wages, he gets outcompeted by a factory that does use slave labor. If the Nike CEO decides to insource their manufacturing to ensure fair working conditions, something actually changes.

…Unless consumers switch to a cheaper, slavery-driven shoe brand.

Which is actually really hard to not do. You could choose more expensive shoes, but the profit margin is still bigger if you shrink expenses, so that doesn’t help (which is why Fairtrade was a failure from the workers’ perspective). You can’t investigate the manufacturing conditions of everything you buy– it’s just too time consuming. But if you punish obvious enslavement and conduct no follow up studies, what you get is obscured enslavement, not decent working conditions.

 

Moral Mazes describes the general phenomenon on page 21:

Moreover, pushing down details relieves superiors of the burden of too much knowledge, particularly guilty knowledge. A superior will say to a subordinate, for instance: “Give me your best thinking on the problem with [X].” When the subordinate makes his report, he is often told: “I think you can do better than that,” until the subordinate has worked out all the details of the boss’s predetermined solution, without the boss being specifically aware of “all the eggs that have to be broken.” It is also not at all uncommon for very bald and extremely general edicts to emerge from on high. For example, “Sell the plant in [St. Louis]; let me know when you’ve struck a deal,” or “We need to get higher prices for [fabric X]; see what you can work out,” or “Tom, I want you to go down there and meet with those guys and make a deal and I don’t want you to come back until you’ve got one.” This pushing down of details has important consequences.

First, because they are unfamiliar with—indeed deliberately distance themselves from—entangling details, corporate higher echelons tend to expect successful results without messy complications. This is central to top executives’ well-known aversion to bad news and to the resulting tendency to kill the messenger who bears the news.

Second, the pushing down of details creates great pressure on middle managers not only to transmit good news but, precisely because they know the details, to act to protect their corporations, their bosses, and themselves in the process. They become the “point men” of a given strategy and the potential “fall guys” when things go wrong. From an organizational standpoint, overly conscientious managers are particularly useful at the middle levels of the structure. Upwardly mobile men and women, especially those from working-class origins who find themselves in higher status milieux, seem to have the requisite level of anxiety, and perhaps tightly controlled anger and hostility, that fuels an obsession with detail. Of course, such conscientiousness is not necessarily, and is certainly not systematically, rewarded; the real organizational premiums are placed on other, more flexible, behavior.

These examples differ in an important way from tax structuring: structuring requires seeking out advice and acting on it to achieve the goal. It’s highly agentic. The Wells Fargo and apparel-outsourcing cases required no such agency on the part of executives. They vaguely wished for something (more revenue, fewer expenses), and somehow it happened. An employee who tried to direct the executives’ attention to the fact that they were indirectly employing slaves would probably be fired before they ever reached the executives. Executives are not only outsourcing their dirty work, they’re outsourcing knowledge of their dirty work. 

[Details of personal anecdotes changed both intentionally and by the vagaries of human memory]

Example/Exception 2.5: Corporate Malfeasance Gone Wrong

The Wells Fargo account fraud scandal: in order to meet quotas, entry level Wells Fargo employees created millions of unauthorized accounts (typically extra services for existing customers). I originally included this as an example of “executives incentivizing entry level employees to commit fraud on their behalf”, but it turns out Wells Fargo made almost no money off the fraud- $2m over five years, which hardly seems worth the employees’ time, much less the $185m fine. I’ve left this in as an example of how the incentives-not-orders system doesn’t always work in powerful people’s favor.

Thanks to Larks for pointing this out.

Example 3: Foreign Medical Care

My cousin Angela broke her leg while traveling in Thailand, and was delighted by the level of care she received at the Thai hospital– not just medically, but socially. Nurses brought her flowers and were just generally nicer than their American counterparts. Her interpretation was that Thailand was a place motivated by love and kindness, not money, and Americans should aspire to this level of regard for their fellow human being. My interpretation was that she had enough money to buy the goodwill of everyone in the room without noticing, so what she should have learned is that being rich is awesome, and that being an American who travels internationally is enough to qualify you as rich.

This is mostly a success story for the free market: Angela got good medical care and the nurses got money (I’m assuming). Any crime in this story were committed off-screen. But Angela was certainly benefiting from the nurses’ restrained choices in life. And had she had actual power to affect healthcare in US, trying to fix it based on what she learned in Thailand would have done a lot of damage.

 

Example 4: My Dating an Artist Experience

My starving-artist ex-boyfriend, Connor, stayed with me for two months after a little bad luck and a lot of bad decisions cost him his job and then apartment (this was back when I had a two bedroom apartment to myself– I miss Seattle). During this time we had one big fight. My view on the fight now is that I was locally in the right but globally the disagreement was indicative of irreconcilable differences that should have led us to break up. That was delayed by months when he capitulated.

One possibility is that he genuinely thought he could change and that I was worth the attempt. Another is that he saw the incompatibility, or knew things that should have led him to see it, but lied or blocked out the knowledge so that he could keep living with me. This would be a shitty, manipulative thing for him to do. On the other hand, what did I expect? If the punishment for breaking up with me was, best case scenario, moving into a homeless shelter, of course he felt pressure to appease me. 

It wasn’t my fault he felt that pressure, any more than it was Angela’s fault her nurses were born with fewer options than her. Time in my spare bedroom was a gift to him I had no obligation to keep giving. But if I’d really valued a coercion free decision, I would have committed to housing him independent of our relationship. Although if that becomes common knowledge, it just means people can’t make an uncoerced decision to date me at all. And if helping Connor at all meant a commitment to do so forever, he would get a lot less help.

This case is more like the Wells Fargo case than Amazon or Nike. I was getting only the appearance of what I wanted (a genuine relationship with a compatible person), not the real thing. Nonetheless, the universe was contorting itself to give me the appearance of what I wanted.

Summary

What all of these stories have in common is that (relatively) powerful people’s desires were met by people less powerful than them, without them having to take responsibility for the action or sometimes even the desire. Society conspired to give them what they wanted (or in the case of Connor and Wells Fargo, a facsimile of what they wanted) without them having to articulate the want, even to themselves. That’s what power means: ability to make the game come out like you want. Disempowered people are forced to consciously notice things (e.g., this budget is unreachable) and make plans (e.g., slavery) where a powerful person wouldn’t. And it’s unfair to judge them for doing so while ignoring the morality of the powerful who never consider the system that brings them such nice things. 

Take home message:

  1. The most agentic person in a situation is not necessarily most morally culpable. One of the things power buys you is distance from the crime.
  2. Power obscures information flow. If you are not proactively looking to see how your wants and needs are being met, you are probably benefiting from something immoral or being tricked.

 

This piece was inspired by a conversation with and benefited from comments by Ben Hoffman. I’d also like to thank several commenters on Facebook for comments on an earlier draft and Justis Mills for copyediting.