Criticism as Entertainment

Media Reviews

There is a popular genre of video that consist of shitting on other people’s work without any generative content. Let me provide some examples.

First, Cinema Sins. This is the first video I selected when looking for a movie I’d seen with a Cinema Sins I hadn’t (i.e. it’s not random, but it wasn’t selected for being particularly good or bad).

The first ten sins are:

  1. Use of a consistent brand for props in the movie they’d have to have anyway, unobtrusively enough that I never noticed until Cinema Sins pointed it out.
  2. A character being mildly unreasonable to provoke exposition.
  3. The logo
  4. Exposition that wasn’t perfectly justified in-story
  5. A convenience about what was shown on screen
  6. A font choice (from an entity that in-universe would plausibly make bad font choices)
  7. An omission that will nag at you if you think about it long enough or expect the MCU to be a completely different thing, with some information about why it happened.
  8. In-character choices that would be concerning in the real world and I would argue are treated appropriately by the movie, although reasonable people could disagree
  9. Error by character that was extremely obviously intentional on the part of the film makers. There is no reasonable disagreement on this point.
  10. An error perfectly in keeping with what we know about the character.

Of those, three to four could even plausibly be called sins of the movie- and if those bother you, maybe the MCU is not for you. The rest are deliberate choices by filmmakers to have characters do imperfect things. Everyone gets to draw their own line on characters being dumb- mine is after this movie but before 90s sitcoms running on miscommunication- but that’s irrelevant to this post because Cinema Sins is not helping you determine where a particular movie is relative to your line. Every video makes the movie sound exactly as bad as the last, regardless of the quality of the underlying movie. It’s like they analyze the dialogue sentence by sentence and look to see if there’s anything that could be criticized about it.

Pitch Meeting is roughly as useful, but instead of reacting to sentences, it’s reading the plot summary in a sarcastic tone of voice.

Pitch Meeting is at least bringing up actual problems with Game of Thrones season 8. But I dare you to tell if early Game of Thrones was better or worse than season 8, based on the Pitch Meeting.

I keep harping on “You can’t judge movie quality by the review”, but I don’t actually think that’s the biggest problem. Or rather, it’s a subset of the problem, which is you don’t learn anything from the review: not whether the reviewer considered the movie “good” or not, and not what could be changed to do make it better. Contrast with Zero Punctuation, a video game review series notorious for being criticism-as-entertainment, that nonetheless occasionally likes things, and at least once per episode displays a deep understanding of the problems of a game and what might be done to fix it.

Why Are You Talking About This?

It’s really, really easy to make something look bad, and the short-term rewards to doing so are high. You never risk looking stupid or having to issue a correction. It’s easier to make criticism funny. You get to feel superior. Not to mention the sheer joy in punishing bad things. But it’s corrosive. I’ve already covered (harped on) how useless shitting-on videos are for learning or improvement, but it goes deeper than that. Going in with those intentions changes how you watch the movie. It makes flaws more salient and good parts less so. You become literally less able to enjoy or learn from the original work.

Maybe this isn’t universal, but for me there is definitely a trade off between “groking the author’s concepts” and “interrogating the author’s concepts and evidence”. Groking is a good word here: it mostly means understand, but includes playing with the idea and applying it what I know.  That’s very difficult to do while simultaneously looking for flaws.

Should it be, though? Generating testable hypotheses should lead to greater understanding and trust or less trust, depending on the correctness of the book. So at least one of my investigation or groking procedures are wrong.

 

What we Know vs. How we Know it?

Two weeks ago I said:

The other concept I’m playing with is that “what we know” is inextricable from “how we know it”. This is dangerously close to logical positivism, which I disagree with my limited understanding of. And yet it’s really improved my thinking when doing historical research.

I have some more clarify on what I meant now. Let’s say you’re considering my ex-roommate, person P, as a roommate, and ask me for information. I have a couple of options.

Scenario 1: I turn over chat logs and video recordings of my interactions with the P. 

E.g., recordings of P playing music loudly and chat logs showing I’d asked them to stop.

Trust required: that the evidence is representative and not an elaborate deep fake.

Scenario 2: I report representative examples of my interactions with P.

E.g., “On these dates P played music really loudly even when I asked them to stop.”

Trust required: that from scenario 1, plus that I’m not making up the examples.

Scenario 3: I report summaries of patterns with P

E.g., “P often played loud music, even when I asked them to stop”

Trust required: that from scenario 2, plus my ability to accurately infer and report patterns from data.

Scenario 4: I report what a third party told me

E.g. “Mark told me they played loud music a lot”

Trust required: that from scenario 3, plus my ability to evaluate other people’s evidence

Scenario 5: I give a flat “yes good” or “no bad” answer.

E.g., “P was a bad roommate.”

Trust required: that from scenario 3 and perhaps 4, plus that I have the same heuristics for roommate goodness that you do.

 

 

The earlier the scenario, the more you can draw your own conclusions and the less trust you need to have in me. Maybe you don’t care about loud music, and a flat yes/no would drive you away from a roommate that would be fine for you. Maybe I thought I was clear about asking for music to stop but my chat logs reveal I was merely hinting, and you are confident you’ll be able to ask more directly. The more specifics I give you, the better an assessment you’ll be able to make.

Here’s what this looks like applied to recent reading:

Scenario 5: Rome fell in the 500s AD.

Even if I trust your judgement, I have no idea why you think this or what it means to you.

Scenario 4: In Rome: The Book, Bob Loblaw says Rome Fell in the 500s AD.

At least I can look up why Bob thinks this.

Scenario 3: Pottery says Rome fell between 300 and 500 AD.

Useful to experts who already know the power of pottery, but leaves newbies lost.

Scenario 2: Here are 20 dig sites in England. Those dated before 323 (via METHOD) contain pottery made in Greece (which we can identify by METHOD), those after 500 AD show cruder pottery made locally.

Great. Now my questions are “Can pottery evidence give that much precision?” and “Are you interpreting it correctly?”

Scenario 1: Please enjoy this pile of 3 million pottery shards.

Too far, too far.

 

In this particular example (from The Fall of Rome), 2-3 was the sweet spot. It allowed me to learn as much as possible with a minimum of trust. But there’s definitely room in life for 4; you can’t prove everything in every paper and sometimes it’s more efficient to offload it.

I don’t view 5 as acceptable for anything that’s trying to claim to be evidenced based, or at least, any basis besides “Try this and see if it helps you.” (which is a perfectly fine basis if it’s cheap).

 

ESC Process Notes: Detail-Focused Books

When I started doing epistemic spot checks, I would pick focal claims and work to verify them. That meant finding other sources and skimming them as quickly as possible to get their judgement on the particular claim. This was not great for my overall learning, but it’s not even really good for claim evaluation: it flattens complexity and focuses me on claims with obvious binary answers that can be evaluated without context. It also privileges the hypothesis by focusing on “is this claim right?” rather than “what is the truth?”.

So I moved towards reading all of my sources deeply, even if my selection was inspired by a particular book’s particular claim. But this has its own problems.

In both The Oxford Handbook of Children and Childhood Education in the Ancient World and Children and Childhood in Roman Italy, my notes sometimes degenerate into “and then a bunch of specifics”. “Specifics” might mean a bunch of individual art pieces, or a list of books that subtly changed a field’s framing.  This happens because I’m not sure what’s important and get overwhelmed.

Knowledge of importance comes from having a model I’m trying to test. The model can be external to the focal book (either from me, or another book), or from it. E.g. I didn’t have a a particular frame on the evolution of states before starting Against the Grain, but James C. Scott is very clear on what he believes, so I can assess how relevant various facts he presents are to evaluating that claim.

[I’m not perfect at this- e.g., in The Unbound Prometheus, the author claims that Europeans were more rational than Asians, and that their lower birth rate was evidence of this. I went along with that at the time because of the frame I was in, but looking back, I think that even assuming Europe did have a lower birth rate, it wouldn’t have proved Europeans were more rational or scientifically minded. This is a post in itself.]

If I’d come into The Oxford Handbook of Children and Childhood Education in the Ancient World or Children and Childhood in Roman Italy with a hypothesis to test, it would have been obvious information was relevant and what wasn’t. But I didn’t, so it wasn’t, and that was very tiring.

The obvious answer is “just write down everything”, and I think that would work with certain books. In particular, it would work with books that could be rewritten in Workflowy: those with crisp points that can be encapsulated in a sentence or two and stored linearly or hierarchically. There’s a particular thing both books did that necessitated copying entire paragraphs because I couldn’t break it down into individual points.

Here’s an example from Oxford Handbook…

“Pietas was the term that encompassed the dutiful respect shown by the Romans towards their gods, the state, and members of their family (Cicero Nat. Deor. 1.116; Rep. 6.16; O . 2.46; Saller 1991: 146–51; 1998). is was a concept that children would have been socialized to understand and respect from a young age. Between parent and child pietas functioned as a form of reciprocal dutiful affection (Saller 1994: 102–53; Bradley 2000: 297–8; Evans Grubbs 2011), and this combination of “duty” and “affection” helps us to understand how the Roman elite viewed and expressed their relationship with their children.”

And from Children and Childhood…

“No doubt families often welcomed new babies and cherished their children, but Roman society was still struggling to establish itself even in the second century and many military, political, and economic problems preoccupied the thoughts and activities of adult Romans”

I summarized that second one as “Families were distracted by war and such up through 0000 BC”, which is losing a lot of nuance. It’s not impossible to break these paragraphs down into constituent thoughts, but it’s ugly and messy and would involve a lot of repetition. The first mixing up what pietas is with how and who it was expressed to. The second is combining a claim about the state of Rome with the state’s effects.

This reveals that calling the two books “lists of facts” was incomplete. Lists of facts would be easier to take notes on.  These authors clearly have some concepts they are trying to convey, but because they’re not cleanly encapsulated in the author’s own mind it’s hard for me to encapsulate them. It’s like trying to lay the threads of a gordian knot in an organized fashion.

So we have two problems: books which have laid out all their facts in a row but not connected them, and books which have entwined their facts too roughly for them to be disentangled. These feel very similar to me but when I write it out the descriptions sure sound like two completely different problems.

Let me know how much sense this makes, I can’t tell if I’ve written something terribly unpolished-but-deep or screamingly shallow.

Epistemic Spot Check: Children and Childhood Education in the Classical World

Introduction

Once upon a time I started fact checking books I read, to see if they were trustworthy. I called this epistemic spot checking because it was not a rigorous or thorough investigation; just following up on things I thought were interesting, likely to be wrong, or easy to check. Eventually I became dissatisfied with this. It placed too much emphasis on a binary decision about a particular book’s trustworthiness, and not enough on building models. So I started working on something better. Something that used multiple sources to build robust models of the world.

The Oxford Handbook of Children and Childhood Education in the Classical World (editors Judith Evan Grubbs and Tim Parkins) (affiliate link) is part of that attempt, but not a very big part, because it failed to be the kind of book I wanted. It was not as bad as Children and Childhood in Roman Italy at being just a bunch of facts with no organizing thesis, but it’s on that scale. And honestly it might be just as bad, I just find literature more interesting than visual arts. Like Children and Childhood… I’m going to write it up anyway, because learning from this kind of book is important.

Typically I read a book in order, but this was a collection of papers from different authors, so one chapter’s epistemics didn’t have much predictive value for the next and they didn’t build on each other the way a single-author book might.  I started with chapter 15 (Children and Childhood in Roman Commemorative Art) because I was checking Children and Childhood…and then chapter 13 (The Socialization of Roman Citizens) because it looked the most interesting.

You can see the entirety of my notes here.

Claims

Claim: Soranus advised swaddling is good for babies because it keeps them from rubbing their eyes (bad for eyesight) and leads to a healthy strong body (p290)
Verdict: Directly confirmed by a translation of Soranus’s Gynecology.

Claim: Seneca’s de Ira (On Anger) recommended:

  • Guiding young children to avoid high-anger personalities in adulthood
  • Not crushing children’s spirits
  • Not spoiling children

(p290)
Verdict: Directly confirmed by a translation of On Anger or as they call it, Of Anger.

Claim: Beryl Rawson called children the explicit aim of marriage in Rome.
Verdict: Yup, I remember that from last week.

Claim: Soranus, Laes, and Rawson all say a typical Roman birth would be witnessed by women from outside the home (p290-291)
Verdict: The report on Rawson is clearly true, and I believe she was quoting Soranus.

Claim: “Juvenal suggests that celebrations were held in the narrow streets outside dwellings ”
Verdict: True (see translation).

Claim:  “Although Cicero did not want to govern the province for an extended period of time, he would have stressed to [his son and nephew] that this was an important duty and that his own dedication to the task was an excellent example of his virtue and self-control (Cic. Att. 5.10.2–3, 5.14.2, 5.15.1).” (p296)
Verdict: Cicero’s letters make it abundantly clear he did not want to be there, but if the author has evidence of his motives for doing so, she doesn’t share it.

Claim: When Cicero went off to war he left his son and nephew with King Deiotarus of Galatia (p296).
Verdict: Confirmed in Cicero’s letters.

Claim: Most Roman girls experienced their first marriage in their mid-to-late teens (p298).
Verdict: Likely but not proven. You can see everything I’ve gathered on this question here. The summary is: the usual view was that Roman girls got married in early-to-mid teens, then someone went through and checked tombstones looking to see who died when and if they mentioned a surviving spouse, and found that Roman women married in their late teens (excellent summary of both sides). Tombstone demographics have their own issues so I don’t consider this proven, but it is suggestive.

ClaimIn 000s, the representation of children in art increased substantially, through the early 200s
Verdict: Rawson says the same thing, with some quibbling about dates.

Claim: The toga was a mark of Roman citizenship and forbidden to slaves. (p329)
Verdict: Confirmed by Wikipedia.

Claim: Quintus Sulpicius Maximus was an 11 year old boy who performed well in a poetry competition and got a nice funerary altar. (p336)
Verdict: Exact same data was in Children and Childhood in Ancient Rome (Beryl Rawson)

Claim: “Funerary reliefs as well as altars were most frequently commissioned by freedmen. To them a freeborn child, especially a son, was a mark of success. It was of particular importance to demonstrate the existence of a freeborn child, even one who had not lived to adulthood, and to show the family’s financial capacity to raise a memorial to a deceased child.” (p343)
Verdict: Likely but I haven’t seen a census. When I was reading Children and Childhood in Roman Italy, enough of the funeral art was about the freeborn children of ex-slaves that I noticed and wondered about it. But that could have been Rawson cherrypicking examples, or that freed couples chose more durable forms of art for their children than citizens.

 

Verdict

Like Children and Childhood in Roman Italy, The Oxford Handbook of Children and Childhood Education in the Classical World isn’t really interesting or ambitious enough to get things wrong. It is nonetheless useful as a repository of facts with which to check more ambitious books (which is in fact why I’m reading it) or generate your own theses.

 

 

 

 

ESC Process Notes: Claim Evaluation vs. Syntheses

Forgive me if some of this is repetitive, I can’t remember what I’ve written in which draft and what’s actually been published, much less tell what’s actually novel. Eventually there will be a polished master post describing my overall note taking method and leaving out most of how it was developed, but it also feels useful to discuss the journey.

When I started taking notes in Roam (a workflowy/wiki hybrid), I would:

  1. Create a page for the book (called a Source page), with some information like author and subject (example)
  2. Record every claim the book made on that Source page
  3. Tag each claim so it got its own page
  4. When I investigated a claim, gather evidence from various sources and list it on the claim page, grouped by source

This didn’t make sense though: why did some sources get their own page and some a bullet point on a claims page? Why did some claims get their own page and some not? What happened if a piece of evidence was useful in multiple claims?

Around this time I coincidentally had a call with Roam CEO Conor White-Sullivan to demo a bug I thought I had found. There was no bug, I had misremembered the intended behavior, but this meant that he saw my system and couldn’t hide his flinch. Aside from wrecking performance, there was no need to give each claim its own page: Roam has block references, so you can point to bullet points, not just pages.

When Conor said this, something clicked. I had already identified one of the problems with epistemic spot checks as being too binary, too focused on evaluating a particular claim or book than building knowledge. The original way of note taking was a continuation of that. What I should be doing was gathering multiple sources, taking notes on equal footing, and then combining them into an actual belief using references to the claims’ bullet points. I call that a Synthesis (example). Once I had an actual belief, I could assess the focal claim in context and give it a credence (a slider from 1-10), which could be used to inform my overall assessment of the book.

Sometimes there isn’t enough information to create a Synthesis, so something is left as a Question instead (example).

Once I’d proceduralized this a bit, it felt so natural and informative I assumed everyone else would find it similarly so.  Finally you didn’t have to take my word for what was important- you could see all the evidence I’d gathered and then click through to see the context on anything you thought deserved a closer look. Surely everyone will be overjoyed that I am providing this

Feedback was overwhelming that this was much worse, no one wanted to read my Roam DB, and I should keep presenting evidence linearly.

I refuse to accept that my old way is the best way of presenting evidence and conclusions about a book or a claim. It’s too linear and contextless. I do accept that “here’s my Roam have fun” is worse. Part of my current project is to identify a third way that shares the information I want to in a way that is actually readable.

Epistemic Spot Check: Children and Childhood in Ancient Rome (Beryl Rawson)

Introduction

Once upon a time I started fact checking books I read, to see if they were trustworthy. I called this epistemic spot checking because it was not a rigorous or thorough investigation; just following up on things I thought were interesting, likely to be wrong, or easy to check. Eventually I became dissatisfied with this. It placed too much emphasis on a binary decision about a particular book’s trustworthiness, and not enough on building models. So I started working on something better. Something that used multiple sources to build robust models of the world.

Children and Childhood in Roman Italy (affiliate link) was supposed to be a step in that process of learning how to extract and test models from books. Unfortunately it turned out to be poor soil. I can tell you what Children and Childhood is about, although not more than you are already capable of guessing, but ⅓ through it I can’t name a thesis. It’s just a collection of facts. So this is going to look a lot more like old epistemic spot checks than I’d hoped. I’m publishing anyway because it’s good practice writing, and because I’ve received grant money for this project and that carries with it an obligation to share as much of my work as is practical.

Claims

Claim: Quintus Sulpicius Maximus is a nice boy who placed beyond his years (11) at a poetry competition and got a very nice funerary altar depicting him as a little scholar
Verdict: Yup, way to not screw up some basic facts (read: The Oxford Handbook of Childhood and Education in the Classical World (my notes in Roam) (affiliate link) agrees with them).

Claim: Children first started appearing in (preserved) Roman art in the first century BC and steadily increased through 0200 AD
Verdict: Plausible, and a weird thing to get wrong, but not proven. I found a similar claim in The Oxford Handbook of Childhood and Education in the Classical World, which cites three sources: Currie 1996 (inaccessible), Rawson 2001 (that’s the same author as the focal book and thus not independent verification), Uzzi 2005 (inaccessible).

Claim: Representations of children picked up about this time because Augustus Caesar was trying to establish his descendants as rightful rulers after his death, and other people copied him.
Verdict: Yeah, sure seems plausible, but I don’t know how we would know that it was that as opposed to…

Claim: Augustus Caesar passed a number of laws (Lex Aelia Sentia and Lex Papia Poppaea) incentivizing marriage and children, most notably giving privileges to people who had sufficient children (3 for citizens, 4 for freedmen). Dead children counted, creating an incentive to have art commemorating your dead offspring.
Verdict: The legal claims are easily verified on wikipedia.

Or perhaps representation of children went up because…

Claim: Rome got significantly wealthier in the first century BC.
Verdict: Unknown and sensitive to definitions. I had the impression this was correct from reading other books and I expected to knock it out in five minutes, but I couldn’t actually find any data clearly laying out the case. The Greenland Ice Core data doesn’t line up

RomanLeadandAntimony graph

Although how it fails to line up depends on who you ask

and while I found some very cool graphs on construction in Rome, they were sourceless.

Or maybe representation of children went up because there were more freed slaves and…

Claim: Former slaves produced art of their citizen children at greatly increased rates to advertise upward mobility.
Verdict: Plausible, but still impossible to distinguish from other explanations. Funerary art of ex-slaves does seem to be overrepresented, but perhaps the upper classes produced just as much in a less durable form.

Claim: Funerary reliefs came into fashion in the first century BC, followed by altars in the first century AD and sarcophagi in the third.
Verdict: Confirmed by The Oxford Handbook of Childhood and Education in the Classical World, with some quibbling about dates.

Claim: In a typical Roman marriage, the man was 10 years older than the woman.
Verdict: Probably true. This was another one I expected to knock out in two minutes but was surprisingly difficult. The best source I found was this blog post which is a summary of one very old paper (1896) that did a demographic survey of graves, and two papers, one of which relies on that same paper and the other of which I didn’t check. It seems that the exact ages at which Roman men and women got married is in dispute, but the relative ages are agreed to be about 10 years apart.

Claim: Contraception and Abortion from the Ancient World to the Renaissance concludes, from a variety of evidence (literary, skeletal, comparative), that ‘classical peoples were somehow regulating their family sizes
Verdict: Accurate citation, but ignores the author’s skepticism of his own evidence.

Claim: “Scott (2000) has recently queried some of the assumptions in discus- sions of infanticide and has suggested that the evidence for alleged bias against females and the disabled in infant deaths is weak. ”
Verdict: Accurate citation.

 

Conclusion

Children and Childhood in Ancient Rome seems too boring to have made any major mistakes. If it gets overturned, I expect it to be because new evidence becomes available, which is a risk inherent to the topic.

If you’re interested in my process, you can see my notes in Roam here. Any claim with a number to its right

Screen Shot 2019-12-12 at 7.14.55 PM

is cited by another page, which you can get to by clicking on the number.  

Any claim with a slider bar has been investigated and assigned a credence. 

 

Many thanks to my Patreon Patrons and the Long Term Future Fund for financial support of this post.