How to Find Sources in an Unreliable World

I spent a long time stalling on this post because I was framing the problem as “how to choose a book (or paper. Whatever)?”. The point of my project is to be able to get to correct models even from bad starting places, and part of the reason for that goal is that assessing a work often requires the same skills/knowledge you were hoping to get from said work. You can’t identify a good book in a field until you’ve read several. But improving your starting place does save time, so I should talk about how to choose a starting place.

One difficulty is that this process is heavily adversarial. A lot of people want you to believe a particular thing, and a larger set don’t care what you believe as long as you find your truth via their amazon affiliate link (full disclosure: I use amazon affiliate links on this blog). The latter group fills me with anger and sadness; at least the people trying to convert you believe in something (maybe even the thing they’re trying to convince you of). The link farmers are just polluting the commons.

With those difficulties in mind, here are some heuristics for finding good starting places.

  • Search “best book TOPIC” on google
    • Most of what you find will be useless listicles. If you want to save time, ignore everything on a dedicated recommendation site that isn’t five books.
    • If you want to evaluate a list, look for a list author with deep models on both the problem they are trying to address, and why each book in particular helps educate on that problem.  Examples:
    • A bad list will typically have a topic rather than a question they are trying to answer, and will talk about why books they recommend are generically good, rather than how they address a particular issue. Quoting consumer reviews is an extremely bad sign and I’ve never seen it done without being content farming.
  • Search for your topic on Google Scholar
    • Look at highly cited papers. Even if they’re wrong, they’re probably important for understanding what else you read.
    • Look at what they cite or are cited by
    • Especially keep an eye out for review articles
  • Search for web forums on your topic (easy mode: just check reddit). Sometimes these will have intro guides with recommendations, sometimes they will have where-to-start posts, and sometimes you can ask them directly for recommendations. Examples:
  • Search Amazon for books on your topic. Check related books as well.
  • Ask your followers on social media. Better, announce what you are going to read and wait for people to tell you why you are wrong (appreciate it, Ian). Admittedly there’s a lot of prep work that goes into having friends/a following that makes this work, but it has a lot of other benefits so if it sounds fun to you I do recommend it. Example:
  • Ask an expert. If you already know an expert, great. If you don’t, this won’t necessarily save you any time, because you have to search for and assess the quality of the expert.
  • Follow interesting people on social media and squirrel away their recommendations as they make them, whether they’re relevant to your current projects or not.

Types of Knowledge

This is a system for sorting types of knowledge. There are many like it, but this one is mine.

First, there is knowledge you could regurgitate on a test. In any sane world this wouldn’t be called knowledge, but the school system sure looks enthusiastic about it, so I had to mention it. Examples:

  • Reciting the symptoms of childbed fever on command 
  • Reciting Newton’s first law of motion
  • Reciting a list of medications’ scientific and brand names
  • Reciting historical growth rate of the stock market
  • Reciting that acceleration due to gravity on Earth is 9.807 m/s²

 

Second, there is engineering knowledge- something you can repeat and get reasonably consistent results. It also lets you hill climb to local improvements. Examples:

  • Knowing how to wash your hands to prevent childbed fever and doing so
  • Driving without crashing
  • Making bread from a memorized recipe.
  • What are the average benefits and side effects from this antidepressant?
  • Knowing how much a mask will limit covid’s spread
  • Investing in index funds
  • Knowing that if you shoot a cannon ball of a certain weight at a certain speed, it will go X far.
  • Knowing people are nicer to me when I say “please” and “thank you”

 

Third, there is scientific knowledge. This is knowledge that lets you generate predictions for how a new thing will work, or how an old thing will work in a new environment, without any empirical knowledge.

Examples: 

  • Understanding germ theory of disease so you can take procedures that prevent gangrene and apply them to childbed fever.
  • Knowing the science of baking so you can create novel edible creations on your first try.
  • Knowing enough about engines and batteries to invent hybrid cars.
  • Actually understanding why any of those antidepressants works, in a mechanistic way, such that you can predict who they will and won’t work for.
  • A model of how covid is spread through aerosols, and how that is affected by properties of covid and the environment.
  • Having a model of economic change that allows you to make money off the stock market in excess of its growth rate, or know when to pull out of stocks and into crypto.
  • A model of gravity that lets you shoot a rocket into orbit on the first try.
  • A deep understanding of why certain people’s “please”s and “thank you”s get better results than others.

 

Engineering knowledge is a lot cheaper to get and maintain than scientific knowledge, and most of the time it works out. Maybe I pay more than I needed to for a car repair; I’ll live (although for some people the difference is very significant). You need scientific knowledge to do new things, which either means you’re trying something genuinely new, or you’re trying to maintain an existing system in a new environment.

I don’t know if you’ve noticed, but our environment was changing pretty rapidly before a highly contagious, somewhat deadly virus was released on the entire world, and while that had made things simpler in certain ways (such as my daily wardrobe), it has ultimately made it harder to maintain existing systems. This requires scientific knowledge to fix; engineering won’t cut it.

And it requires a lot of scientific knowledge at that- far more than I have time to generate. I could trust other people’s answers, but credentials and authority have never looked more useless, and identifying people I trust on any given subject is almost as time consuming as generating the answers myself.  And I don’t know what to do about that.

 

What to write down when you’re reading to learn

One of the hardest questions I’ve had to answer as part of the project formerly known as epistemic spot checks is: “how do I know what to write down?”

This will be kind of meandering, so here’s the take home. 

For shallow research:

  • Determine/discover what you care about before you start reading.
  • Write down anything relevant to that care.

For deep research:

  • Write down anything you find interesting.
  • Write down anything important to the work’s key argument.
  • Write down anything that’s taking up mental RAM, whether it seems related or interesting or not. If you find you’re doing this a lot, consider you might have a secret goal you don’t know about.
  • The less 1:1 the correspondence between your notes and the author’s words the better. Copy/pasting requires little to no engagement, alternate theories for the explanations spread over an entire chapter require a lot.

 

Now back to our regularly scheduled blog post.

Writing down a thing you’ve read (/heard/etc) improves your memory and understanding, at the cost of disrupting the flow of reading. Having written a thing down makes that one thing easier to rediscover, at the cost of making every other thing you have or will ever write down a little harder to find. Oh, and doing the math on this tradeoff while you’re reading is both really costly and requires knowing the future. 

I would like to give you a simple checklist for determining when to save a piece of information. Unfortunately I never developed one. There are obvious things like “is this interesting to me (for any reason)?” and “is this key to the author’s argument?”, but those never got rid of the nagging feeling that I was losing information I might find useful someday, and specifically that I was doing shallow research (which implies taking the author’s word for things) and not deep (which implies making my own models). 

The single most helpful thing in figuring out what to write down was noticing when my reading was slowing down, which typically meant either there was a particular fact that needed to be moved from short to long term storage, or that I needed to think about something. Things in these categories need to be written down and thought about regardless of their actual importance, because their perceived importance is eating up resources, and 30 seconds writing something down to regain those resources is a good trade even if I never use that information again. If I have one piece of advice, it’s “learn to recognize the subtle drag of something requiring your attention.”

An obvious question is “how do I do that though?”. I’m a mediocre person to answer this question because I didn’t set out to learn the skill, I just noticed I was doing it. But for things in this general class, the best thing I have found to do is get yourself in a state where you are very certain you have no drag (by doing a total brain dump), do some research, and pay attention to when drag develops. 

But of course it’s much better if my sense of “this is important, record it” corresponds with what is actually important. The real question here is “Important to what?” When I was doing book-based reviews, the answer at best was “the book’s thesis”, which as previously discussed gives the author a huge amount of power to control the narrative. But this became almost trivial when I switched the frame to answering a specific set of questions. As long as I had a very clear goal in mind, my subconscious would do most of the work. 

This isn’t a total solution though, because of the vast swath of territory labeled “getting oriented with what I don’t know”. For example right now I want to ask some specific questions about the Great Depression and what it can tell us about the upcoming economic crisis, but I don’t feel I know enough. It is very hard to get oriented with patchwork papers: you typically need books with cohesive narratives, and then to find other ways to undo the authors’ framing. Like a lot of things, this is solved by going meta. “I want to learn enough about the Great Depression that I have a framework to ask questions about parallels to the current crisis” was enough to let me evaluate different “Top Books about the Great Depression” lists and identify the one whose author was most in line with my goals (it was the one on fivebooks, which seems to be the case much more often than chance).

I mentioned “losing flow” as a cost of note taking in my opening, but I’m not actually convinced that’s a cost. Breaking flow also means breaking the author’s hold on you and thinking for yourself. I’ve noticed a pretty linear correlation between “how much does this break flow?” and “how much does this make me think for myself and draw novel conclusions?”. Copy/pasting an event that took place on a date doesn’t break flow but doesn’t inspire much thought. Writing down your questions about information that seems to be missing, or alternate interpretations of facts, takes a lot longer.

Which brings me to another point: for deep reading, copy pasting is almost always Doing It Wrong. Even simple paraphrasing requires more engagement than copy/pasting. Don’t cargo cult this though: there’s only so many ways to say simple facts, and grammar exercises don’t actually teach you anything about the subject.

So there is my very unsatisfying list of how to know what to write down when you’re reading to learn. I hope it helps.

Where to Start Research?

When I began what I called the knowledge bootstrapping project, my ultimate goal was “Learn how to learn a subject from scratch, without deference to credentialed authorities”. That was too large and unpredictable for a single grant, so when I applied to LTFF, my stated goal was “learn how to study a single book”, on the theory that books are the natural subcomponents of learning (discounting papers because they’re too small). This turned out to have a flawed assumption baked into it.

As will be described in a forthcoming post, the method I eventually landed upon involves starting with a question, not a book. If I start with a book and investigate the questions it brings up (you know, like I’ve been doing for the last 3-6 years), the book is controlling which questions get brought up. That’s a lot of power to give to something I have explicitly decided not to trust yet. 

Examples:

  • When reading The Unbound Prometheus, I took the book’s word that a lower European birth rate would prove Europeans were more rational than Asians and focused on determining whether Europe’s birth rates were in fact lower (answer: it’s complicated), when on reflection it’s not at all clear to me that lower birth rates are evidence of rationality.
  • “Do humans have exactly 4 hours of work per day in them?” is not actually a very useful question. What I really wanted to know is “when can I stop beating myself up for not working?“, and the answer to the former doesn’t really help me with the latter. Even if humans on average have 4 hours, that doesn’t mean I do, and of course it varies by circumstances and type of work… and even “when can I stop beating myself up?” has some pretty problematic assumptions built into it, such as “beating myself up will produce more work, which is good.” The real question is something like “how can I approach my day to get the most out of it?”, and the research I did on verifying a paper on average daily work capacity didn’t inform the real question one way or the other.

 

What would have been better is if I’d started with the actual question I wanted to answer, and then looked for books that had information bearing on that question (including indirectly, including very indirectly). This is what I’ve started doing.

This can look very different depending on what type of research I’m doing. When I started doing covid research, I generated a long list of  fairly shallow questions.  Most of these questions were designed to inform specific choices, like “when should I wear what kind of mask?” and “how paranoid should I be about people without current symptoms?”, but some of them were broader and designed to inform multiple more specific questions, such as “what is the basic science of coronavirus?”. These broader, more basic questions helped me judge the information I used to inform the more specific, actionable questions (e.g., I saw a claim that covid lasted forever in your body the same way HIV does, which I could immediately dismiss because I knew HIV inserted itself your DNA and coronaviruses never enter the nucleus).

 


 

I used to read a lot of nonfiction for leisure. Then I started doing epistemic spot checks– taking selected claims from a book and investigating them for truth value, to assess the book’s overall credibility- and stopped being able to read nonfiction without doing that, unless it was one of a very short list of authors who’d made it onto my trust list. I couldn’t take the risk that I was reading something false and would absorb it as if it were true (or true but unrepresentative, and absorb it as representative). My time spent reading nonfiction went way down.

About 9 months ago I started taking really rigorous notes when I read nonfiction. The gap in quality of learning between rigorous notes and my previous mediocre notes was about the same as the gap between doing an epistemic spot check and not. My time spent reading nonfiction went way up (in part because I was studying the process of doing so), but my volume of words read dropped precipitously.

And then three months ago I shifted from my unit of inquiry being “a book”, to being “a question”. I’m sure you can guess where this is going- I read fewer words, but gained more understanding per word, and especially more core (as opposed to shell or test) understanding. 

The first two shifts happened naturally, and while I missed reading nonfiction for fun and with less effort, I didn’t feel any pull towards the old way after I discovered the new way. Giving up book-centered reading has been hard. Especially after five weeks of frantic covid research, all I wanted to do was to be sat down and told what questions were important, and perhaps be walked through some plausible answers. I labeled this a desire to learn, but when I compared it to question-centered research, it became clear that’s not what it was. Or maybe it was a desire to go through the act of learning something, but it was not a desire to answer a question I had and was not prioritized by the importance of a question. It was best classified as leisure in the form of learning, not resolving a curiosity I had.  And if I wanted leisure, better to consume something easier and less likely to lead me astray, so I started reading more fiction, and the rare non-fiction of a type that did not risk polluting my pool of data. And honestly I’m not sure that’s so safe: humans are built to extract lessons from fiction too.

Put another way: I goal factored (figured out what I actually wanted from) reading a nonfiction book, and the goal was almost never best served by using a nonfiction book as a starting point. Investigating a question I cared about was almost always better for learning (even if it did eventually cash out in reading a book), and fiction was almost always better for leisure, in part because it was less tiring, and thus left more energy for question-centered learning when that was what I wanted.

 

Really Ridiculously Thorough Notes

Recently I tried an experiment. My note taking method already involves trying to record every single claim a book makes- I added to that “record every thought I have about the claim.” This included information that bore on the claim (e.g. if the claim was “A wave of German Catholics emigrated to American colonies from 1720-1741”, my thoughts would include “wait, wasn’t Germany Protestant?” and “Germany didn’t have any colonies of its own”) , questions it raised (e.g., “what was the state of German Protestant immigration to American colonies?”, “what constitutes a wave of immigration?”, “how did they fund the travel?”) and potential implications (“They would learn to need English”). Obviously this would be incredibly onerous to do all the time; my goal was to see what changes occurred when I did it, and perhaps train the muscle so it would be easier to do so in the future.

For a test subject I chose Children in Colonial America, of which I had skipped the last three chapters because they didn’t bear on my overall question that much. However they were a much better size and format than my the next two books in my queue, and I’d be able to get to the meat faster because I’d already read the previous chapters.

You can see my notes for the book as a whole here, the experiment starts with Chapter 10

Day 1, Round 1 (Chapter 10):

  • Not a perfect experiment; I’d taken Ritalin for the first time in a while before deciding to run the experiment and it obviously altered the experience a lot.
  • I got through a pre-read (basically a non-exhaustive reading of the first and last few paragraphs) and two pages in 1.5 hours.
  • After 1.5 hours I was done. Could not continue with the experiment for love or money. I went on to work on a blog post about cat-mitigation strategies for an hour, so it’s not that the Ritalin quit.
  • Even explicitly giving myself prompts to write down *everything* I thought related to a claim, I would sometimes notice new thoughts well after I’d left a particular claim.

Day 1, Round 2:

  • Tried for a bit but couldn’t muster the energy to go really into detail like I did above.

 

Day 2, Round 1 (Chapter 10):

  • More intense than D1R2 but less than D1R1, finished early when I finished a chapter.

 

Day 3 and 4, Round 1 (Chapter 11):

  • Started (day 3) and finished (day 4) Chapter 11 of Children in Colonial America. Either something has changed in my capacity to this work, or the work showed me something was wrong with the chapter, even though I can’t put my finger on it.

 

Day 5 (Chapter 12):

  • Became irritated in the pre-reading phase, spent 2 hours writing a blog post about why the final paragraph signaled low quality.

 

Day 6, Round 1 (Chapter 1):

  • Coincidentally experimented with caffeine + theanine + MCT oil in the morning.
  • Published complaints about Chapter 12.
  • I wanted to extend the experiment- both the deep note-taking, and predicting work quality in the pre-reading stage. I have some books out from the library, but they’re full books, not anthologies, and I feel like stand-alone chapters give me faster feedback.
  • Discovered that Children in Colonial America is book 3 in a series on children in America, and there are three other books with the same editor on different time periods. This is great because it lets me minimize the changing variables as I continue the experiment.
  • Read Chapter 1 and deep note take Children and Youth in a New Nation (notes). I’m not able able to go quite as deep as in attempt 1, but then, Ritalin. Chapter 1 of CaYiaNN is one of those middling history works that doesn’t have an overarching thesis but knows it is structurally expected to have one, so makes a thesis out of its uncertainty: “Some people had a variety of experiences with X for a variety of reasons.” Become inspired to write Fake Thesis Vs Absent Thesis.

 

Both my note-taking process and the notes I took on it gradually declined as I attempted to read Chapter 2 of Children and Youth in a New Nation, culminating in going days where I couldn’t even push myself into reading. So… can’t say I recommend this. I’m working out other ways to approach the goal of contextualizing information as I read.

 

 

 

 

 

Fake Thesis vs. Absent Thesis

Yesterday I complained about a stand-alone chapter whose opening and especially closing paragraphs immediately made me think it was low quality, which was correct. Today I want to talk about something that looks similar, but isn’t.

The chapter I complained about was the last chapter in Children in Colonial America (affiliate link), but it turns out CiCA is the third book in a series on the history of childhood in America, all in the same format with the same editor. This lets me minimize variables as I compare chapters. Then as luck would have it, the first chapter of the first-published book, Children and Youth in the New Republic (affiliate link), was the perfect foil for Chapter 12 of Children in Colonial America.

Chapter 1 of …New Republic, “Boy Soldiers of the American Revolution: The Effects of War on Society”, demonstrates a fairly common pattern. The author has a bunch of data and no single frame to capture it all, so they say something like “X is a complicated subject. Different people related to it in a variety of ways for a variety of reasons.” In the particular case of Boy Soldiers… X is “boys/young men fighting for America in the Revolutionary War”, and the variety is “Some boys chose to fight for patriotism, material advancement, or to help their family. Some but not all did this against their parents’ objections. Other boys were forced to fight by their families over their objections.”

Or in more words

The factors that drew these soldiers into the service indicate the great diversity of experience in boyhood in revolutionary America. In the glimpses we have of them and their families at the moment of enlistment, we see that some enjoyed the care and protection of their parents while others suffered at the hands of mercenary ones. Some went to war against their parents’ inclinations while others were thrust into it over their own objections. A few boys thought that the army would be a more hospitable place than the places they lived or that it would be an escape from jobs that were tedious and frustrating. For many more, military service was an issue around which they could negotiate with their fathers when they wanted opportunities that could take them away from home. Some saw a real opportunity to contribute to the financial well-being of their families. They could do this either directly by turning over their pay or bounty money or indirectly by substituting for an older family member, allowing the more needed laborer to stay home, or by relieving their families of the need to support and feed them. A few, such as Josiah Brandon, were drawn by the cause itself to set their own course.

Boyer, Paul S.. Children and Youth in a New Nation (Children and Youth in America) (pp. 26-27). NYU Press short. Kindle Edition.

 

The chapter consists of first hand accounts of different boys enlisting in the army for different reasons, and some comments on the state of the evidence.

I could easily see a world where the same amount of actual facts, models, and and narratives led to both the theses of the style of Chapter 1 of CaYiaNN and Chapter 12 of CiCA, depending on writing skill and adherence to a guide book but independent of the quality of information or author understanding. “Iunno” and “A bunch of things happened for a bunch of reasons” are both good descriptions of a pile of data you don’t have a cohesive explanation for. If anything I’d expect “Iunno” to be associated with higher quality works, since it’s more honest. That’s clearly not happening in these cases, although obviously a sample size of two is too small to draw any conclusions.

 

Set Ups and Summaries

Part of the research process I’m developing involves reading and thinking through the first and last chapter of a book, and first and last few paragraphs of a chapter, to get an expectation of what’s to come (combined with some other stuff I call this pre-reading). I’m currently pondering how much you can get out of this, and specifically if it’s fair to reject a work because it failed pre-reading.

Pre-reading is in part derived from advice in How to Read a Book to find a books “Unity”, the idea being that you’ll better incorporate information into your understanding if you know how it connects to the author’s larger point. I objected to HtRaB’s advice on this topic in my review, because it seemed to be trying to enforce an orderliness that reality does not support. Looking for a unified narrative encourages the author to throw out anything that doesn’t fit their narrative, and the reader to ignore it even when it’s included. Even in situations where there is a fairly clear narrative you might not know it yet, and it’s important to be able to share raw data without prematurely deciding what it means.

Then I pre-read chapter 12 of Children in Colonial America (an anthology: chapters have a common theme but each is meant to stand alone), and my immediate reaction was “nothing with this ending is going to be any good.”  The chapter is discussing specific individuals down to the last paragraph, with no attempt to summarize what’s come before. The start is better but not by much- the last sentence of the first paragraph should clearly be the first, and the contextualization the rest of the paragraph provides should have come after, not before, when I know why I care what percentage of 1770s Boston’s population was made up of children.

The paragraphs in question:

Boston, the American Revolution’s “cradle of liberty,” was a town full of children. As in British North America as a whole, over half the population was under the age of adulthood. Children participated in political actions as early as Boston’s first public protest against the Stamp Act, on August 14, 1765: an organizer described “two or three hundred little boys with a Flagg marching in a Procession on which was King, Pitt & Liberty for ever.”1 The first Bostonian to die in political violence was a young boy. Apprentices both brought on and suffered in the Massacre of 1770, and pushed their way into the Tea Party of 1773. How did those children interpret the political conflict, and what motivated many of them to participate?

Children in Colonial America (Children and Youth in America) (p. 204). NYU Press. Kindle Edition.

 

When war finally came, the little boys who marched against the Stamp Act in 1765 were of prime age to be soldiers. Some younger boys took war as a chance to assume adult freedoms. On April 19, thirteen-year-old Benjamin Russell and several friends left their Writing School and followed the redcoat reinforcements out of town, attaching themselves to the provincial camp by the end of the day. Teenagers enlisted with and without their parents’ consent. Thirteen-year-old Daniel Granger of Andover was so small that when he was singled out for praise, his captain “sat me down on his Knees.”37 These boys took on men’s roles in the fight for liberty, leaving the symbolic battles of childhood behind.

Children in Colonial America (Children and Youth in America) (pp. 213-214). NYU Press. Kindle Edition.

Those opening and closing paragraphs clearly fail at the goals of orienting and summarizing the work. But a lot of my posts are kind of crap at that too. I’m writing about ideas in their preliminary stages in a way that forces a lot of the work onto the reader. I’m doing it right now, although I have spiffed up the opening and closing to avoid embarrassment. Maybe this author is doing that.  It’s not like having a good introduction and summary are a guarantee of quality. Chapter 11 of the same book does a great job telling you what it’s going to tell you and what it’s told you, but I am pretty dissatisfied with it in ways I am even less able to articulate.

Back on the first hand, maybe factual chapters in professionally edited books should be help to a different standard than blog posts describing bursts of an in-progress project. Maybe my scattered opening and closing paragraphs should cause you to downgrade your assessment of these post (although if you could keep in mind what I’m capable of when I’m prioritizing idea transmission, that would be cool).

I don’t think books/chapters/blog posts should be held to a single unifying narrative. But facts and models are a lot more useful connected to other facts and models than they are in isolation. The author making no attempt to do so makes my job harder- perhaps impossibly so given how little I know about the chapter’s topic.

Yeah, that feels quite fair-  this chapter might be very useful to people more familiar with the field, but that doesn’t mean it’s very helpful to me, a non-expert trying to bootstrap her way up.

A thing I would normally praise Children chapter 12 for, and did praise other chapters of the same book for, is providing a lot of concrete examples to shore up general assertions (e.g. “A large number of the trans-Atlantic slave trade was children” was followed by references to demographic counts from a large number of ships). But the information doesn’t feel quite right. For example, when describing the youth of men who enlisted in the American army, the chapter uses an anecdote about a 13 year old sitting on his commander’s knee. That doesn’t address the paragraph’s stated concern of “how did the Revolutionary War affect teenage boys’ options?” and it’s also a really terrible way of assessing the prevalence of 13 year olds enlisting. That’s one of those questions best answered by counting. And neither really belongs in the final paragraph of a chapter, which chapter 3 knew and chapter 12 didn’t.

Ah, this is a thing: tallies of thousands of people across dozens of ships is not really comparable to an anecdote about one 13 year old. The anecdote just isn’t useful data, except maybe as a pointer to where to find more data. Anecdotes have their place, but the bare minimum to make a compilation of anecdotes useful is knowing how they were generated. Are they representative? Slanted towards some group or ideology?

I’ll look like a real ass here if I don’t have a summary, but I’m still not sure what I’ve learned. I still think How to Read a Book is wrong to insist every book have a clearly defined Unity. I think Children in Colonial America Chapter 12’s opening and especially closing paragraphs signal failure on some level, although I am not absolutely certain what signal I’m picking up on. I’ve spent longer writing this and skimming the chapter than it would have taken to read it deeply, but that’s okay because it was a better use of my time.

 

The Purpose of Lectures

How to Take Smart Notes (affiliate link) posits that students who handwrite lecture notes gain as many facts and more conceptual understanding than students who type notes to the same lecture, because the slowness of handwriting forces you to compress ideas down to their core, whereas typing lets you transcribe a lecture without reflection. While I agree that translating things in your own words and compressing ideas is better than rote transcription, I have two problems with this.

One, it preemptively gives up on a practical question of which side of a trade-off is better without examining either the conditions or ways to improve the trade off. Given the enormous benefits of electronic storage of notes, maybe we should spend 45 seconds thinking about how to port the benefits of handwritten notes over, or under what circumstances the benefits of quick and high-fidelity transcription outweighs the push to engage more deeply with data.

Two, and this is harder to articulate… there is a reason students are defaulting to transcriptions of lectures, and it’s not because they’re bad or lazy or don’t like thinking. If lecturers actually wanted you to think conceptually about a topic, they would, I don’t know, leave any time at all for that in a lecture (my STEM background may be showing here. Movies tell me English class has more of this). As it is, conceptual understanding and translation requires that you stop listening to the professor- the dreaded multitasking thing that luddites are always going on about.

This is really a college student issue. On the rare occasion I’m trying to learning something from a live lecture, it’s still a non-mandatory event where the speaker cares about either actually teaching something or being entertaining, which solves a lot of these problems. But I’m angry that blame is being placed on students for acquiescing to what the system very strongly pushes them towards.

 

Review: How to Read a Book (Mortimer Adler, Charles Van Doren)

As part of my research on how to bootstrap understanding in a field, I’m reading books that attempt to answer that question. You might think I should have started with that, but it was useful to get a sense of what problems I needed to solve before I looked for the solution. How to Read a Book (affiliate link) is generally very well regarded in this area and came with a strong recommendation from the CEO of Roam, who I would expect to have pretty good thoughts on learning structure. Nonetheless, I was quite disappointed. It took me a long time to put my disappointment into words, but with the help of someone on Facebook I finally figured it out: How to Read a Book is aimed at a narrower subset of books than it acknowledges. What subset, you might ask? I don’t have a great answer, because the authors clearly consider the subset to be the only books, or the only books worth reading, so they didn’t leave a lot of clues. 

What I can say is that it expects books to follow a rigid structure, and to have a single unifying point (what they call “the unity”). This seems to me to be setting up both the author and the reader to throw out a lot of information because they weren’t expecting to see it or couldn’t fit it into their existing frameworks- reading like a state, in essence. This is not the only thing that makes me think HtRaB is more about being able to understand a book than understand the world, although it’s the only one I can articulate.

How to Read a Book didn’t even attempt to answer my current most important question in reading: How do I know what to save or pay attention to? More attentive reading (including but not limited to note-taking) takes more time and more mental effort. Even if it was free, every additional memory or note eats up space in my brain or exobrain and makes it harder to find other thoughts when I look for them. But I don’t necessarily know how important a piece of information is when I read it. Good pre-reading might help me know how important it is to that particular work, but never to my life as a whole.

HtRaB acknowledges that different works have vastly different returns to attention and you should allocate your reading effort accordingly, but I don’t feel like it gave me guidance for choosing what to pay attention to, and I have a suspicion that if I pressed the point the authors’ answer would be extremely in line with the literary and scientific canon of the 1940s-1970s.

My favorite section of How to Read a Book is also the most mechanically detailed: the algorithm for pre-processing a book (explained in detail here). I don’t know if this was the most useful to me because it was the most detailed or because a teacher once told me skimming was immoral and I needed that to be challenged.

For the meat of reading, How to Read a Book suggests questions to ask but not how to determine the answer. To be fair, this is hard. As I work on my own guide to reading I’m intensely aware of how difficult it is to translate the intentions and external appearance of what I’m doing to inner workings comprehensible to many, or even to other people very similar to myself.  I suspect there are people for whom reading these questions causes something to click in their brain and they suddenly start reading better, and that’s great, but it makes the book lucky, not good. Which is nothing to be ashamed of: sometimes a stab at a hard problem is worth more than a perfect solution to an easy one.  But HtRaB’s stab did not happen to hit my particular problem, nor contain enough deep models to let me make the stab myself.

My overall impression is that this is one of those books that is helpful if you read it at the right time and pretty meh otherwise, and it was the wrong time for me. I also predicted it would be one of those books that’s notable for founding a genre but goes on to be surpassed by later books that learned from it, but when I looked at Amazon I found very little. There’s lots on speedreading, confusing memorization with learning, and  “how to study to pass a test designed by someone else”, and I may end up reading some of those because the field is that sparse, but they’re not what I actually want. So if there’s a work you or someone you trust likes that attempts to answer any of the following questions, please share it:

  1. How do you find the most likely sources of relevant or useful information?
  2. How do you get the most (useful) information out of a sources?
  3. How do you decide what information to save?
  4. How do you save it in a maximally useful way?

I’m also interested if you have opinions on any of the following:

  • Mind Mapping: Improve Memory, Concentration, Communication, Organization, Creativity, and Time Management
    • Kindle Unlimited
  • The Lifetime Learner’s Guide to Reading and Learning
  • Extend Your Mind: Praxis Volume 2
  • How to Take Smart Notes
  • Accelerated Learning for Expertise: Rapid Knowledge Acquisition Skills to Learn Faster, Comprehend Deeper, and Reach a World-Class Level (Learning how to Learn Book 6) Kindle Edition
  • The Self-Learning Blueprint: A Strategic Plan to Break Down Complex Topics, Comprehend Deeply, and Teach Yourself Anything (Learning how to Learn Book 3)
  • Writing to Learn
  • Unlimited Memory: How to Use Advanced Learning Strategies to Learn Faster, Remember More and be More Productive
  • The Art of Reading

 

So many thanks to my Patreon supporters and the Long Term Future Fund for their support of this research.

Epistemic Spot Check: Children in Colonial America

Introduction

Once upon a time I started fact checking books I read, to see if they were trustworthy. I called this epistemic spot checking because it was not a rigorous or thorough investigation; just following up on things I thought were interesting, likely to be wrong, or easy to check. Eventually I became dissatisfied with this. It placed too much emphasis on a binary decision about a particular book’s trustworthiness, and not enough on building models. So I started working on something better. Something that used multiple sources to build robust models of the world. This book is part of that attempt

Children in Colonial America (affiliate link) is a compilation of chapters from many authors, all centered around what you would guess from the title. This lowers the predictive value of one chapter for assessing the credibility of the whole book, since they’re different authors, but has more than zero value. For purposes of this post, I’m going to look at three chapters. As usual, you can view all my notes in Roam here.

Claims

Chapter 1: Indian Children in Early Mexico

Claim: Mexico was ruled by Spain from 1521-1821, and was known as New Spain during this time (p28).
Verdict: True (source). This is not very impressive to get right, except that I originally recorded it as 1831 and briefly went very down on the chapter. So we know I can catch both embarrassingly obvious factual errors made in books, and my own  embarrassing typos.

 

Claim:  In the 1700s Mexico City had 112,000 people, mostly natives, the largest city in the Americas (p28).
Verdict: Highly defensible but unproven. This is almost exactly what wikipedia reports an inaccessible source reports a primary source recorded in 1793, although 112,926 rounds to 113k, not 112k. It’s not at all clear to me how good this census was, given that even modern censuses have a lot of problems.

Additionally, I’m not thrilled about her using one number to refer to the population throughout an entire century. According to wikipedia the population varied a lot, probably due to competition between disease and immigration, but possibly due to to changes in methodology.

This is still well over the recorded populations of various future-USA cities recorded in the same decade.  However that’s still sensitive to definitions- how do you draw the boundaries between a city and its surrounding farmland?

 

Claim: In the 1700s New Spain had 6m inhabitants.
Verdict: Right order of magnitude. This graph

which is originally from a Spanish-language work I read about here estimates about 1/3 lower than that, but admits that it’s order-of-magnitude at best.

 

Claim: Aztecs buried newborn’s umbilical cord under the hearth (girl) or battlefield (boy) (p29).
Verdict: Well justified (source 1, 2, 3). AFAICT everyone’s ultimate source for this is the Florentine Codex, which has all the problems noted above.

 

Claim:  “Children were the most apt to die in the epidemics because they had no natural immunity” (p36).
Verdict: Close enough. Ignores the fragility of old people and there are flukes like the Spanish Flu, but close enough.

 

Claim: “Probably the happiest moments for boys and girls were the processions, fireworks, and communal meals that took place about eight times a year in the Indian pueblos.” (p37), “another favorite time for Indian youngsters was the feast of the patron saint of their home towns.” (p38)
Verdict: I don’t know how the author could possibly know this with the information they have.

 

Claim: “The Jesuits were expelled by Charles III in 1767 from all of the Spanish territories, resulting in the exile of four hundred priests from New Spain, one-third of whom were teachers.” (p39).
Verdict: Probably wrong (evidence compiled in Roam). There was definitely an expulsion, but the more common number given for number of exiled priests was 678, and that comes from an enumerated census. It’s possible these were referring to different geographic areas, but the book specifically refers to New Spain, not just Mexico. It’s possible it was referring to just a subset of priests- but it specifically names priests and teacher priests. It’s possible they meant some third thing I haven’t thought of, or have a different sources, but in that case they really need to provide it.

Chapter 2: Colonizing Childhood: Religion, Gender, and Indian Children in Southern New England, 1600–1720

Claim: New England colonists used indentured servitude as a way to both extract labor and indoctrinate Native American children (p48).
Verdict: Seems likely but leaves out important facts, such as the fact that women were also forced to provide labor in New England, and native men were sold into slavery in the Bahamas (wikipedia).

It’s a little hard to tell what source the chapter is using for this- there’s a citation 80% of the way through a very long paragraph that lists several sources. The most relevant looking one, Colonizing the Children: Indian Youngsters in Servitude in Early Rhode Island, is focused 50-100 years later than this book, and mentions Christianity once. The Impact of Indentured Servitude on the Society and Culture of Southern New England Indians, 1680-1810 is also set later than this chapter.

 

Claim: “In a reflection of the importance of religious practice to the major transitions in an individual’s life, Indian parents and kin actively assisted children in cultivating relationships with a range of powerful supernatural entities.” (p49)
Verdict: Dubious and undifferentiated.

This quote is a stand-in for the numerous times the author talks about the importance of religion to Native New England children during colonial times. And I don’t doubt that it was, because it’s important to most if not cultures. It certainly was to the Puritan children that are the most natural comparison group. It’s ambiguous if the chapter means to claim the native children were especially religious, and if so, how we know that.

 

Claim: Edward Winslow reported Wampanoag beliefs about a deity who created life, Keihtan. (p51)
Verdict: True but waaaaaaay to much work to verify (source).

How could this take so long to verify, you might ask yourself? It’s just reporting that a dude said a thing, surely you just need to point to the page where he said it and everyone can go on with their day. Alternately, it’s a pretty distinct word, can’t you just google for it?

Yes, either one of those would be easy, but:

  1. The author continued their delightful pattern of putting all their citations in a single superscript that is almost, but not quite entirely, at the end of a paragraph, rather than where he’s asserting the thing.
  2. He spelled the damn name differently than Winslow.

The misspelling isn’t a big deal, phoneticizing a language that may not even have a written form is not easy, and this deity has lots of names. But the combination of the two meant I lost 45 minutes of my life to tracking down this simple fact that wasn’t even relevant to my interests.

 

Claim: English missionaries established Praying Towns, their version of missionary villages in the mid-1600s
Verdict: Obviously true. After the last one I’m just going to accept a tertiary source. There is some conflict among sources about when they started (notes in Roam), which I think is more about the definition of “start” than about when specific things actually happened.

 

 

Chapter 3: Imperial Ideas, Colonial Realities Enslaved Children in Jamaica, 1775–1834

Claim: Children accounted for a “significant” portion of the slave trade, especially compared to other  (p63).
Verdict: Vague but true (Roam notes). The exact number varies based on time, origin, and destination so it’s not clear how meaningful an average is, but 25% looks roughly correct.

More specifically: Was the Slave Trade Dominated by Men? (Roam notes), a survey of mostly British vessels, reports 12-40% of imported slaves being children, depending on the era, based on a survey of ships that covers 8-10% of all slaves transported to the New World (my guess is this is based on the population that arrived alive, not who was on the boat when they left Africa, but it’s not specified).

Sex Ratio, Age and Ethnicity in the Atlantic Slave Trade: data from French shipping and plantation records (Roam notes) finds an overall average of 26.6% children, varying from 9-43% using the standard definition of child in a slavery context at that time, which is “shorter than 4’4”.

26% or even 12% is not trivial, but it’s especially striking when compared to people arriving in the New World as indentured servants, which really were overwhelmingly adult and male, something like 70-90% (although the numbers aren’t quite comparable; I believe European immigrants use their actual birthdates to determine age, as opposed to height) (source: Was the Slave Trade Dominated by Men? )

 

Claim: Children were historically used as sources of labor, including in Britain, in the 1700s and 1800s
Verdict: True, although very hard to find hard numbers for (Roam synthesis). The best estimate I could find was this summary article which cited multiple papers I could not access. It gives the following numbers for child participation in mining and textiles, which it claims were the most popular industries for children

The demographic distribution of Industrial-Revolution Great Britain is also very hard to find, but the following is implied to be the result of a census (source):

If you assume that 1/3 of children were between 10 and 15, and that all working children were between 10 and 15, that would make their labor force participation in mining and textiles (very) approximately equal to their proportion of population in 1860. Of course this is very rough- God knows what the demographics of children were during the Demographic Transition, and some children started working younger than 10. But this does suggestion that children as a whole were less likely to participate in work – at least, work outside the home; it’s not clear how they’re counting work on the family farm.

 

Claim: “the age and social standing of the enslaved are two factors, among others, that would also influence attitudes and reactions to enslavement in the Americas.”
Verdict: Unverified by I believe it. The original source was inaccessible and verifying it was proving to require a lot of general reading that didn’t bear on my question, so I stopped after 15 minutes of trying.

 

 

Process Notes

I had a really hard time verifying anything interesting in Indian Children in Early Mexico, because all of the primary and deeper secondary sources are in Spanish or Nahuatl, neither of which I read. I say “sources”, but it appears there’s only one real source for any information on pre-Spanish Aztecs, which is one missionary (Bernardino de Sahagún), who wrote a general history of the Aztecs, known in English as either the Florentine Codex or The General/Universal History of the Things of New Spain. At least, that’s what everything I read seemed to be relying on, when they gave a source at all. Parts of the codex are available for free in English, but not the relevant ones. Even assuming I could read it, one dude whose official mission was very much to destroy Aztec culture and replace it with his own probably introduced biases to his reports of their culture.

Sometimes I would luck out and a source would be in English but inaccessible. What I could verify with English sources was pretty limited to names and dates.

Colonizing Childhood… brings up the question: how long should I spend verifying minor claims? There’s a fair number of claims I go into with the attitude of “this ought to be easy to find.” You could argue these are low value because it’s very unlikely the author screwed up a basic name or date- but if that’s true, it would be a very bad sign about the author/work if they did screw it up. And surprisingly often, there’s debate about even very simple facts. E.g., different sources listed different start times for Praying Towns. These sources didn’t differ in their facts, but in what they counted as the start- the first sermon or the first town incorporation. I wouldn’t have learned about those distinctions if I hadn’t looked up an “easy to find” fact.

Overall, I was really not impressed by how little context this chapter gave for the tribes it talked about, primarily the Wampanoag. I never would have guessed that they were only semi-sedentary (which is important context for the missionaries’ efforts to move them into Praying Towns) or matrilineal, if I hadn’t sought out other sources. It could be aimed at a more educated audience than me- but the book covers a really wide geographic and social range, and it seems unfair to expect people to be experts in all of them.

Imperial Ideas… was flat out a better chapter than the other two.

    1. It cited sources such that I could actually find them and match them to claims.
    2. It had a delightful pattern of making a general claim and then backing it up with specific instances. E.g.

      young African children were available for sale on the island. In 1792, in a letter reporting the sale of the slave cargo on the ship Ruby, the Jamaican merchant John Cunningham wrote, “I sold 129 slaves. . . . There were 45 boys and girls . . . many not more than 8 or 9 years of age.”9 Over 30 percent of the captives sold from this ship were children.

I found myself marking more claims as potential epistemic spot checks for Imperial Ideas… than I did for the other two chapters. In fact, notetaking was just easier and more free flowing with this chapter. I’m not sure how to operationalize this: punishing books for taking more effort to read has some obvious failure modes.

Actually this brings up something I’ve been wanting to talk about for a while, which is how Epistemic Spot Checks have changed my definition of easy. Time was a book’s easiness rating was based on how much effort I needed to put into understanding the author. It rewarded spoon feeding me conclusions and punished provoking certain kinds of thought. Now easiness is much more based on how much effort I have to put into verifying the author’s claims.

I originally didn’t record any notes from Was the Slave Trade Dominated by Men? about indentured servitude, because it wasn’t strictly relevant to the question I was asking. Luckily I wrote this up the same night and noticed that it seemed worth including.

Verdict

Indian Children in Early Mexico got most of its basic facts right, or at least defensible, and had one probable error. I couldn’t verify anything interesting because of language barrier.

Colonizing Childhood: Religion, Gender, and Indian Children in Southern New England, 1600–1720 also seems fine, although it was more a jumble of facts than a cohesive thesis, and the closest thing it had to a thesis seemed to indicate a specialness that was not backed up by facts.

Imperial Ideas, Colonial Realities Enslaved Children in Jamaica, 1775–1834 was pretty great. It backed up its claims better than the other two and provoked more thought in me.

 

Many thanks to my Patreon patrons and the Long Term Future Fund for funding this post.