This is pretty inside baseball and I suspect boring to most readers. You have my blessing to skip it.

Two weeks ago I published Knowledge Boostrapping v0.1, an algorithm for turning questions into answers. Since then I’ve gotten a moderate amount of feedback, a major theme of which is lack of clarity in certain sections. That’s entirely fair, but not something I immediately know how to fix. In an attempt to make some headway on the problem, I recorded in detail the steps I took in a recent attempt at using a method.

This isn’t really a central case of use- the question was of business norms, not The Truth, and the research I did more primed me to think of the solution myself, more than it provided the answer. But non-central examples are sometimes more useful and the cost to sharing is low given, so here we go.



  1. 12:30 Working on a project with someone, get to the point where we think “we should have a white paper”

  2. Problem: I’ve never written a white paper, and he has but isn’t really clear on what the goals or audience are. I decide this is worth an hour to get right.

  3. 12:45 Follow my own advice to come up with questions to investigate.

  4. Step 1 is make a mind map of questions. Discover Whimsical’s free plan only allows four creations, ever, even if you delete old work.

    1. Go on FB to complain to friend who recommended it to me

    2. 12:57. Try to create in Roam instead
      Start page currently looks like:

  5. 12:58 Google “white papers 101”

  6. 12:58: first hit . Seems…uncomprehensive? Not high level enough? Extremely vague to the point of uselessness?

  7. 12:59 fight with NYTimes paywall . Article itself is useless but links to some promising pages.

    1. 1:01 . It’s basically a link to

      1. 1:02 no longer has the PDF referenced above

    2. 1:02

      1. Not useless. Begin notes on the top level Roam page, because I expect to only extract a few things from it.

      2. White papers can be aimed at a variety of audiences. Know which one you want and tailor it to them.

        1. 1:06 He doesn’t say it explicitly, but presumably a white paper is aimed at people you want things from. I think about who [partner] and I want things from

          1. This is kind of a dumb revelation. If white papers weren’t aimed at who we wanted things from, we should write a different kind of thing aimed at people we wanted things from.

          2. Rest of page is kind of vapid but I’m feeling pretty inspired.







  • 1:10 Remember to tag parent Roam page as a question. I would show you what Roam page looks like now, but apparently I forgot to take a picture.

  • Take that prompt and think a bit, till I come up with an algorithm I’m happy with


Knowledge Bootstrapping Steps v0.1


I wanted to have this published weeks ago, but the process of writing it highlighted some gaps, and the experiments to fill in those gaps highlighted more gaps, and… it’s become clear there’s always going to be one more experiment to run, and if I’m going to publish at all, it will be with gaps.  Furthermore, publishing may elicit feedback that alleviates the need to run some of the experiments, or suggests better ones. So with shaking fingers I am going to publish an incomplete instruction set, and hope it does more good than harm.

The goal of this system is to turn questions into answers. It does this by collecting questions and assembling data that relate to them until you can come to a confident answer, or at least an answer with a confidence interval.  This system has worked amazingly well for me, but I am one person. I am very sure that 

  1. It will need to be tweaked for other people 
  2. There are important steps I’m doing without noticing and thus didn’t know to write down. 
  3. This isn’t its final form for me either.

One of my goals in publishing is to get data from other people on how it doesn’t work for them, so I can universalize it. To that end:

  1. You can comment here
  2. You can email me at
  3. You can talk to me for an hour
  4. I am offering a limited amount of coaching on using the method. If you’re interested in this, please reach out at that same e-mail address.

The following system was designed to be run on Roam, which when I started was free and freely available. There is now a subscription fee, and off and on a waitlist. For me it’s worth it (well, will be, when they get around to charging legacy accounts), but I don’t know where the line of “worth it” falls for other people. I’ve got a list of untested alternatives at the bottom of this post, but the instructions will assume you’re working in Roam.


The Basics

The basic gameplay loop of this system is:

  1. Create a question
  2. Read sources to gather evidence bearing on question, and save it in notes
  3. Assemble evidence new and old into an answer (synthesis).

That looks pretty simple, but raises a number of questions:

  1. How do you create a good question?
  2. How do you choose what sources to read?
  3. How do you know what to write down?
  4. How do you synthesize the information into an answer?

I’ve linked to the questions where I have stabs at an answer; the remainder is still an intuition-based art to me.


The Details

Those steps are also pretty vague. Here’s the detailed version:

  1. What do you want to know? Try to be as specific as possible, but no simpler (i.e. leave room to be surprised). Create a Question page for this question (example, template, justification). 
  2. Break that question into smaller sub-questions, and continue until it feels indivisible at your current level of knowledge. Create links on the main question to all of your sub questions (example).
  3. Answer any questions you are confident you already know the answer to
  4. Pick the question that feels most live/interesting to you
  5. Write down everything you already know about the question, including things that don’t seem relevant but keep popping into your brain.
  6. Find sources for it (“How do I do that, Elizabeth?” “Still figuring out how to make that knowledge explicit, reader, but take a look here”).
  7. Pick the most promising looking source
    1. Create a Source page for it (template, example). Fill in metadata, including (in order of declining importance):
      1. URL: If you need to look something up about the work, you want to make sure you’re referring to the same version every time.
      2. Pagination: the type of pagination system you’re using (Kindle location and PDF page number are my most popular) and what it’s total count is in the version of the work you’re using. This is very useful when there are multiple versions or editions of a work and you want to make
      3. Preparatory brain dump: write down everything you already know about this work, or the topic in general (if you didn’t get it out of your head already). 
        1. Things this might include:
          1. Feelings on the author
          2. Why I chose to read this particular work
          3. Questions I have about the book already.
        2. This is important to me for reducing drag while reading: Everything I write in this spot would otherwise be taking up mental RAM.
      4. Subject: Tags for the subjects of the work (e.g., history, biology, economics, the Great Depression, covid). You can have as many tags as you want, and more is generally better, limited by your patience.
      5. Author: This seems really important for collating work from specific authors, but to be honest it’s never come up for me.
      6. Table of contents (long form only): Writing this down (or more likely, copy/pasting it and then correcting the formating) is helpful to me in orienting to topics and structure of the book. Theoretically I could achieve the same thing by reading the table of contents and paying appropriate attention, but in practice I don’t.
      7. Year published
    2. The last section on the form is Notes. 
      1. It will look something like this:
        1. Notes
        2. Introduction Pre-read
        3. Introduction
        4. Introduction summary
        5. Chapter 1 Pre-read
        6. Chapter 1
        7. Chapter 1 summary
      2. A “pre-read” is reading a few paragraphs from the beginning or ending of a chapter, and summarizing what they tell you about the chapter (it’s topic, approach, criticisms the author felt the need to preemptively defend against…). This will make it easier to determine which of the chapter’s claims are important.
      3. The actual chapter content looks like the following:
        1. Chapter 1
          1. Claim: Columbus set sail from Europe in 1492
            1. Pg 1
          2. Claim: Columbus’s expedition was financed by Queen Isabel of Spain
            1. Pg 2
        2. Not sure what to write down? You probably need to refine your question.
        3. Write down something relevant to one of your questions? Create a block reference to it under the Question’s evidence bullet, or copy/paste it if you’re not using Roam
      4. The chapter summary is what you think it is. I typically have to refer back to my notes to create the summary, but sometimes realize in the process of summarizing that I’ve left out something key and need to add it back in.
      5. Rinse and repeat.
      6. Did you record a claim that seems particularly interesting, important to the author’s thesis, or suspicious? Time to recurse. Create a Question for which that claim could be evidence (e.g. “Who financed Columbus’s expedition?”). Repeat this process with that question, from the beginning.
  8. Assemble claims from multiple sources under questions until you feel confident in an answer. 
  9. Change title of page to Synthesis: My Conclusion (example).
  10. Work your way up until you’re at your top question.
  11. You should now have answers to all your sub questions, allowing you to answer it, or failing that be confident in your uncertainty.
    1. If a work is only ever going to be referenced by one question, you don’t need to create a separate Source page for it, just give it a bullet under evidence (example).



The following are oft-mentioned competitors to Roam, although to the best of my knowledge none of them have block embedding that I consider crucial:

  1. Google Docs
  2. Obsidian (locally stored)
  3. Workflowy (lacks bidirectional links and cross-list embedding)
  4. Notion (I’ve used this for large projects and hate it)


EDIT 2020-07-20: Added some small changes based on feedback, and remembered to give credit to the Long Term Future Fund for funding this research.

Breaking Questions Down

Previously I talked about discovering that my basic unit of inquiry should be questions, not books. But what I didn’t talk about was how to generate those questions, and how to separate good questions from bad. That’s because I don’t know yet; my own process is mysterious and implicit to me. But I can give a few examples.

For any given question, your goal is to disambiguate it into smaller questions that, if an oracle gave you the answers to all of them, would allow you to answer the original question. Best case scenario, you repeat this process and hit bedrock, an empirical question for which you can find accurate data. You feed that answer into the parent question, and eventually it bubbles up to answering your original question.

That does not always happen. Sometimes the question is one of values, not facts. Sometimes sufficient accurate information is not available, and you’re forced to use a range- an uncertainty that will bubble up through parent answers. But just having the questions will clarify your thoughts and allow you to move more of your attention to the most important things.

Here are a few examples.  First, a reconstructed mind map of my process that led to several covid+economics posts. In the interests of being as informative as possible, this one is kind of stylized and uses developments I didn’t have at the time I actually did the research.

Vague covid panic@2x.png

If you’re curious about the results of this, the regular recession post is here and the oil crisis post is here.

Second, a map I created but have not yet researched, on the cost/benefit profile of a dental cleaning while covid is present.

Risk model of dental cleanings in particular@2x.png

Aside: Do people prefer the horizontal or vertical displays? Vertical would be my preference, but Whimsical does weird things with spacing so the tree ends up with a huge width either way.

Honestly this post isn’t really done; I have a lot more to figure out when it comes to how to create good questions. But I wanted to have something out before I published v0.1 of my Grand List of Steps, so here we are.

Many thanks to Rosie Campbell for inspiration and discussion on this idea.