Truthseeking is not Just About Proving Yourself Wrong

Person: I feel like my partner is hurting me, but I want to be fair, so let me seek outside opinions, as is best practice
Everyone and their dog: yup, you sure are justified in feeling hurt by that and the mountain of other things your SO has done and continues to do.
Person: that’s not what I wanted to hear at all.

Real World Example (especially this comment).

Person looks like they are enacting a truthseeking protocol, but they still have an answer in mind and are searching for that rather than the truth. If they actually wanted the truth they would stop when people unanimously told them they were right.* It’s essentially self gas-lighting, where people look for reasons to undercut their own beliefs because they don’t want them to be true.

 

*There are cases where this is not true, but these aren’t them.

 

Epistemic Spot Check: The Role of Deliberate Practice in the Acquisition of Expert Performance

Epistemic spot checks typically consist of references from a book, selected by my interest level, checked against either the book’s source or my own research. This one is a little different that I’m focusing on a single paragraph in a single paper. Specifically as part of a larger review I read Ericsson, Krampe, and Tesch-Römer’s 1993 paper, The Role of Deliberate Practice in the Acquisition of Expert Performance (PDF), in an attempt to gain information about how long human beings can productivity do thought work over a time period.

This paper is important because if you ask people how much thought work can be done in a day, if they have an answer and a citation at all, it will be “4 hours a day” and “Cal Newport’s Deep Work“. The Ericsson paper is in turn Newport’s source. So to the extent people’s beliefs are based on anything, they’re based on this paper.

In fact I’m not even reviewing the whole paper, just this one relevant paragraph: 

When individuals, especially children, start practicing in a given domain, the amount of practice is an hour or less per day (Bloom, 1985b). Similarly, laboratory studies of extended practice limit practice to about 1 hr for 3-5 days a week (e.g., Chase & Ericsson, 1982; Schneider & Shiffrin, 1977; Seibel, 1963). A number of training studies in real life have compared the efficiency of practice durations ranging from 1 -8 hr per day. These studies show essentially no benefit from durations exceeding 4 hr per day and reduced benefits from practice exceeding 2 hr (Welford, 1968; Woodworth & Schlosberg, 1954). Many studies of the acquisition of typing skill (Baddeley & Longman, 1978; Dvorak et al.. 1936) and other perceptual motor skills (Henshaw & Holman, 1930) indicate that the effective duration of deliberate practice may be closer to 1 hr per day. Pirolli and J. R. Anderson (1985) found no increased learning from doubling the number of training trials per session in their extended training study. The findings of these studies can be generalized to situations in which training is extended over long periods of time such as weeks, months, and years

Let’s go through each sentence in order. I’ve used each quote as a section header, with the citations underneath it in bold.

“When individuals, especially children, start practicing in a given domain, the amount of practice is an hour or less per day”

 Generalizations about talent development, Bloom (1985)

“Typically the initial lessons were given in swimming and piano for about an hour each week, while the mathematics was taught about four hours each week…In addition some learning tasks (or homework) were assigned to be practiced and perfected before the next lesson.” (p513)

“…[D]uring the week the [piano] teacher expected the child to practice about an hour a day.” with descriptions of practice but no quantification given for swimming and math (p515).

The quote seems to me to be a simplification. “Expected an hour a day” is not the same as “did practice an hour or less per day.”

“…laboratory studies of extended practice limit practice to about 1 hr for 3-5 days a week”

Skill and working memory, Chase & Ericsson (1982)

This study focused strictly on memorizing digits, which I don’t consider to be that close to thought work.

Controlled and automatic human information processing: I. Detection, search, and attention. Schneider, W., & Shiffrin, R. M. (1977)

This study had 8 people in it and was essentially an identification and reaction time trial.

Discrimination reaction time for a 1,023-alternative task, Seibel, R. (1963)

3 subjects. This was a reaction time test, not thought work. No mention of duration studying.

 

“These studies show essentially no benefit from durations exceeding 4 hr per day and reduced benefits from practice exceeding 2 hr”

Fundamentals of Skill, Welford (1968)

In a book with no page number given, I skipped this one.

Experimental Psychology, Woodworth & Schlosberg (1954)

This too is a book with no page number, but it was available online (thanks, archive.org) and I made an educated guess that the relevant chapter was “Economy in Learning and Performance”. Most of this chapter focused on recitation, which I don’t consider sufficiently relevant.

p800: “Almost any book on applied psychology will tell you that the hourly work output is higher in an eight-hour day than a ten-hour day.”(no source)

Offers this graph as demonstration that only monotonous work has diminishing returns.

Screen Shot 2019-05-16 at 9.08.22 PM.png

 

p812: An interesting army study showing that students given telegraphy training for 4 hours/day  (and spending 4 on other topics) learned as much as students studying 7 hours/day. This one seems genuinely relevant, although not enough to tell us where peak performance lies, just that four hours are better than seven. Additionally, the students weren’t loafing around for the excess three hours: they were learning other things. So this is about how long you can study a particular subject, not total learning capacity in a day.

Many studies of the acquisition of typing skill (Baddeley & Longman, 1978; Dvorak et al.. 1936) and other perceptual motor skills (Henshaw & Holman, 1930) indicate that the effective duration of deliberate practice may be closer to 1 hr per day

The Influence of Length and Frequency of Training Session on the Rate of Learning to Type, Baddeley & Longman (1978)

“Four groups of postmen were trained to type alpha-numeric code material using a conventional typewriter keyboard. Training was based on sessions lasting for one or two hours occurring once or twice per day. Learning was most efficient in the group given one session of one hour per day, and least efficient in the group trained for two 2-hour sessions. Retention was tested after one, three or nine months, and indicated a loss in speed of about 30%. Again the group trained for two daily sessions of two hours performed most poorly.It is suggested that where operationally feasible, keyboard training should be distributed over time rather than massed”

 

Typewriting behavior; psychology applied to teaching and learning typewriting, Dvorak et al (1936)

Inaccessible book.

The Role of Practice in Fact Retrieval, Pirolli & Anderson (1985)

“We found that fact retrieval speeds up as a power function of days of practice but that the number of daily repetitions beyond four produced little or no impact on reaction time”

Conclusion

Many of the studies were criminally small, and typically focused on singular, monotonous tasks like responding to patterns of light or memorizing digits.  The precision of these studies is greatly exaggerated. There’s no reason to believe Ericsson, Krampe, and Tesch-Römer’s conclusion that the correct number of hours for deliberate practice is 3.5, much less the commonly repeated factoid that humans can do good work for 4 hours/day.

 

[This post supported by Patreon].

Kencko Fruit Powder: Better Than Anything I’m Actually Going to do

UPDATE: As much as I love the concept of Kencko, if I drink them too fast they make me vomit, even when very diluted. I reluctantly withdraw my seal of approval.

 

Sometimes the modern economy really delivers.

As longtime readers know, I have two strikes against me when it comes to food: it requires both chewing and digesting. Chewing is painful for me due to nerve damage in my jaw. Digesting… well some of the problems with digestion are caused by insufficient stomach acid, but those are easily treated with a pill. I still have problems when I take the pills and no one knows why. So I eat a lot of things that require minimal chewing and are easy to digest. This set of things has very little overlap with the set of things my nutritionist wants me to eat, such as produce. I eat some fruits and veggies, especially in the summer, but not nearly enough.

Enter Kencko.

 

 

Kencko produces small packets of powderized fruits and vegetables. This requires no chewing and substantially less digestion. They taste fine. Not amazing, but fine. You could probably make them taste better by adding sugar or honey. Because they’re produced by freeze drying, they’re better nutritionally than other preserved fruit. Not as good as fresh, but, in the words of my nutritionist, “better than anything you’re actually going to do.” 

Nutritionists are hit-or-miss, so I double checked the nutrition claim myself. Based on a rushed review (primary source), I find that freeze drying has some nutritional loss, exact amount depending on the nutrient, but within the range that could conceivably counterbalanced by the increased digestibility of powder (this also means the sugar hits faster), and also the fact that I’m eating them at all. I suspect the biggest loss is the absence of probiotic flora in the sterilized powder packets.

There’s the issue of price. I was originally going to apologize for the price, chalk it up to convenience, and plead necessity for myself, but it turns out the packets are not that expensive relative to comparables. Ordered in the largest size, Kencko is $3.07/ounce. I spent 45 minutes finding prices for other freeze dried fruit powders, and that’s as good as you can do short of wholesale (spreadsheet). There are cheaper powders, but they’re inevitably something other than freeze-dried.

How about compared to actual fruit? It takes .44 lbs of fruit to produce one 20 g Kencko packet (price: $2.16-$3.30, depending on quantity ordered). According to this USDA report (chart on page 3), .5 lbs is $0.66-$0.90 cents worth of apples, $2.16 worth of blueberries, or $2.10-$2.85 worth of cherries. Note that those are advertised prices, so probably less than what you’d pay on average and certainly less than what you’d pay out of season, and for conventional produce rather than the organics Kencko uses. Kencko is definitely more expensive than in-season, on-sale produce, but not ridiculously so. Plus it never goes bad so you’re not paying for produce you throw away.

The worst thing I will say about Kencko is that their mixer bottle sucks. It mixes less well and is harder to clean than a Blender Bottle (affiliate link), buy one of those or use a spoon.

Obviously if you can just eat a vegetable you should do that. But if you find that untenable for some reason, Kencko is a reasonable way to turn money into consumed produce. This is an incredibly good trade for me and I’m really happy it exists.

[Kencko has not paid me for this post and I’m not in contact with them beyond ordering the product and following them on Twitter.]

How Accurate Do Citations Need to be?

As part of an investigation in how much capacity for thought work humans actually have in a day, I read Ericsson, Krampe, and Tesch-Römer’s 1993 paper, The Role of Deliberate Practice in the Acquisition of Expert Performance (PDF). This paper is important because if you ask people how much thought work can be done in a day, if they have an answer and a citation at all, it will be “4 hours a day” and “Cal Newport’s Deep Work“. Newport in turn cites the Ericsson paper. I checked Ericsson et al‘s sources, but have hit something of a conundrum.

One specific claim in the paper, the first one relevant to my question, is:

When individuals, especially children, start practicing in a given domain, the amount of practice is an hour or less per day

The source for this is the final chapter of Developing Talent in Young People, by Benjamin S. Bloom. That chapter states “…[D]uring the week the [piano] teacher expected the child to practice about an hour a day.” with descriptions of practice but no quantification given for swimming and math (p515).

I don’t think Ericsson et al‘s summary is accurate.  “Teachers in one specific domain expect one hour of practice a day” is not the same as “In any domain, all individuals do one hour or less.” They differ in the generality of the statement, and one is about expectations, the other achievement.

How much should I penalize the paper for that inaccurate summary, especially given that I don’t think their statement is actually false (who practices a new hobby more than an hour a day?), just that it failed to validate itself within the narrow confines of peer review? Do I conclude Ericsson, Krampe, and Tesch-Römer are inattentive, or that they had a thing they wanted to say and looked for the nearest source to justify it in the way required by peer reviewed papers.

This is harder for me because while it’s the first citation in the paper that I checked, it was actually the last I looked up, because everything else was online and this required interlibrary loan. I already had my opinion and was doing this out of thoroughness. I’m deliberately not sharing that opinion here, because I want others to consider the quote in isolation.

Literature Review: Distributed Teams

My new research report on distributed vs. colocated teams is up on LessWrong. TL;dr: I still love distributed teams, but you have to commit to it. Compromises are doomed.

Is there some topic you’re dying to see me write about? Good news: I’m hirable. In addition to the low-polish social science reports like the one above above, I am also available for:

Prices range from $500-$5000, depending on topic, scope, rigor, and polish.

I’ve had one request for a research report on IBS, but the requester didn’t have sufficient funding. If this is a thing that interests you to the point you’d be willing to contribute, ping me and we’ll see if it’s worth setting up a Kickstarter.

Horrible Stories from Ray, Part 1

Ray is my nearest homeless neighbor, who I sometimes bring food and talk to. Ray tells me lots of stories (and has given me permission to share them here), but today was the most horrifying. To explain why, I have to explain a hierarchy of badness.

First are the stories of things people do for other reasons that happen to hurt him, like the street cleaning chemicals. No one is trying to hurt him, it’s just that they want the other thing more than they want him not hurt, or haven’t thought of him at all.

Second are the spontaneously malicious. His tent has been destroyed repeatedly. People fuck up his stuff a lot. Obviously this is terrible, but at least it’s running on id.

What he told me tonight are stories from the third category, the planned malicious. For example, someone gave him a bag of fried prawns, and mixed in fried feeder mice. That means the someone, somewhere, went and bought mice (or maybe had them around for their snake), went through the trouble to deep fry them, spent money on prawns, mixed them in, and then went looking for someone to accept them. Another person sliced up raw chicken to imitate sashimi and gave it to him in udon.  How terrible a person do you have to be to not, at any point in that process, stop and decide not to? Planned evil is so much worse than spontaneous evil.