How Accurate Do Citations Need to be?

As part of an investigation in how much capacity for thought work humans actually have in a day, I read Ericsson, Krampe, and Tesch-Römer’s 1993 paper, The Role of Deliberate Practice in the Acquisition of Expert Performance (PDF). This paper is important because if you ask people how much thought work can be done in a day, if they have an answer and a citation at all, it will be “4 hours a day” and “Cal Newport’s Deep Work“. Newport in turn cites the Ericsson paper. I checked Ericsson et al‘s sources, but have hit something of a conundrum.

One specific claim in the paper, the first one relevant to my question, is:

When individuals, especially children, start practicing in a given domain, the amount of practice is an hour or less per day

The source for this is the final chapter of Developing Talent in Young People, by Benjamin S. Bloom. That chapter states “…[D]uring the week the [piano] teacher expected the child to practice about an hour a day.” with descriptions of practice but no quantification given for swimming and math (p515).

I don’t think Ericsson et al‘s summary is accurate.  “Teachers in one specific domain expect one hour of practice a day” is not the same as “In any domain, all individuals do one hour or less.” They differ in the generality of the statement, and one is about expectations, the other achievement.

How much should I penalize the paper for that inaccurate summary, especially given that I don’t think their statement is actually false (who practices a new hobby more than an hour a day?), just that it failed to validate itself within the narrow confines of peer review? Do I conclude Ericsson, Krampe, and Tesch-Römer are inattentive, or that they had a thing they wanted to say and looked for the nearest source to justify it in the way required by peer reviewed papers.

This is harder for me because while it’s the first citation in the paper that I checked, it was actually the last I looked up, because everything else was online and this required interlibrary loan. I already had my opinion and was doing this out of thoroughness. I’m deliberately not sharing that opinion here, because I want others to consider the quote in isolation.

Literature Review: Distributed Teams

My new research report on distributed vs. colocated teams is up on LessWrong. TL;dr: I still love distributed teams, but you have to commit to it. Compromises are doomed.

Is there some topic you’re dying to see me write about? Good news: I’m hirable. In addition to the low-polish social science reports like the one above above, I am also available for:

Prices range from $500-$5000, depending on topic, scope, rigor, and polish.

I’ve had one request for a research report on IBS, but the requester didn’t have sufficient funding. If this is a thing that interests you to the point you’d be willing to contribute, ping me and we’ll see if it’s worth setting up a Kickstarter.