Quotes from Michael J. Mauboussin

Barlow writes, "Associations discovered must be genuine ones and not due to chance. The mind that seizes on any random coincidence and sees it as a sign of an important permanent association will not guess right and is not conceded to be intelligent, though it may be at times entertaining.

Not everything that matters can be measured, and not everything that can be measured matters.

What gets measured, gets managed." If we measure the wrong things, we will not achieve our goals.

The location of an activity on the continuum provides guidance on how much reversion to the mean is necessary in making your predictions. High correlations imply limited reversion to the mean; the best estimate for the next outcome is something close to the previous one. Low correlations require substantial reversion to the mean, and the most logical guess for the next outcome is the average. Psychologists have demonstrated that we typically fail to regress to the mean as much as we should.

Further, the size of the sample you take and the length of time over which you measure are essential elements of making a prediction. And of course, you have to be using valid data. So it's important to balance the statistics and the context.

Here's the main point: if you have an activity where the results are nearly all skill, you don't need a large sample to draw reasonable conclusions.