Quotes About Analysis

The purpose of any program evaluation is to provide some kind of counterfactual against which a treatment or intervention can be measured.

In the case of a randomized, controlled experiment, the control group is the counterfactual

You should see that a bigger sample makes for a shrinking standard error, which is how large national polls can end up with shockingly accurate results.

But even the finest recipe isn't going to salvage a meal that begins with spoiled ingredients. So it is with statistics; no amount of fancy analysis can make up for fundamentally flawed data. Hence the expression "garbage in, garbage out.

Although the field of statistics is rooted in mathematics, and mathematics is exact, the use of statistics to describe complex phenomena is not exact.

The decision tree lets you know that your expected payoff is far higher than what you are being asked to invest.

Given the disparity in the size of the states, it's entirely possible that the majority of states are doing worse while the majority of Americans are doing better. The key lesson is to pay attention to the unit of analysis.

The same data can (and should) be interpreted entirely differently if one changes the unit of analysis. We don't care about poor countries; we care about poor people.

Intelligence Gathering and Crime Analysis

The irony is that more data can often present less clarity.

The whole point of a representative sample is that it looks like the underlying population.

Specifically, the sample means will form a normal distribution around the population mean, which in this case is $70,900.

The world is producing more and more data, ever faster and faster. Yet, as the New York Times has noted, "Data is merely the raw material of knowledge."3* Statistics is the most powerful tool we have for using information to some meaningful end

we have now introduced two different measures of dispersion: the standard deviation and the standard error.

The standard deviation measures dispersion in the underlying population

these cases, the goal is to find two groups of subjects who are broadly similar except for the application of whatever "treatment" we care about.

Life gets a little trickier when we are doing our regression analysis (or other forms of statistical inference) with a small sample of data.

Logic suggests that we should be less confident about generalizing our results to the entire adult population from a sample of 25 than from a sample of 3,000.

Our sample of 25 will still give us meaningful information, as would a sample of 5 or 10—but how meaningful? The t-distribution answers that question.

Specifically, the more data we have in our sample, the more "degrees of freedom" we have when determining the appropriate distribution against which to evaluate our results.

Statistics cannot prove anything with certainty. Instead, the power of statistical inference derives from observing some pattern or outcome and then using probability to determine the most likely explanation for that outcome.

For large samples, we can assume that the standard deviation of the sample is reasonably close to the standard deviation of the population.*

The overall lesson of this chapter is that statistical malfeasance has very little to do with bad math.

Statistical analysis is the detective work that crafts the raw data into some meaningful conclusion.