Quotes from Philip Tetlock

It is wise to take admissions of uncertainty seriously," Daniel Kahneman noted, "but declarations of high confidence mainly tell you that an individual has constructed a coherent story in his mind, not necessarily that the story is true."17

statisticians sleeping with their feet in an oven and their head in a freezer because the average temperature is comfortable.

Forget the old advice to think twice. Superforecasters often think thrice—and sometimes they are just warming up to do a deeper-dive analysis.

The foundations of our decision making were gravely flawed," McNamara wrote in his autobiography. "We failed to analyze our assumptions critically, then or later."5

With firmness in the right, as God gives us to see the right, let us strive on to finish the work we are in," Abraham Lincoln

Forecasters who can't cope with the dissonance risk making the most serious possible forecasting error in a conflict: underestimating your opponent.

Forecasters who see illusory correlations and assume that moral and cognitive weakness run together will fail when we need them most.

The difference between heavyweights and amateurs, she said, is that the heavyweights know the difference between a 60?40 bet and a 40?60 bet.

More often forecasts are made and then…nothing. Accuracy is seldom determined after the fact and is almost never done with sufficient regularity and rigor that conclusions can be drawn. The reason? Mostly it's a demand-side problem: The consumers of forecasting—governments, business, and the public—don't demand evidence of accuracy. So there is no measurement. Which means no revision. And without revision, there can be no improvement.

It's a rare day when a journalist says, "The market rose today for any one of a hundred different reasons, or a mix of them, so no one knows.

Archilochus: "The fox knows many things but the hedgehog knows one big thing.

if you have the time to think before making a big decision, do so—and be prepared to accept that what seems obviously true now may turn out to be false later.

but declarations of high confidence mainly tell you that an individual has constructed a coherent story in his mind, not necessarily that the story is true.

And only when we are proven wrong so clearly that we can no longer deny it to ourselves will we adjust our mental models of the world—producing a clearer picture of reality. Forecast

When the facts change, I change my mind," the legendary British economist John Maynard Keynes declared.

produce forecast-wrecking

Mauboussin notes that slow regression is more often seen in activities dominated by skill, while faster regression is more associated with chance.15

Machines may get better at "mimicking human meaning," and thereby better at predicting human behavior, but "there's a difference between mimicking and reflecting meaning and originating meaning," Ferrucci said. That's a space human judgment will always occupy.

There's also the "premortem," in which the team is told to assume a course of action has failed and to explain why—which makes team members feel safe to express doubts they may have about the leader's plan. But the superteams did not start with leaders and norms, which created other challenges.

was one of the worst—arguably the worst—intelligence failure in modern history.

The one undeniable talent that talking heads have is their skill at telling a compelling story with conviction, and that is enough. Many have become wealthy peddling forecasting of untested value to corporate executives, government officials, and ordinary people who would never think of swallowing medicine of unknown efficacy and safety but who routinely pay for forecasts that are as dubious as elixirs sold from the back of a wagon.

Natural as such thinking may be, it is problematic. Lay out the tangled chain of reasoning in a straight line and you see this: "The probability that I would meet the love of my life was tiny. But it happened. So it was meant to be. Therefore the probability that it would happen was 100%." This is beyond dubious. It's incoherent. Logic and psycho-logic are in tension.

scientist and science fiction writer Arthur C. Clarke famously observed, "Any sufficiently advanced technology is indistinguishable from magic.

Then came the waiting, a test of patience for even the tenured.