Quotes from Philip E. Tetlock

We are all forecasters. When we think about changing jobs, getting married, buying a home, making an investment, launching a product, or retiring, we decide based on how we expect the future will unfold.

Scientists must be able to answer the question "What would convince me I am wrong?" If they can't, it's a sign they have grown too attached to their beliefs.

There are no certainties in life—not even death and taxes if we assign a nonzero probability to the invention of technologies that let us upload the contents of our brains into a cloud-computing network and the emergence of a future society so public-spirited and prosperous that the state can be funded with charitable donations.

Once we know the outcome of something, that knowledge skews our perception of what we thought before we knew the outcome: that's hindsight bias. Baruch Fischhoff was the first to document the phenomenon in a set of elegant experiments.

Consumers of forecasting will stop being gulled by pundits with good stories and start asking pundits how their past predictions fared—and reject answers that consist of nothing but anecdotes and credentials.

Take the price of oil, long a graveyard topic for forecasting reputations.

Not knowing is exciting. It's an opportunity to discover.

Like everyone else, scientists have intuitions. Indeed, hunches and flashes of insight—the sense that something is true even if you can't prove it—have been behind countless breakthroughs. The interplay between System 1 and System 2 can be subtle and creative.

When you combine the judgments of a large group of people to calculate the "wisdom of the crowd" you collect all the relevant information that is dispersed among all those people. But none of those people has access to all that information. One person knows only some of it, another knows some more, and so on.

This approach, built on the "wisdom of the crowd" concept, has been called "the crowd within." The billionaire financier George Soros exemplifies it. A key part of his success, he has often said, is his mental habit of stepping back from himself so he can judge his own thinking and offer a different perspective—to himself.

Maxims Hidden in the Text Try, fail, analyze, adjust, try again. John Maynard Keynes cycled through these steps ceaselessly. 178 An imperfect decision made in time is better than a perfect one made too late. 215-216 Plans are merely a platform for change. Israeli Defense Forces slogan. 222 If we ask many tiny pertinent questions, we can close in on an answer for the big question. 263

The truth is, the truth is elusive.

To illustrate, imagine

Knowledge is something we can all increase, but only slowly. People who haven't stayed mentally active have little hope of catching up to lifelong learners.

Foresight isn't a mysterious gift bestowed at birth. It is the product of particular ways of thinking, of gathering information, of updating beliefs.

Stock prices do not always reflect the true value of companies, so an investor should study a company thoroughly and really understand its business, capital, and management when deciding whether it had sufficient underlying value to make an investment for the long term worthwhile.

the CIA was training Cuban exiles to land in Cuba and launch a guerrilla war against the new government of Fidel Castro.

But groups also let people share information and perspectives. That's good. It helps make dragonfly eye work, and aggregation is critical to accuracy.

For centuries, it hobbled progress in medicine. When physicians finally accepted that their experience and perceptions were not reliable means of determining whether a treatment works, they turned to scientific testing—and medicine finally started to make rapid advances. The same revolution needs to happen in forecasting.

If forecasters can keep questioning themselves and their teammates, and welcome vigorous debate, the group can become more than the sum of its parts.

on average, teams were 23% more accurate than individuals.

People take things differently. What one person would consider a helpful inquiry another might take as an aggressive criticism.

I have been struck by how important measurement is to improving the human condition," Bill Gates wrote. "You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal….

My research collaborator Don Moore points out that police officers spend a lot of time figuring out who is telling the truth and who is lying, but research has found they aren't nearly as good at it as they think they are and they tend not to get better with experience. That's because experience isn't enough. It must be accompanied by clear feedback.