Quotes from Annie Duke

For reasons that are going to become clear, a good decision tool seeks to reduce the role of cognitive bias (such as overconfidence, hindsight bias, or confirmation bias) and a pros and cons list tends to amplify the role of bias.

Determining whether a decision is good or bad means examining the quality of the beliefs informing the decision, the available options, and how the future might turn out given any choice you make.

To figure out whether a decision is good or bad, you need to know not just the things that might reasonably happen and what could be gained or lost, but also the likelihood of each possibility unfolding. That means, to become a better decision-maker, you need to be willing to estimate those probabilities.

There's a name for this: Resulting. When people result, they look at whether the result was good or bad to figure out if the decision was good or bad. (Psychologists

Here's a secret: All guesses are educated guesses because there is almost no estimate you could make about which you literally know nothing.

When the outcome turns out poorly, it's easy to focus on the details that suggest the decision process was poor. We think we are seeing the decision quality rationally because the bad process is obvious.

Andrew Mauboussin and Michael Mauboussin came up with this pretty comprehensive list of these types of terms for a survey they conducted: Almost always More often than not Serious possibility Almost certainly Never Slam dunk Always Not often Unlikely Certainly Often Usually Frequently Possibly With high probability Likely Probably With low probability Maybe Rarely With moderate probability Might happen Real possibility

The quality of the outcome casts a shadow over our ability to see the quality of the decision.

Left to our own devices, we will notice some of the bad luck but overlook most of the dumb luck.

We want outcome quality to align with decision quality.

Experience is necessary for learning. But we process that experience in a biased way. This means that the very feedback you need to become a better decision-maker can interfere with your ability to learn good lessons from experience.

A lot of experience can be an excellent teacher. A single experience, not so much. Looking across a large enough set of decisions and outcomes, we can start to tease out the lessons experience might offer us. Looking at just one outcome, we get resulting and hindsight bias.

The decision you make determines which set of outcomes are possible and how likely each of those outcomes is. But it doesn't determine which of that set of outcomes will actually happen.

We process outcomes sequentially, treating each outcome as if it stands alone. We don't sit back and wait to update our beliefs until we have enough data to overcome the uncertain relationship between outcomes and decisions.

Luck is what intervenes between your decision (which has a range of possible outcomes) and the outcome that you actually get. Because any decision determines only the set of possible outcomes (some good, some bad, some in between), this means good outcomes can result from both good and bad

Some of these terms had startlingly wide ranges, which I imagine you experienced in your four-person survey. For instance, "real possibility" had a range of about 20% to 80%. A quarter of the people taking the survey thought the term meant 40% of the time or less. A quarter thought it meant 40% to 60%. A quarter thought it meant 60% to 75%. Finally, a quarter thought it meant over 75% of the time.

People didn't even agree on what the terms always and never meant! If you're like most people, you were pretty surprised by these results. Most of us aren't aware of the wide range of what these words mean to different people. We assume that when we use a term, other people use it in the same way we do and mean the same thing we do.

Most decisions have a mix of upside and downside potentials. When figuring out whether a decision is good or bad, you are essentially asking if the upside potential compensates for the risk of the downside.

The actual outcome casts a shadow over your ability to remember what you knew at the time of the decision.

The same thing happened after Donald Trump won the presidency. There was a huge outcry about the polls being wrong. Nate Silver, the founder of FiveThirtyEight.com, drew a lot of that criticism. But he never said Clinton was a sure thing. Based on his aggregation and weighting of polling data, he had Trump between 30% and 40% to win (approximately between two-to-one and three-to-two against) in the week before the election. An event predicted to happen 30% to 40% of the time will happen a lot.

Whether it is the forming of a group of friends or a pod at work—or hiring for diversity of viewpoint and tolerance for dissent when you are able to guide an enterprise's culture toward accuracy—we should guard against gravitating toward clones of ourselves. We should also recognize that it's really hard: the norm is toward homogeneity; we're all guilty of it; and we don't even notice that we're doing it.

In fact, losing feels about two times as bad to us as winning feels good to us.

They learned the lesson the ants have down pat: Don't wait to be forced to quit to start exploring alternatives.

Poker players live in a world where that risk is made explicit. They can get comfortable with uncertainty because they put it up front in their decisions. Ignoring the risk and uncertainty in every decision might make us feel better in the short run, but the cost to the quality of our decision-making can be immense. If we can find ways to become more comfortable with uncertainty, we can see the world more accurately and be better for it.