Quotes from David Franklin

It turns out that you only need 23 people for it to be more likely to find a shared birthday than not.

With 80 people, the size of this classroom, the probability that there will be no shared birthdays is so tiny as to be virtually impossible.

Dan's goal, taken from his mentor Richard Zeckhauser, is to get students to think probabilistically about the world. This means engaging with the challenge of understanding and accepting what 29% means in the context of a Trump victory, and fighting the brain's natural inclination to see things in binary terms.

Acknowledging that students have competing purposes is the first step to managing them.

Dan's use of two-stage exams kills two birds with one stone. Firstly, he maximises learning by ensuring that the exam itself is a learning experience. Second, in doing so, he makes clear that the grade is less important than the learning. Two-stage exams have not yet 'taken off' around the world, and grades remain the key outcome of most exams for most students. Dan, though, has taken advantage of his position in a graduate university environment to push the idea forward.

The inclusive atmosphere of the class was the key driver of allowing the class to be eager to learn. The focus really shifted away from grades and points, and towards understanding concepts and learning." The key link is between inclusivity and the shift of focus away from test scores. When students are given the responsibility to support each other, their own score falls down the list of priorities.

In Dan's case, the interest in policymaking is so central to the students' presence at Harvard that it would be foolish to run a statistics course that did not acknowledge it. He goes as far as to build this into the purpose of the course: as we heard earlier, his purpose is "not just to maximise learning about statistics, but also to maximise learning of the skills that will be useful to have out there in the world.

Availability bias prevents us from setting our prior beliefs appropriately, whereas confirmation bias stops us from updating them for new information.

The phrases 'certainty is an illusion' and 'think probabilistically about the world', both introduced in the first class, are the first two examples of high-level concepts which Dan calls airport ideas.

The great economist John Maynard Keynes is supposed to have said "when the facts change, I change my mind. What do you do, sir?

But few of us really stop to ask ourselves what new information might change our minds, especially on questions of politics and identity. We fall into the trap of confirmation bias, in which rather than updating our prior beliefs, we mould new information into a form that will confirm them.

The first key idea is that when you have a challenge for which measuring progress is hard, the ability to adapt is just as important as the ability to plan.

For these 'future leaders' in the classroom, it is more important to understand the psychology of certainty. Often, we believe what we want to believe. Dan starts with an example of the 2016 Presidential Election in the United States. He asks the question: "Relative to what you expected to happen, how surprised were you when you learned that Trump had won the election?

The students are asked to rate their level of surprise on a five-point scale from 'not surprised at all' to 'beyond shock'. Many describe feelings of total shock and numbness: a large majority indicate some level of surprise. He asks them: "Why were you surprised?

The brain has conflicting goals: to provide you with information, and to reduce your anxiety about the worrying outcome. If you wanted Clinton to win in 2016, your brain achieved both goals by accepting the 30% figure but telling you it would not happen.

Dan asks the class whether any of them has ever checked the weather on their phone to see if it's likely to rain while they're outside. Everyone has. "What do you do if you see there's a 30% chance of rain?" "Take an umbrella", several students all say at once.

It sounds like a lot of you are not too surprised if this 30% chance materialises," Dan replies. "You take precautions against it. The rain, if it comes, doesn't shock you, and you have your umbrella. So why was everyone so shocked about Trump?

I guess… it was just a shocking event. We're not used to someone like him being President." Here, the student reveals an availability bias in which we expect future outcomes to look like what has gone before.

After pausing, the student continues: "There's also an element of believing something because you want it to happen." This betrays a second bias in which we believe what we want to be true. This is a form of confirmation bias, which exacerbates the distorting effect of availability bias. We seek out information that confirms our existing beliefs or desires, and ignore information that refutes them.

He asks the 80 students to respond based on their 'gut feeling'. Again, students are given five options, ranging from 'less than 1%' to 'above 40%'. About half of them believe the true answer is less than 5%, of which plenty go for the 'less than 1%' option. Only 1 in 6 get it right, picking the highest option: it turns out that the true figure is 41%. He invites those people – 13 in total – to stand up.

Now – those who got it right. On the count of three, I'd like you to sit down if you'd seen the question before. 1, 2, 3!"  Everyone who was standing sits down. This gets a huge laugh. The students who got the answer wrong suddenly feel much better about their mistake: in a Harvard class, not a single person got it right unless they had seen it before.

But everyone who guessed in this way went low, rather than high. It suggests that there is some form of systemic bias in play.