Right about everything except the cause

Kahneman was telling Israeli Air Force flight instructors that reward works better than punishment. A senior instructor cut him off. Every time he praised a cadet for a clean maneuver, the next one was worse. Every time he screamed at a cadet for a bad one, the next one was better. So, with respect, the finding was nonsense.

His observations were accurate. His conclusion was wrong. An unusually good flight is partly luck, and luck does not repeat, so the next attempt drifts back toward average whether the instructor praises, shouts or says nothing. The statistics had spent years rewarding him for being harsh. Kahneman later had officers throw coins at a target without looking, and the same pattern appeared with no praise or blame at all.

Stories have a home advantage in the mind. A few linked events feel complete. Probability asks awkward questions instead, about sample size, about what usually happens, about the cases you never saw.

The county that wrote its own health story

A map of kidney cancer across the 3,141 counties of the United States shows the lowest rates in rural, sparse counties in the Midwest, South and West. Clean air, fresh food, a quiet life. Now look at the counties with the highest rates. Rural, sparse, Midwest, South and West. A small county has few people, so a handful of cases swings its rate either way. The pattern was sample size.

The Gates Foundation put around 1.7 billion dollars into smaller schools after noticing that small schools were overrepresented among the best performers. They were overrepresented among the worst too. Trained researchers fell for it in their own labs, running studies so small they had roughly a fifty percent chance of missing a real effect. The same craving for pattern turns a short run of basketball hits into a hot hand; in the analysis the book reports, the sequences passed tests of randomness.

The number on the wheel

A rigged wheel of fortune stopped at 10 or 65. People then guessed what percentage of African countries are in the United Nations. The 10 group averaged 25, the 65 group 45. German judges who rolled loaded dice showing 3 or 9 then proposed shoplifting sentences of five or eight months.

Memory has its own version of the wheel. Whatever comes to mind easily feels common. Strokes killed almost twice as many people as all accidents, yet 80 percent of participants judged accidental death more likely. Accidents make the news. Liking a technology shrinks its risks and swells its benefits in the same breath, and once the media, the public and politicians start feeding each other's alarm, as happened with the apple chemical Alar, attention becomes its own evidence. Citizens who care about dread and fairness are asking a different question, and policy needs both.

Linda and the blue cab

Linda is thirty-one, single, outspoken, a philosophy major who marched against nuclear weapons. Is Linda more likely a bank teller, or a bank teller who is active in the feminist movement? In the original study, 89 percent of undergraduates picked the second. Every feminist bank teller is also a bank teller, so the second cannot be more probable. Detail makes a story more convincing and less probable at once.

Kahneman's repair is dull and it works. Start from how common the outcome is, and let the description move that number only as far as its reliability deserves. Then the cab. In a city where 85 percent of cabs are green, a witness who is right 80 percent of the time says the cab was blue. The real chance is about 41 percent, because the huge green fleet offers many more chances for a mistake. Tell them instead that green cabs cause 85 percent of accidents and they suddenly use the number, because now it describes reckless drivers. A statistic that suggests a mechanism gets into the story. One that just sits there gets ignored, and Kahneman notes that this same shortcut keeps unfair stereotypes usable.

The story told backwards

Google's rise reads as a sequence of brilliant decisions once you leave out the competitors, the near misses and the luck. Kahneman offers a generous calculation. If a chief executive's quality correlates with a firm's success at about three tenths, the apparently stronger of two bosses has roughly a 60 percent chance of running the more successful firm.

In ten thousand brokerage accounts, the stocks people sold went on to beat the stocks they bought by 3.2 percentage points a year, before costs. At one wealth advisory firm, eight years of rankings for 25 advisers correlated at almost exactly zero from year to year. The firm went on paying bonuses for skill anyway. Chief financial officers gave ranges they were 80 percent sure of, and were wrong 67 percent of the time.

Stable ability leaves a trace. The same people should rank high again and again. Without that, a winner may simply be lucky.

Where a formula beats the expert

Paul Meehl compared trained clinicians with simple statistical rules across about two hundred studies. In roughly 60 percent the formula was clearly more accurate and it tied in the rest. The formula understood nothing. It applied the same evidence the same way every time, while radiologists in the studies cited contradicted their own reading of the same image about one time in five. Virginia Apgar's five newborn signs, scored zero to two and added, turned that consistency into a lifesaver. Meehl still allowed the broken leg. If the formula says a man will go to the cinema tonight and he broke his leg this morning, override it. "This case feels different" is not a broken leg.

Intuition keeps its place under conditions Kahneman spells out. Gary Klein's firefighters did not compare options. A familiar pattern brought one action to mind and they ran it in their heads first. Fires are regular enough to learn and the feedback is fast. Stock picking is not, and confidence feels identical in both. One physician diagnosed typhoid by palpating tongues and, unwashed, spread the disease that kept confirming him.

The team that would not look outside

A year into writing a school curriculum, Kahneman's team estimated eighteen months to two and a half years to finish. He asked Seymour Fox how comparable teams had fared. About 40 percent never finished. The ones that did took seven to ten years. Fox rated this team slightly below average. They kept going. It took eight more years and was never used. Only about 35 percent of American small businesses survive five years, yet founders in a study the book cites put their own odds at 70 percent or better, and a third said they could not fail. Optimism gets people to work; it should not write the probability.

The inside view pictures every chapter of its own plan and none of the illnesses and bureaucratic deaths that delayed everyone else. The outside view asks what usually happens. Models for the decisions that repeat, structured evidence before the expert's verdict, intuition only where the world keeps honest score. Everything else is the flight instructor, sure he has learned something, being fooled by the coin.

Three things to keep

Before explaining an extreme result, ask how much of it was luck.

Start from what usually happens to cases like this, then adjust a little.

Trust an expert's gut only where practice came with fast, honest feedback.