A positive medical test can sound like a verdict. In an example from Nassim Nicholas Taleb, however, most people who test positive are healthy. Understanding why requires counting the people behind the result. The same habit helps us judge a market forecast. Examine how the evidence was produced before accepting the explanation attached to it.
Suppose a disease affects one person in a thousand. A test wrongly returns a positive result for five percent of healthy people. For this example, assume it misses nobody who has the disease. Test a thousand people chosen at random. About one has the disease and tests positive. Of the other nine hundred ninety-nine, roughly fifty also test positive.
We now have about fifty-one positive results, only one of them a true case. Under these assumptions, the chance of disease after a positive result is about two percent. The many healthy people have produced enough false alarms to outnumber the genuine case. Change the disease's prevalence or the test's properties and the answer changes. Testing people because they have symptoms would give us a different starting population.
Correct arithmetic depends on choosing the right comparison. A result rarely tells us how to do that by itself.
How the pattern reached you
Taleb describes trying trading rules against historical prices. Change the conditions for buying, vary a threshold, add another condition, and eventually a wonderful record appears. The rule fits the past because that fit was the reason to select it. Its success carries less weight than a prediction specified before the results were known.
The unsuccessful rules belong in the experiment too. Without them, we cannot tell how many opportunities there were to find a lucky match. Scientific publications can hide a similar selection. Unpublished studies with no striking result leave readers with an incomplete literature, which can exaggerate the apparent support for an effect.
Chance can also produce clusters. Random events need not spread themselves evenly across a map. A concentration of disease cases may deserve investigation, but its location alone cannot establish a local cause.
Even a real recovery leaves the cause unsettled. Taleb recounts seeing a recovered cancer patient advertise a vitamin treatment. The patient could be entirely sincere. Recovery might also have occurred without the treatment. To credit the vitamins, we need evidence that separates their effect from that alternative. A statistically convincing association would still need a causal explanation.
Restraint works in both directions. If a study cannot distinguish an apparent benefit from chance, it has not established the benefit. It has not thereby established that the treatment has no benefit. An inconclusive answer must remain open, however much we want a definite one.
Tomorrow can change the rules
Markets add another problem. Even a genuine historical pattern may stop working. Taleb imagines traders discovering that shares reliably rise on Mondays. They buy on Fridays to get ahead, and their response can erase the opportunity. The people using the forecast help change what it forecasts.
A long record also cannot prove that an unseen loss is impossible. The worst loss recorded so far is merely the worst recorded so far. Different rules can match the same past and disagree about what happens next.
Taleb uses Monte Carlo simulation to explore such uncertainty. A computer generates many possible sequences under specified rules. Comparing their paths and outcomes reveals how much the result depends on luck within that model. Widely scattered results suggest that different random sequences can produce very different fortunes. Similar results suggest less dependence on the particular sequence, within the possibilities allowed. Possible outcomes need not be equally likely; their assigned probabilities require justification too. Running more simulations cannot establish that the rules describe the world. Missing dependencies or abrupt changes in how a system responds can defeat an elegant calculation.
When a model labels an actual loss almost impossible, the model itself deserves scrutiny. The event may reveal a faulty estimate rather than exceptional bad luck. Likewise, an expert's confidence should be checked against forecasting performance, including predictions that failed. A credible forecast needs room for error, expressed as a plausible range, because decisions depend on what might happen around the central estimate. The same average temperature calls for different packing if the possible swings are much wider.
Leaving room for correction
Taleb draws on Karl Popper's demand that a claim expose itself to evidence against it. One black swan refutes the universal claim that all swans are white. A rare event does not automatically refute a model that allowed it. We need to know what the claim actually excludes, rather than invent a fresh explanation that rescues it after every outcome.
Keeping beliefs open requires more than personal resolve. Money and reputation can make abandoning a belief feel like betrayal. A close group can reassure itself until agreement feels like independent confirmation. Other people must be able to challenge the evidence and attempt to reproduce the result. Science can correct ideas even when their original defenders refuse to budge.
These checks let us use evidence without pretending it delivers certainty. Confidence can rise or fall as explanations survive criticism or fail it. The positive test becomes informative once its assumptions and denominator are visible. A forecast earns trust in the same spirit, with its limitations available for inspection and its author prepared to change course.
Three things to keep
A result needs its denominator, its assumptions, and the unsuccessful attempts behind it.
An inconclusive study has not proved that an effect is absent.
Trust grows through testing and correction, while forecasts remain conditional.