Bayes' Theorem

A two-line formula scribbled by an 18th-century clergyman, and the reason a positive test for a rare disease usually doesn't mean what you think it means.

In the middle of the eighteenth century, a Presbyterian minister called Thomas Bayes wrote a paper that he never bothered to publish. He died in 1761. His friend Richard Price found the manuscript among his effects, polished it, and read it to the Royal Society two years later. It was titled “An Essay towards solving a Problem in the Doctrine of Chances,” and it contained a single rearrangement of probabilities that, two and a half centuries on, runs much of modern science, medical diagnostics, machine learning, and the email filter that decides what reaches your inbox.

The rearrangement, in modern notation, is shorter than its name. P(H | E) = P(E | H) · P(H) / P(E). Read aloud: the probability of a hypothesis H given some evidence E equals the probability of seeing that evidence if H were true, multiplied by your prior belief in H, divided by the total probability of seeing the evidence at all. It is a rule for flipping a conditional around — you knew how likely the evidence was given the cause, and now you want to know how likely the cause is given the evidence.

That sentence sounds dry. It is not. Once you understand it, you stop being able to read the news the same way.

The test that lies more than it tells

A standard story — beloved of statistics teachers because it works on doctors too. A disease afflicts 1 in 1,000 people. There is a test for it. The test is 99% accurate: 99% of the time it correctly identifies a sick person as sick, and 99% of the time it correctly identifies a healthy person as healthy. You take the test. It comes back positive. What is the probability that you actually have the disease?

Most people — including, in repeated studies, a majority of physicians — answer “99%.” The correct answer is about 9%.

The intuition almost everyone misses: the test's accuracy is being applied to a population where the disease is rare. Imagine testing ten thousand people. Ten of them have the disease and all ten test positive. But of the 9,990 healthy people, 1% — about a hundred — also test positive in error. So 110 people receive a positive result, and only 10 of them are actually sick. Roughly 9%.

The base rate dominates. The test's accuracy is enormous; the prior is more enormous still. Bayes' theorem is the only thing that tells you how to weigh them. Ignore it and you will, very confidently, be wrong.

A rule for changing your mind

There is another way to read the formula. Drop the medical setting and treat it as a recipe for changing your mind.

You start with a prior — what you thought the probability of some claim was before any new evidence came in. Evidence arrives. Bayes' theorem tells you exactly how much to adjust. If the evidence is roughly as likely under H as under not-H, your belief shouldn't move much. If the evidence is far more likely under H than under any alternative, you should move a lot. If it is far less likely, you should move the other way.

This is profound for two reasons. First, it gives you a single procedure that works whether you are a scientist evaluating a paper, a doctor evaluating a symptom, a juror evaluating a witness, or a child working out if it is going to rain. Second, it dissolves an old philosophical worry. Hume, writing about induction, had pointed out that no finite amount of evidence can logically prove a universal claim. Bayes does not pretend otherwise. It says: you don't need proof, you need leverage. Each piece of evidence shifts probabilities by definite amounts. Belief slides; it doesn't snap.

“Bayes' theorem is to the theory of probability what Pythagoras's theorem is to geometry.” — Sir Harold Jeffreys

Frequentists and Bayesians

For most of the twentieth century, statistics was dominated by a different tribe — the frequentists. To a frequentist, probability is what happens in the long run if you repeat an experiment many times. The probability of heads is 1/2 because if you flip a coin a million times, about half land heads. This is rigorous, and it is useful, but it has a strange consequence: you cannot really speak of the probability that a hypothesis is true. The hypothesis is either true or it is not. There is nothing to repeat. The p-value was invented to live with that restriction.

Bayesians treat probability differently. To a Bayesian, probability is degree of belief. “P(it will rain tomorrow) = 0.4” doesn't mean we are going to run tomorrow many times. It means your evidence supports rain to that degree, and a rational gambler should set odds accordingly. With this framing, asking “what's the probability my hypothesis is true given the data?” is no longer a strange question. It is the only sensible one. Bayes' theorem is how you compute it.

For most of the twentieth century, the Bayesian view was a minority position — partly because the prior felt subjective (“where do you get a prior from?”), partly because computing posteriors for anything realistic was intractable by hand. The first complaint is real and durable. The second died, in stages, with the rise of cheap computation in the 1990s. Once you can run a million simulations to approximate a posterior, much of modern statistics quietly switches sides. Today every search engine, every spam filter, every autonomous car, every protein-folding model leans heavily on Bayesian machinery.

The mathematics of changing your mind

There is one more thing to notice. Bayes' theorem is not an algorithm for being right. It is an algorithm for being calibrated. If you apply it honestly, your beliefs over a lifetime will be wrong as often as you say they will. The things you call 90% likely will happen 90% of the time. That is the goal — not certainty, but well-tuned uncertainty.

Most everyday reasoning fails this test. We anchor on first impressions, ignore base rates, double-count evidence, treat absence of evidence as evidence of absence (or refuse to). The mistakes are not random. They are systematic, and Kahneman and Tversky catalogued them under the banner of heuristics and biases. Bayes' theorem is, in one sense, simply what you would do if you weren't doing those things.

A reverend's two-page note, scribbled before lithium batteries or vaccines or radio, turns out to be the closest thing we have to a mathematics of changing your mind. Use it on a test result, on a courtroom verdict, on a startup's pitch deck, on a paper you are refereeing. The lesson is the same in each: never ignore the prior, never confuse the conditional with its reverse, and always be willing to move — but only by the amount the evidence actually licenses.