Bayes' Theorem
16 min
Bayes' theorem flips a conditional probability around: it lets you compute P(A|B) — "given that B happened, what's the probability A is true?" — from P(B|A), the reverse and often much easier direction to measure directly.
P(A|B) = P(B|A) × P(A) / P(B)
Classic use case: you know how accurate a medical test is (P(positive test | actually sick)), but what you actually want to know when someone tests positive is P(actually sick | positive test) — a different question entirely, and Bayes' theorem is the bridge between them.
This is the single most common intuition-breaking result in probability: even an accurate test gives a surprisingly low P(disease | positive) when the disease itself is rare — because false positives from the huge healthy population outnumber true positives from the tiny sick population.
In the example above, P(B) — the overall probability of testing positive — was just given to you. In practice you usually have to compute it yourself from the law of total probability: sum P(B|A)×P(A) over every possible cause of B.
For a yes/no cause A, that's just two terms — B happening because A is true, plus B happening even though A is false:
def bayes_full(p_a, p_b_given_a, p_b_given_not_a):
p_b = p_b_given_a * p_a + p_b_given_not_a * (1 - p_a)
return p_b_given_a * p_a / p_b
This version only needs the test's true-positive rate and false-positive rate — you never have to separately go measure "what fraction of ALL tests come back positive," which is often the harder number to get.
Write `bayes(p_a, p_b_given_a, p_b)`, returning P(A|B) using Bayes' theorem: P(A|B) = P(B|A) × P(A) / P(B).
Write `bayes_full(p_a, p_b_given_a, p_b_given_not_a)`. First compute P(B) using the law of total probability — P(B) = P(B|A)×P(A) + P(B|¬A)×(1−P(A)) — then return P(A|B).
What does Bayes' theorem let you compute?
You can apply Bayes' theorem directly, and derive P(B) yourself via the law of total probability when it isn't given.