Bayes' theorem

The reverse question: a result A has been observed; how likely is it that it came through a particular case Eᵢ? We are given P(A|Eᵢ) and want P(Eᵢ|A).

Bayes' theorem: if E₁, …, Eₙ form a partition of S and A is an event with P(A) ≠ 0, then for each i, P(Eᵢ|A) = P(Eᵢ)P(A|Eᵢ) ÷ P(A), where P(A) = Σ P(Eⱼ)P(A|Eⱼ) is the total over all n parts.

Proof in two lines: P(Eᵢ|A) = P(A ∩ Eᵢ)/P(A). The numerator is P(Eᵢ)P(A|Eᵢ) by the multiplication rule, and the denominator is P(A) written out by total probability.

On a tree: the branch through Eᵢ that ends in A, divided by the sum of all branches that end in A.

Vocabulary: the Eᵢ are the hypotheses, P(Eᵢ) is the a priori (prior) probability of a hypothesis, and P(Eᵢ|A) is its a posteriori (posterior) probability. The Eᵢ act as possible causes of A, which is why the theorem is also known as the formula for the probability of "causes".

The posterior probabilities P(E₁|A), …, P(Eₙ|A) add to 1.

Bag I has 3 red and 4 black balls, Bag II has 5 red and 6 black. A bag is picked at random and a ball drawn from it is red. P(Bag II | red) = (1/2)(5/11) / [(1/2)(3/7) + (1/2)(5/11)] = 35/68.

The theorem carries the name of Thomas Bayes and appeared in print in 1763, after his death.