Theory

Theory 1

Axioms of probability

A probability measure is a function $P:ℱ\to ℝ$ satisfying:

Kolmogorov Axioms:

• Axiom 1: $P[A]\ge 0$ for every event $A$ (probabilities are not negative!)

• Axiom 2: $P[S]=1$ (probability of “anything” happening is 1)

• Axiom 3: additivity for any countable collection of mutually exclusive events:

$$P[A_1\cup A_2\cup A_3\cup \cdots ]=P[A_1]+P[A_2]+P[A_3]+\cdots$$ $$\text{when:}{A}_i\cap A_j=\varnothing \text{for all }i\ne j$$

Notation: we write $P[A]$ instead of $P(A)$, even though $P$ is a function, to emphasize the fact that $A$ is a set.

Probability model

A probability model or probability space consists of a triple $(S,ℱ,P)$:

• $S$ the sample space

• $ℱ$ the set of valid events, where every $A\in ℱ$ satisfies $A\subset S$

• $P:ℱ\to ℝ$ a probability measure satisfying the Kolmogorov Axioms

Finitely many exclusive events

It is a consequence of the Kolmogorov Axioms that additivity also works for finite collections of mutually exclusive events:

$$\begin{array}{c}P[A\cup B]=P[A]+P[B]\\ \\ P[A_1\cup \cdots \cup A_n]=P[A_1]+\cdots +P[A_n]\end{array}$$

Inferences from Kolmogorov

A probability measure satisfies these rules. They can be deduced from the Kolmogorov Axioms.

• Negation: Can you find $P[A^c]$ but not $P[A]$? Use negation:

$$P[A]=1-P[A^c]$$

• Monotonicity: Probabilities grow when outcomes are added:

$$A\subset B\gg \gg P[A]\le P[B]$$

• Inclusion-Exclusion: A trick for resolving unions:

$$P[A\cup B]=P[A]+P[B]-P[A\cap B]$$

(even when $A$ and $B$ are not exclusive!)

Inclusion-Exclusion

The principle of inclusion-exclusion generalizes to three events:

$$\begin{array}{c}P[A\cup B\cup C]=\\ \\ P[A]+P[B]+P[C]-P[A\cap B]-P[A\cap C]-P[B\cap C]+P[A\cap B\cap C]\end{array}$$

The same pattern works for any number of events!

The pattern goes: “include singles” then “exclude doubles” then “include triples” then …

Include, exclude, include, exclude, include, …