Theory

Theory 1

Two events are independent when information about one of them does not change our probability estimate for the other.

Independence

Events $A$ and $B$ are independent when these (logically equivalent) equations hold:

• $P[B|A]=P[B]$
• $P[A|B]=P[A]$
• $P[BA]=P[B]\cdot P[A]$

Note that the last equation is symmetric in $A$ and $B$:

• Check: $BA=AB$ and $P[B]\cdot P[A]=P[A]\cdot P[B]$
• This symmetric version is the preferred definition of the concept of independence.

Multiple-independence

A collection of events $A_1,\dots ,A_n$ is mutually independent when every subcollection $A_{i_1},\dots ,A_{i_k}$ satisfies:

$$P[A_{i_1}\cdots A_{i_k}]=P[A_{i_1}]\cdots P[A_{i_k}]$$

A potentially weaker condition for a collection $A_1,\dots ,A_n$ is called pairwise independence, which holds when all 2-member subcollections are independent:

$$P[A_i{A}_j]=P[A_i]\cdot P[A_j]\text{for all}i\ne j$$

One could also define $3$-member independence, or $n$-member independence. Plain ‘independence’ means any-member independence.