Theory

Theory 1

Independent random variables

Random variables $X,Y$ are independent when they satisfy the product rule for all valid subsets $B_1,B_2\subset ℝ$:

$$P[X\in B_1,Y\in B_2]=P[X\in B_1]\cdot P[Y\in B_2]$$

Since $\{X\in B_1,Y\in B_2\}=\{X\in B_1\}\cap \{Y\in B_2\}$, this definition is equivalent to independence of all events constructible using the variables $X$ and $Y$.

For discrete random variables, it is enough to check independence for simple events of type $\{X=k\}$ and $\{Y=\ell \}$ for $k$ and $\ell$ any possible values of $X$ and $Y$.

---

The independence criterion for random variables can be cast entirely in terms of their distributions and written using the PMFs or PDFs.

Independence using PMF and PDF

Discrete case:

$$P_{X,Y}(x,y)=P_X(x)\cdot P_Y(y)$$

Continuous case:

$$f_{X,Y}(x,y)=f_X(x)\cdot f_Y(y)$$

Independence via joint CDF

Random variables $X$ and $Y$ are independent when their CDFs obey the product rule:

$$F_{X,Y}(x,y)=F_X(x)\cdot F_Y(y)$$