Theory

Theory 1

Conditional distribution - fixed event

Suppose $X$ is a random variable, and suppose $A\subset ℝ$. The distribution of $X$ conditioned on $A$ describes the probabilities of values of $X$ given knowledge that $X\in A$.

Discrete case:

$$P_{X|A}(k)=\{\begin{array}{cc}{\displaystyle \frac{1}{P[A]}{P}_X(k)}& k\in A\\ & \\ 0& k\notin A\end{array}$$

Continuous case:

$$f_{X|A}(x)=\{\begin{array}{cc}{\displaystyle \frac{1}{P[A]}{f}_X(x)}& x\in A\\ & \\ 0& x\notin A\end{array}$$

There is also a conditional CDF, of which this conditional PDF is the derivative:

$$F_{X|A}(x)=P[X\le x|A],f_{X|A}(x)=\frac{d}{dx}{F}_{X|A}(x)$$

The Law of Total Probability has versions for distributions:

$$\begin{array}{cc}{P}_X(k)& =P_{X|A_1}(k)P[A_1]+\cdots +P_{X|A_n}(k)P[A_n]\\ & \\ f_X(x)& =f_{X|A_1}(x)P[A_1]+\cdots +f_{X|A_n}(x)P[A_n]\end{array}$$

Conditional distribution - variable event

Suppose $X$ and $Y$ are any two random variables. The distribution of $X$ conditioned on $Y$ describes the probabilities of values of $X$ in terms of $y$, given knowledge that $Y=y$.

Discrete case:

$$\begin{array}{cc}{P}_{X|Y}(k|\ell )& =P[X=k|Y=\ell ]\\ & \\ & =\frac{P_{X,Y}(k,\ell )}{P_Y(\ell )}(\text{assuming }{P}_Y(\ell )\ne 0)\end{array}$$

Continuous case:

$$f_{X|Y}(x|y)=\frac{f_{X,Y}(x,y)}{f_Y(y)}(\text{assuming }{f}_Y(y)\ne 0)$$

Remember: $P_{X,Y}(k,\ell )$ is the probability that “$X=k$ and $Y=\ell$.”

Sometimes it is useful to have the formulas rewritten like this:

$$\begin{array}{cc}{P}_{X,Y}(k,\ell )& =P_{X|Y}(k|\ell )P_Y(\ell )\\ & \\ f_{X,Y}(x,y)& =f_{X|Y}(x|y)f_Y(y)\end{array}$$

Extra - Deriving $f_{X|Y}(x|y)$

The density $f_{X|Y}$ ought to be such that $f_{X|Y}(x|y)dx$ gives the probability of $X\in [x,x+dx]$, given knowledge that $Y\in [y,y+dy]$. Calculate this probability:

$$\begin{array}{c}P[x\le X\le x+dx|y\le Y\le y+dy]\\ \\ \gg \gg \frac{P[x\le X\le x+dx,y\le Y\le y+dy]}{P[y\le Y\le y+dy]}\\ \\ \gg \gg \frac{f_{X,Y}(x,y)dxdy}{f_Y(y)dy}\\ \\ \gg \gg \frac{f_{X,Y}(x,y)}{f_Y(y)}dx\end{array}$$