Theory

Theory 1

Expectation conditioned by a fixed event

Suppose $X$ is a random variable and $A\subset ℝ$. The expectation of $X$ conditioned on $A$ describes the typical value of $X$ given the hypothesis that $X\in A$ is known.

Discrete case:

$$\begin{array}{cc}E[X|A]& =\sum _{k}k{P}_{X|A}(k)\\ & \\ E[g(X)|A]& =\sum _{k}g(k)P_{X|A}(k)\end{array}$$

Continuous case:

$$\begin{array}{cc}E[X|A]& ={\int }_{-\infty }^{+\infty }x{f}_{X|A}(x)dx\\ & \\ E[g(X)|A]& ={\int }_{-\infty }^{+\infty }g(x)f_{X|A}(x)dx\end{array}$$

Conditional variance:

$$\mathrm{Var}[X|A]=E[(X-{\mu }_{X|A}{)}^2|A]=E[X^2|A]-{\mu }_{X|A}^2$$

Division into Cases / Total Probability applied to expectation:

$$E[X]=E[X|A_1]P[A_1]+\cdots +E[X|A_n]P[A_n]$$

Linearity of conditional expectation:

$$E[a{X}_1+b{X}_2+c|Y=y]=aE[X_1|Y=y]+bE[X_2|Y=y]+c$$

Extra - Proof: Division of Expectation into Cases

We prove the discrete case only.

1. Expectation formula:

$$E[X]=\sum _{k}k{P}_X(k)$$

2. Division into Cases for the PMF:

$$P_X(k)=\sum _{i=1}^n{P}_{X|A_i}(k)P[A_i]$$

3. Substitute in the formula for $E[X]$:

$$\begin{array}{c}\sum _{k}k{P}_X(k)\gg \gg \sum _{k}k\sum _{i=1}^n{P}_{X|A_i}(k)P[A_i]\\ \\ \gg \gg \sum _{i=1}^{n}P[A_i]\sum _{k}k{P}_{X|A_i}(k)\\ \\ \gg \gg \sum _{i=1}^{n}P[A_i]E[X|A_i]\end{array}$$

---

Expectation conditioned by a variable event

Suppose $X$ and $Y$ are any two random variables. The expectation of $X$ conditioned on $Y=y$ describes the typical of value of $X$ in terms of $y$, given the hypothesis that $Y=y$ is known.

Discrete case:

$$\begin{array}{cc}E[X|Y=y]& =\sum _{k}k{P}_{X|Y}(k|y)\text{(}k\text{ over all poss. vals.)}\\ & \\ E[g(X,Y)|Y=y]& =\sum _{k}g(k,y)P_{X|Y}(k|y)\end{array}$$

Continuous case:

$$\begin{array}{cc}E[X|Y=y]& ={\int }_{-\infty }^{+\infty }x{f}_{X|Y}(x|y)dx\\ & \\ E[g(X,Y)|Y=y]& ={\int }_{-\infty }^{+\infty }g(x,y)f_{X|Y}(x|y)dx\end{array}$$

Theory 2

Expectation conditioned by a random variable

Suppose $X$ and $Y$ are any two random variables. The expectation of $X$ conditioned on $Y$ is a random variable giving the typical value of $X$ on the assumption that $Y$ has value determined by an outcome of the experiment.

$$E[X|Y]=g(Y)\text{where}g(y)=E[X|Y=y]$$

In other words, start by defining a function $g(y)$:

$$\begin{array}{cc}g:ℝ& \to ℝ\\ y& \mapsto E[X|Y=y]\end{array}$$

Now $E[X|Y]$ is defined as the composite random variable $g(Y)$.

Considered as a random variable, $E[X|Y]$ takes an outcome $s\in S$, computes $Y(s)$, sets $y=Y(s)$, then returns the expectation of $X$ conditioned on $Y=y$.

Notice that $X$ is not evaluated at $s$, only $Y$ is.

Because the value of $E[X|Y]$ depends only on $Y(s)$, and not on any additional information about $s$, it is common to represent a conditional expectation $E[X|Y]$ using only the function $g$.

---

Iterated Expectation

$$E[E[X|Y]]=E[X]$$

Proof of Iterated Expectation, discrete case

$$\begin{array}{cc}E[E[X|Y]]& =\sum _{\ell }E[X|Y=\ell ]P_Y(\ell )\\ & \\ & =\sum _{\ell }\sum _{k}k{P}_{X|Y}(k|\ell )P_Y(\ell )\\ & \\ & =\sum _{k}k\sum _{\ell }{P}_{X,Y}(k,\ell )\\ & \\ & =\sum _{k}k{P}_X(k)=E[X]\end{array}$$