Examples

Conditional PMF, fixed event, expectation

Suppose $X$ measures the lengths of some items and has the following PMF:

$$P_X(k)=\{\begin{array}{cc}0.15& k=\mathrm{1,2,3,4}\\ 0.1& k=\mathrm{5,6,7,8}\\ 0& \text{ otherwise }\end{array}$$

Let $L=\text{“}X\ge 5\text{”}$, an event.

(a) Find the conditional PMF of $X$ given that $L$ is known.

(b) Find the conditional expected value and variance of $X$ given $L$.

Solution

(a)

Conditional PMF formula with $k\in L$ plugged in:

$$P_{X|L}(k)=\{\begin{array}{cc}{\displaystyle \frac{P_X(k)}{P[L]}}& k=\mathrm{5,6,7,8}\\ 0& \text{ otherwise }\end{array}$$

Compute $P[L]$ by adding cases:

$$P[L]=\sum _{k=5}^8{P}_X(k)\gg \gg 0.4$$

Divide nonzero PMF entries by $0.1$:

$$P_{X|L}(k)=\{\begin{array}{cc}0.25& k=\mathrm{5,6,7,8}\\ 0& \text{ otherwise }\end{array}$$

---

(b)

Find $E[X|L]$:

$$\begin{array}{c}E[X|L]=\sum _{k=5}^{8}k{P}_{X|L}(k)\\ \\ \gg \gg 5\cdot (0.25)+6\cdot (0.25)+7\cdot (0.25)+8\cdot (0.25)\\ \\ \gg \gg 6.5\min \end{array}$$

Find $E[X^2|L]$:

$$\begin{array}{c}E[X^2|L]=\sum _{k=5}^8{k}^2{P}_{X|L}(k)\\ \\ \gg \gg 5^2\cdot (0.25)+6^2\cdot (0.25)+7^2\cdot (0.25)+8^2\cdot (0.25)\\ \\ \gg \gg 43.5{\min }^2\end{array}$$

Find $\mathrm{Var}[X|L]$ using “short form” with conditioning:

$$\mathrm{Var}[X|L]=E[X^2|L]-E{[X|L]}^2\gg \gg 1.25{\min }^2$$

Conditional PMF, variable event, via joint density

Suppose $X$ and $Y$ have joint PMF given by:

$$P_{X,Y}(k,\ell )=\{\begin{array}{cc}{\displaystyle \frac{k+\ell }{21}}& k=\mathrm{1,2,3};\ell =\mathrm{1,2}\\ 0& \text{otherwise}\end{array}$$

Find $P_{X|Y}(k|\ell )$ and $P_{Y|X}(\ell ,k)$.

Solution

Marginal PMFs:

$$P_X(k)=\frac{2k+3}{21},k=\mathrm{1,2,3}$$ $$P_Y(\ell )=\frac{\ell +2}{7},\ell =\mathrm{1,2}$$

Assuming $\ell =1$ or $2$, for each $k=1,2,3$ we have:

$$P_{X|Y}(k|\ell )=\frac{P_{X,Y}(k,\ell )}{P_Y(\ell )}\gg \gg \frac{k+\ell }{3\ell +6}$$

Assuming $k=1$, $2$, or $3$, for each $\ell =\mathrm{1,2}$ we have:

$$P_{Y|X}(\ell |k)=\frac{P_{Y,X}(\ell ,k)}{P_X(k)}\gg \gg \frac{k+\ell }{2k+3}$$

Proof of Iterated Expectation, continuous case

Prove Iterated Expectation for the continuous case.

Conditional expectations from joint density

Suppose $X$ and $Y$ are random variables with joint density given by:

$$f_{X,Y}(x,y)=\{\begin{array}{cc}{\displaystyle \frac{1}{y}{e}^{-x/y}{e}^{-y}}& x,y\in (0,\infty )\\ 0& \text{otherwise}\end{array}$$

Find $E[X|Y=y]$. Use this to compute $E[X]$.

Solution

(1) Derive the marginal density $f_Y(y)$:

$$\begin{array}{c}{f}_Y(y)\gg \gg {\int }_0^{+\infty }\frac{1}{y}{e}^{-x/y}{e}^{-y}dx\\ \\ \gg \gg -e^{-x/y}{e}^{-y}{|}_{x=0}^{\infty }\gg \gg e^{-y}\end{array}$$

---

(2) Use $f_Y(y)$ to compute $f_{X|Y}(x|y)$:

$$\begin{array}{c}{f}_{X|Y}(x|y)\gg \gg \frac{f_{X,Y}(x,y)}{f_Y(y)}\\ \\ \gg \gg \frac{1}{y}{e}^{-x/y}{e}^{-y}\cdot {(e^{-y})}^{-1}\gg \gg \frac{1}{y}{e}^{-x/y}\end{array}$$

---

(3) Use $f_{X|Y}(x|y)$ to calculate expectation conditioned on the variable event:

$$\begin{array}{c}E[X|Y=y]\gg \gg {\int }_{-\infty }^{+\infty }x{f}_{X|Y}(x|y)dx\\ \\ \gg \gg {\int }_0^{\infty }\frac{x}{y}{e}^{-x/y}dx\gg \gg y\end{array}$$

---

(4) Apply Iterated Expectation:

Set $g(y)=y$. By Iterated Expectation, we know that $E[X]=E[g(Y)]$. Therefore:

$$\begin{array}{c}E[X]=E[g(Y)]={\int }_{-\infty }^{+\infty }g(y)f_Y(y)dy\\ \\ \gg \gg {\int }_0^{+\infty }y{e}^{-y}dy\gg \gg 1\end{array}$$

Notice that $g(Y)=Y$, so $E[X|Y]=Y$, and Iterated Expectation says that $E[X]=E[Y]$.

Flip coin, choose RV

Suppose $X\sim \mathrm{Ber}(1/3)$ and $Y\sim \mathrm{Ber}(1/4)$ represent two biased coins, giving 1 for heads and 0 for tails.

Here is the experiment:

1. Flip a fair coin.
2. If heads, flip the $X$ coin; if tails, flip the $Y$ coin.
3. Record the outcome as $Z$.

What is $E[Z]$?

Solution

Let $G\sim \mathrm{Ber}(1/2)$ describe the fair coin. Then:

$$\begin{array}{c}E[Z]=E[E[Z|G]]\\ \\ \gg \gg E[Z|G=0]P_G(0)+E[Z|G=1]P_G(1)\\ \\ \gg \gg E[Y]P_G(0)+E[X]P_G(1)\\ \\ \gg \gg \frac{1}{4}\cdot \frac{1}{2}+\frac{1}{3}\cdot \frac{1}{2}\gg \gg \frac{7}{24}\end{array}$$

Sum of random number of RVs

Let $N$ denote the number of customers that enter a store on a given day.

Let $X_i$ denote the amount spent by the $i^{th}$ customer.

Assume that $E[N]=50$ and E[X_i]=\ ParseError: Unexpected character: '\' at position 8: E[X_i]=\̲8$foreach$i$.

What is the expected total spend of all customers in a day?

Solution

A formula for the total spend is $X={\sum }_{i=1}^N{X}_i$.

By Iterated Expectation, we know $E[X]=E[E[X|N]]$.

Now compute $E[X|N]$ as a function of $N$:

$$\begin{array}{cc}E[X|N=n]& \gg \gg E[\left(\sum _{i=1}^N{X}_i\right)|N=n]\\ & \\ & \gg \gg E[\left(\sum _{i=1}^n{X}_i\right)|N=n]\\ & \\ & \gg \gg \sum _{i=1}^{n}E[X_i|N=n]\\ & \\ & \gg \gg \sum _{i=1}^{n}E[X_i]\gg \gg 8n\end{array}$$

Therefore $g(n)=8n$ and $g(N)=8N$ and $E[X|N]=8N$.

Then by Iterated Expectation, E[X]=E[8N]=8E[N]=\ ParseError: Unexpected character: '\' at position 18: …X]=E[8N]=8E[N]=\̲400$.