W10 Homework B

Due date: Sunday 3/22, 11:59pm

Covariance and correlation

01 $★$

04

Correlation between overlapping coin flip sequences

Suppose a coin is flipped 30 times.

Let $X$ count the number of heads among the first 20 flips, and $Y$ count the heads in the last 20.

Find $\rho [X,Y]$.

Hint: Partition the flips into three groups of 10. Create three variables, counting heads, and express $X$ and $Y$ using these. What is the variance of a binomial distribution?

Link to original

Solution

Solutions - 5240-04

(1) Define random variables for partitioning the 30 flips into groups of 10:

Let $A$ be the number of heads in the first 10 flips.

Let $B$ be the number of heads in the middle 10 flips.

Let $C$ be the number of heads in the last 10 flips.

Clearly, $A,B,C\sim \text{Bin}(10,\frac{1}{2})$ and are independent. Note that $X=A+B$ and $Y=B+C$.

---

(2) Compute $E[X]$ and $E[Y]$:

$$\begin{array}{c}E[X]=E[A+B]=E[A]+E[B]=5+5=10\\ \\ E[Y]=E[B+C]=E[B]+E[C]=5+5=10\end{array}$$

---

(3) Compute $E[XY]$:

$$\begin{array}{c}E[XY]=E\left[(A+B)(B+C)\right]\\ \\ \gg \gg E[AB]+E[AC]+E[BC]+E\left[B^2\right]\\ \\ \gg \gg E[A]E[B]+E[A]E[C]+E[B]E[C]+E\left[B^2\right]=75+E\left[B^2\right]\end{array}$$

---

(4) Compute $E\left[B^2\right]$:

Since $E[B]=5$ and $\text{Var}[B]=\frac{5}{2}$, we have $E\left[B^2\right]=5^2+\frac{5}{2}=\frac{55}{2}$.

Thus, $E[XY]=\frac{205}{2}$.

---

(5) Compute $\text{Cov}[X,Y]$:

$$\begin{array}{c}\text{Cov}[X,Y]=E[XY]-E[X]E[Y]=\frac{205}{2}-10\cdot 10=\frac{5}{2}\end{array}$$

---

(6) Compute $\rho [X,Y]$:

$$\begin{array}{c}\rho [X,Y]=\frac{\text{Cov}[X,Y]}{\sqrt{\text{Var}[X]\text{Var}[Y]}}=\frac{\frac{5}{2}}{\sqrt{5\cdot 5}}\gg \gg \frac{1}{2}\end{array}$$Link to original

02 $★$

05

Variance puzzle: indicators

Suppose $A$ and $B$ are events satisfying:

$$P[A]=0.5,P[B]=0.2,P[AB]=0.1$$

Let $X$ count the number of these events that occur. (So the possible values are $X=\mathrm{0,1,2}$.)

Find $\mathrm{Var}[X]$.

Hint: Try setting $X=X_A+X_B$.

Link to original

Solution

Solutions - 5240-05

(1) Define indicator variables $X_A$ and $X_B$:

Let $X_A$ denote whether $A$ occurs and $X_B$ denote whether $B$ occurs. They are independent since $A$ and $B$ are independent ($P[AB]=P[A]P[B]$). Let $X=X_A+X_B$.

---

(2) Compute $\text{Var}[X]$:

$$\begin{array}{cc}\text{Var}[X]& =\text{Var}[X_A+X_B]\\ & =\text{Var}[X_A]+\text{Var}[X_B]\\ & =(E\left[X_A^2\right]-{\left(E[X_A]\right)}^2)+(E\left[X_B^2\right]-{\left(E[X_B]\right)}^2)\\ & =(0.5-{0.5}^2)+(0.2-{0.2}^2)\\ & =0.25+0.16=0.41\end{array}$$Link to original

03

07

Covariance etc. from joint density

Suppose $X$ and $Y$ are random variables with the following joint density:

$$f_{X,Y}(x,y)=\{\begin{array}{cc}\frac{3}{2}(x^2+y^2)& x,y\in [\mathrm{0,1}]\\ 0& \text{ otherwise }\end{array}$$

Compute:

(a) $E[X]$ $$ (b) $E[Y]$ $$ (c) $E[XY]$ $$ (d) $\mathrm{Var}[X]$

(e) $\mathrm{Var}[Y]$ $$ (f) $\mathrm{Cov}[X,Y]$ $$ (g) $\rho [X,Y]$ $$ (h) Are $X$ and $Y$ independent?

(It is worth thinking through which of these can be computed in multiple ways.)

Link to original

Solution

Solutions - 5240-07

(a)

$$\begin{array}{c}E[X]=\frac{3}{2}{\int }_0^1{\int }_0^{1}x(x^2+y^2)dydx\gg \gg \frac{3}{2}{\int }_0^1{x}^3+\frac{x}{3}dx\\ \\ \gg \gg \frac{3}{2}{[\frac{x^4}{4}+\frac{x^2}{6}]}_0^1\gg \gg \frac{3}{2}[\frac{1}{4}+\frac{1}{6}]\gg \gg \frac{5}{8}\end{array}$$

---

(b)

By symmetry, $E[Y]=E[X]$:

$$E[Y]=\frac{5}{8}$$

---

(c)

$$\begin{array}{c}E[XY]=\frac{3}{2}{\int }_0^1{\int }_0^{1}xy(x^2+y^2)dydx\gg \gg \frac{3}{2}{\int }_0^1\frac{x^3}{2}+\frac{x}{4}dx\\ \\ \gg \gg \frac{3}{2}{[\frac{x^4}{8}+\frac{x^2}{8}]}_0^1\gg \gg \frac{3}{2}[\frac{1}{8}+\frac{1}{8}]\gg \gg \frac{3}{8}\end{array}$$

---

(d)

(1) Compute $E\left[X^2\right]$:

$$\begin{array}{c}E\left[X^2\right]=\frac{3}{2}{\int }_0^1{\int }_0^1{x}^2(x^2+y^2)dydx\gg \gg \frac{3}{2}{\int }_0^1{x}^4+\frac{x^2}{3}dx\\ \\ \gg \gg \frac{3}{2}{[\frac{x^5}{5}+\frac{x^3}{9}]}_0^1\gg \gg \frac{3}{2}[\frac{1}{5}+\frac{1}{9}]\gg \gg \frac{7}{15}\end{array}$$

---

(2) Compute $\text{Var}[X]$:

$$\begin{array}{c}\text{Var}[X]=E\left[X^2\right]-{\left(E[X]\right)}^2=\frac{7}{15}-{\left(\frac{5}{8}\right)}^2\gg \gg \frac{73}{960}\end{array}$$

---

(e)

(1) Compute $E\left[Y^2\right]$:

By symmetry, $E\left[Y^2\right]=E\left[X^2\right]=\frac{7}{15}$.

---

(2) Compute $\text{Var}[Y]$:

$$\begin{array}{c}\text{Var}[Y]=\text{Var}[X]\gg \gg \frac{73}{960}\end{array}$$

---

(f)

$$\begin{array}{c}\text{Cov}[X,Y]=E[XY]-E[X]E[Y]=\frac{3}{8}-\frac{25}{64}\gg \gg -\frac{1}{64}\end{array}$$

---

(g)

$$\begin{array}{c}\rho [X,Y]=\frac{\text{Cov}[X,Y]}{\sqrt{\text{Var}[X]\text{Var}[Y]}}=\frac{-\frac{1}{64}}{\sqrt{{\left(\frac{73}{960}\right)}^2}}\gg \gg -\frac{15}{73}\end{array}$$

---

(h)

Since $\text{Cov}[X,Y]\ne 0$, we can conclude that $X$ and $Y$ are not independent.

Link to original

Optional challenge problem

The next problem is purely optional, but worthwhile for the stouthearted. The result is interesting and meaningful.

04

06

When $\rho [X,Y]=1$

Suppose $\rho [X,Y]=1$ for two random variables $X$ and $Y$.

Prove that $Y=aX+b$, where $a={\sigma }_Y/{\sigma }_X$, and find the formula for $b$.

Hint: Study the derivation that $-1\le \rho [X]\le 1$, and think about $E[{(\tilde{X}-\tilde{Y})}^2]$.

(Note: A similar result and argument holds for $\rho [X,Y]=-1$.)

Link to original

Solution

Solutions - 5240-06

(1) Recall the formula for $\rho [X,Y]$:

$$\rho [X,Y]=\frac{\text{Cov}[X,Y]}{{\sigma }_X{\sigma }_Y}$$

Therefore, if $\rho =1$, then $\text{Cov}[X,Y]={\sigma }_X{\sigma }_Y$, and note that $\rho [X,Y]=\text{Cov}[\tilde{X},\tilde{Y}]=1$.

---

(2) Compute $E\left[{(\tilde{X}-\tilde{Y})}^2\right]$:

$$E\left[{(\tilde{X}-\tilde{Y})}^2\right]=E\left[{\tilde{X}}^2\right]-2E\left[\tilde{X}\tilde{Y}\right]+E\left[{\tilde{Y}}^2\right]=1-2+1=0$$

Thus, $\frac{X-{\mu }_X}{{\sigma }_X}=\frac{Y-{\mu }_Y}{{\sigma }_Y}$.

---

(3) Isolate $Y$ in the above equation:

$$Y=\frac{{\sigma }_Y}{{\sigma }_X}X-\frac{{\sigma }_Y}{{\sigma }_X}{\mu }_X+{\mu }_Y$$

Thus, $Y=aX+b$, where $a=\frac{{\sigma }_Y}{{\sigma }_X}$ and $b=-\frac{{\sigma }_Y}{{\sigma }_X}{\mu }_X+{\mu }_Y$.

Link to original

Review

Independent random variables

05

02

Factorizing the density

Consider two joint density functions for $X$ and $Y$:

$$\begin{array}{cc}{f}_1(x,y)& =6e^{-2x}{e}^{-3y}x,y>0,\\ f_2(x,y)& =24xyx,y\in [\mathrm{0,1}],x+y\in [\mathrm{0,1}].\end{array}$$

(Assume the densities are zero outside the given domain.)

Supposing $f_1$ is the joint density, are $X$ and $Y$ independent? Why or why not?

Supposing $f_2$ is the joint density, are $X$ and $Y$ independent? Why or why not?

Link to original

Solution

Solutions - 5200-02

(1) Compute the marginal distribution of $X$ by integrating $f_1$ with respect to $y$:

$$\begin{array}{c}{f}_X(x)={\int }_0^{\infty }6e^{-2x}{e}^{-3y}dy\\ \\ \gg \gg e^{-2x}\underset{b\to \infty }{\lim }{[-2e^{-3y}]}_0^b\gg \gg 2e^{-2x},x>0\end{array}$$

---

(2) Compute the marginal distribution of $Y$ by integrating $f_1$ with respect to $x$:

$$\begin{array}{c}{f}_Y(y)={\int }_0^{\infty }6e^{-2x}{e}^{-3y}dx\gg \gg 3e^{-3y},y>0\end{array}$$

---

(3) Determine independence by multiplying the marginal PDFs:

$$f_X(x)f_Y(y)=6e^{-2x}{e}^{-3y}=f_1$$

Since the product of the marginal PDFs equals the joint PDF, $X$ and $Y$ are independent.

---

(4) Compute the marginal distribution of $X$ by integrating $f_2$ with respect to $y$:

$$\begin{array}{c}{f}_X(x)={\int }_0^{1-x}24xydy\\ \\ \gg \gg 24x{\left[\frac{y^2}{2}\right]|}_0^{1-x}\gg \gg 12x{(1-x)}^2\end{array}$$

---

(5) Compute the marginal distribution of $Y$ by integrating $f_2$ with respect to $x$:

$$\begin{array}{c}{f}_Y(y)={\int }_0^{1-y}24xydx\\ \\ \gg \gg 24y{\left[\frac{x^2}{2}\right]|}_0^{1-y}\gg \gg 12y{(1-y)}^2\end{array}$$

---

(6) Determine independence by multiplying the marginal PDFs:

$$f_X(x)f_Y(y)=144xy{(1-x)}^2{(1-y)}^2\ne 24xy=f_2$$

Therefore, $X$ and $Y$ are not independent.

Link to original