Examples

Minimal MSE estimate given PMF

Suppose $X$ has the following PMF:

$k$	1	2	3	4	5
$P_X(k)$	0.15	0.28	0.26	0.19	0.13

Find the minimal MSE estimate of $X$, given that $X$ is even. What is the error of this estimate?

Solution

The minimal MSE given $A$ is just $E[X|A]$ where $A=\{\mathrm{2,4}\}$.

First compute the conditional PMF:

$$P_{X|A}(k)=\{\begin{array}{cc}0.19/0.47& k=4\\ 0.28/0.47& k=2\\ 0& k\ne \mathrm{2,4}\end{array}$$

Therefore:

$${\hat{x}}_A=2\frac{0.28}{0.47}+4\frac{0.19}{0.47}\approx 2.80851$$

The error is:

$$\begin{array}{c}{e}_{X|A}={(2-2.81)}^2\frac{0.28}{0.47}+{(4-2.81)}^2\frac{0.19}{0.47}\\ \\ \gg \gg \approx 0.9633\end{array}$$

Estimating on a variable interval

Suppose that $R\sim \mathrm{Unif}(\mathrm{0,1})$ and suppose $X\sim \mathrm{Unif}(0,R)$.

(a) Find ${\hat{x}}_M(r)$ $$ (b) Find ${\hat{r}}_M(x)$ $$ (c) Find ${\hat{R}}_{L_{\min }}(X)$

Solution

(a) Find ${\hat{x}}_M(r)$:

We know ${\hat{x}}_M(r)=E[X|R=r]$.

Given $R=r$, so $X$ is uniform on $(0,r)$, we have $E[X|R=r]=\frac{r}{2}$.

$${\hat{x}}_M(r)=\frac{r}{2}$$

---

(b) Find ${\hat{r}}_M(x)$:

We know ${\hat{r}}_M(x)=E[R|X=x]$.

We know $f_R$ and $f_{X|R}$. From these we derive the joint distribution $f_{X,R}$:

$$f_R(r)=\{\begin{array}{cc}1& r\in (\mathrm{0,1})\\ 0& \text{otherwise}\end{array}{f}_{X|R}(x|r)=\{\begin{array}{cc}1/r& x\in (0,r)\\ 0& \text{otherwise}\end{array}$$ $$\gg \gg f_{X,R}(x,r)=f_{X|R}\cdot f_R=\{\begin{array}{cc}1/r& 0<x<r<1\\ 0& \text{else}\end{array}$$

Now extract the marginal $f_X$:

$$\begin{array}{c}\gg \gg f_X(x)={\int }_{-\infty }^{\infty }{f}_{X,R}(x,r)dr\\ \\ \gg \gg {\int }_x^1\frac{1}{r}dr\gg \gg -\ln x(0<x<1)\end{array}$$

Now deduce the conditional $f_{R|X}$:

$$f_{R|X}=\frac{f_{X,R}}{f_X}=\{\begin{array}{cc}\frac{-1}{r\ln x}& 0<x<r<1\\ 0& \text{otherwise}\end{array}$$

Then:

$$\begin{array}{c}E[R|X=x]\gg \gg {\int }_x^{1}r\frac{-1}{r\ln x}dr\\ \\ \gg \gg {\hat{r}}_M(x)=\frac{x-1}{\ln x}\end{array}$$

---

(c) Find ${\hat{R}}_{L_{\min }}(X)$:

We need all the basic statistics.

$E[R]=1/2$ because $R\sim \mathrm{Unif}((\mathrm{0,1}))$.

${\sigma }_R^2=\frac{{(b-a)}^2}{12}=1/12$.

$E[X]=1/4$ using the marginal PDF $f_X(x)=-\ln x$ on $x\in (\mathrm{0,1})$. (IBP and L’Hopital are needed.)

${\sigma }_X=\sqrt{7}/12$ also using the marginal $f_X(x)=-\ln x$.

$E[XR]=1/6$ using $f_{X,R}(x,r)$, namely:

$$\begin{array}{c}E[XR]={\int }_{r=0}^1{\int }_{x=0}^{r}xr\frac{1}{r}dxdr\\ \\ \gg \gg {\int }_0^1\frac{x^2}{2}dx\gg \gg \frac{1}{6}\end{array}$$

From all this we infer $\mathrm{Cov}[X,R]=1/24$ and ${\rho }_{X,R}=\sqrt{3/7}$.

Hence:

$$L_{\min }(x)=\frac{6}{7}x+\frac{2}{7}$$

Thus:

$${\hat{R}}_{L_{\min }}(X)=\frac{6}{7}X+\frac{2}{7}$$