Examples

ML test: Smoke detector

Suppose that a smoke detector sensor is configured to produce $8\mathrm{V}$ when there is smoke, and $0\mathrm{V}$ otherwise. But there is background noise with distribution $𝒩(0,3^2)$.

Design an ML test for the detector electronics to decide whether to activate the alarm.

What are the three error probabilities? (Type I, Type II, Total.)

Solution

First, establish the conditional distributions:

$$X|H_0\sim 𝒩(0,3^2)X|H_1\sim 𝒩(8,3^2)$$

Density functions:

$$f_{X|H_0}=\frac{1}{\sqrt{2\pi 9}}{e}^{-\frac{1}{2}{\left(\frac{x-0}{3}\right)}^2}{f}_{X|H_1}=\frac{1}{\sqrt{2\pi 9}}{e}^{-\frac{1}{2}{\left(\frac{x-8}{3}\right)}^2}$$

---

The ML condition becomes:

$$\begin{array}{c}\frac{1}{\sqrt{2\pi 9}}{e}^{-\frac{1}{2}{\left(\frac{x-0}{3}\right)}^2}\stackrel{?}{\ge }\frac{1}{\sqrt{2\pi 9}}{e}^{-\frac{1}{2}{\left(\frac{x-8}{3}\right)}^2}\\ \\ \gg \gg -\frac{1}{2}{\left(\frac{x-0}{3}\right)}^2\stackrel{?}{\ge }-\frac{1}{2}{\left(\frac{x-8}{3}\right)}^2\\ \\ \gg \gg x^2\stackrel{?}{\le }{(x-8)}^2\\ \\ \gg \gg x\le 4\end{array}$$

---

Therefore, $A_0$ is $x\le 4$, while $A_1$ is $x>4$.

The decision rule is: activate alarm when $x>4$.

---

Type I error:

$$\begin{array}{c}{P}_{FA}=P[A_1|H_0]\gg \gg P[X>4|H_0]\\ \\ \gg \gg 1-P[\frac{X-0}{3}\le \frac{4}{3}|H_0]\\ \\ \gg \gg 1-P[Z\le 1.3333]\gg \gg \approx 0.0912\end{array}$$

Type II error:

$$\begin{array}{c}{P}_{\text{Miss}}=P[A_0|H_1]\gg \gg P[X\le 4|H_1]\\ \\ \gg \gg P[\frac{X-8}{3}\le \frac{4-8}{3}|H_1]\\ \\ \gg \gg P[Z\le -1.3333]\gg \gg \approx 0.0912\end{array}$$

Total error:

$$P_{\mathrm{ERR}}=P_{FA}\cdot 0.5+P_{\text{Miss}}\cdot 0.5\gg \gg \approx 0.0912$$

MAP test: Smoke detector

Suppose that a smoke detector sensor is configured to produce $8\mathrm{V}$ when there is smoke, and $0\mathrm{V}$ otherwise. But there is background noise with distribution $𝒩(0,3^2)$.

Suppose that the background chance of smoke is $5\%$. Design a MAP test for the alarm.

What are the three error probabilities? (Type I, Type II, Total.)

Solution

First, establish priors:

$$P[H_0]=0.95P[H_1]=0.05$$

The MAP condition becomes:

$$\begin{array}{c}\frac{1}{\sqrt{2\pi 9}}{e}^{-\frac{1}{2}{\left(\frac{x-0}{3}\right)}^2}\cdot \text{0}\text{.}\text{9}\text{5}\stackrel{?}{\ge }\frac{1}{\sqrt{2\pi 9}}{e}^{-\frac{1}{2}{\left(\frac{x-8}{3}\right)}^2}\cdot \text{0}\text{.}\text{0}\text{5}\\ \\ \gg \gg e^{-\frac{1}{2}{\left(\frac{x-0}{3}\right)}^2}\stackrel{?}{\ge }{e}^{-\frac{1}{2}{\left(\frac{x-8}{3}\right)}^2}\cdot \frac{0.05}{0.95}\\ \\ \gg \gg -\frac{1}{2}{\left(\frac{x-0}{3}\right)}^2\stackrel{?}{\ge }-\frac{1}{2}{\left(\frac{x-8}{3}\right)}^2+\ln \left(\frac{0.05}{0.95}\right)\\ \\ \gg \gg x^2\stackrel{?}{\le }{(x-8)}^2-18\ln \left(\frac{0.05}{0.95}\right)\\ \\ \gg \gg x\le 7.31\end{array}$$

---

Therefore, $A_0$ is $x\le 7.31$, while $A_1$ is $x>7.31$.

The decision rule is: activate alarm when $x>7.31$.

---

Type I error:

$$\begin{array}{c}{P}_{FA}=P[A_1|H_0]\gg \gg P[X>7.31|H_0]\\ \\ \gg \gg 1-P[Z\le 2.4367]\gg \gg \approx 0.007411\end{array}$$

Type II error:

$$\begin{array}{c}{P}_{\text{Miss}}=P[A_0|H_1]\gg \gg P[X\le 7.31|H_1]\\ \\ \gg \gg P[Z\le -0.23]\gg \gg \approx 0.4090\end{array}$$

Total error:

$$P_{\mathrm{ERR}}=P_{FA}\cdot 0.95+P_{\text{Miss}}\cdot 0.05\approx 0.02749$$

MC Test: Smoke detector

Suppose that a smoke detector sensor is configured to produce $8\mathrm{V}$ when there is smoke, and $0\mathrm{V}$ otherwise. But there is background noise with distribution $𝒩(0,3^2)$.

Suppose that the background chance of smoke is $5\%$. Suppose the cost of a miss is $50\times$ the cost of a false alarm. Design an MC test for the alarm.

Compute the expected cost.

Solution

We have priors:

$$P[H_0]=0.95P[H_1]=0.05$$

And we have costs:

$$C_{10}=1C_{01}=50$$

(The ratio of these numbers is all that matters in the inequalities of the condition.)

The MC condition becomes:

$$\begin{array}{c}\frac{1}{\sqrt{2\pi 9}}{e}^{-\frac{1}{2}{\left(\frac{x-0}{3}\right)}^2}\cdot 0.95\cdot 1\stackrel{?}{\ge }\frac{1}{\sqrt{2\pi 9}}{e}^{-\frac{1}{2}{\left(\frac{x-8}{3}\right)}^2}\cdot 0.05\cdot 50\\ \\ \gg \gg e^{-\frac{1}{2}{\left(\frac{x-0}{3}\right)}^2}\stackrel{?}{\ge }{e}^{-\frac{1}{2}{\left(\frac{x-8}{3}\right)}^2}\cdot \frac{2.5}{0.95}\\ \\ \gg \gg -\frac{1}{2}{\left(\frac{x-0}{3}\right)}^2\stackrel{?}{\ge }-\frac{1}{2}{\left(\frac{x-8}{3}\right)}^2+\ln \left(\frac{2.5}{0.95}\right)\\ \\ \gg \gg x^2\stackrel{?}{\le }{(x-8)}^2-18\ln \left(\frac{2.5}{0.95}\right)\\ \\ \gg \gg x\le 2.91\end{array}$$

---

Therefore, $A_0$ is $x\le 2.91$, while $A_1$ is $x>2.91$.

The decision rule is: activate alarm when $x>2.91$.

---

Type I error:

$$\begin{array}{c}{P}_{FA}=P[A_1|H_0]\\ \\ \gg \gg P[X>2.91|H_0]\gg \gg \approx 0.1660\end{array}$$

Type II error:

$$\begin{array}{c}{P}_{\text{Miss}}=P[A_0|H_1]\\ \\ \gg \gg P[X\le 2.91]\gg \gg \approx 0.04488\end{array}$$

Total error:

$$P_{\mathrm{ERR}}=P_{FA}\cdot 0.95+P_{\text{Miss}}\cdot 0.05\approx 0.1599$$

---

PMF of total cost:

$$P_C(c)=\{\begin{array}{cc}0.002244& c=50\\ 0.1577& c=1\\ 0.840056& c=0\end{array}$$

Therefore $E[C]=0.27$.