Theory

Theory 1

Normal distribution

A variable $X$ has a normal distribution, written $X\sim 𝒩(\mu ,{\sigma }^2)$ or “$X$ is Gaussian $(\mu ,\sigma )$,” when it has PDF given by:

$$f_X(x)=\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-{(x-\mu )}^2/2{\sigma }^2}$$

The standard normal is $Z\sim 𝒩(\mathrm{0,1})$ and its PDF is usually denoted by $\phi (x)$:

$$\phi (x)=\frac{1}{\sqrt{2\pi }}{e}^{-x^2/2}$$

The standard normal CDF is usually denoted by $\mathrm{\Phi }(z)$:

$$\mathrm{\Phi }(z)={\int }_{-\infty }^z\frac{1}{\sqrt{2\pi }}{e}^{-u^2/2}du$$

• To show that $\phi (x)$ is a valid probability density, we must show that ${\int }_{-\infty }^{+\infty }\phi (x)dx=1$.
  • This calculation is not trivial; it requires a double integral in polar coordinates!
• There is no explicit antiderivative of $\phi$
  • A computer is needed for numerical calculations.
  • A chart of approximate values of $\mathrm{\Phi }$ is provided for exams.

---

• To check that $E[Z]=0$:
  • Observe that $x\phi (x)$ is an odd function, i.e. symmetric about the $y$-axis.
  • One must then simply verify that the improper integral converges.
• To check that $\mathrm{Var}[Z]=1$:
  • Since $\mu =E[Z]=0$, we find:

$$\mathrm{Var}[Z]=E[Z^2]\gg \gg \frac{1}{\sqrt{2\pi }}{\int }_{-\infty }^{+\infty }{x}^2{e}^{-x^2/2}dx=:I$$

• Use integration by parts to compute that $I=1$. (Select $u=x$ and $dv=x{e}^{-x^2/2}dx$.)

---

General and standard normals

Assume that $Z\sim 𝒩(\mathrm{0,1})$ and $\sigma ,\mu$ are constants. Define $X=\sigma Z+\mu$. Then:

$$f_X=\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-{(x-\mu )}^2/2{\sigma }^2}$$

That is, $\sigma Z+\mu$ has the distribution type $𝒩(\mu ,{\sigma }^2)$.

Derivation of PDF of $\sigma Z+\mu$

Suppose that $X=\sigma Z+\mu$. Then:

$$\begin{array}{cc}{F}_X(x)& =P[X\le x]\\ & \\ & =P[\sigma Z+\mu \le x]\\ & \\ & =P[Z\le \frac{x-\mu }{\sigma }]\\ & \\ & =\mathrm{\Phi }\left(\frac{x-\mu }{\sigma }\right)\end{array}$$

Differentiate to find $f_X$:

$$\begin{array}{cc}{f}_X(x)& =\frac{d}{dx}{F}_X(x)\\ & \\ & =\frac{d}{dx}\mathrm{\Phi }\left(\frac{x-\mu }{\sigma }\right)\\ & \\ & =\frac{1}{\sigma }\phi \left(\frac{x-\mu }{\sigma }\right)\\ & \\ & =\frac{1}{\sqrt{2\pi {\sigma }^2}}{e}^{-{(x-\mu )}^2/2{\sigma }^2}\end{array}$$

From this fact we can infer that $E[X]=\mu$ and $\mathrm{Var}[X]={\sigma }^2$ whenever $X\sim 𝒩(\mu ,{\sigma }^2)$.