Theory

Theory 1

Exponential variable

A random variable $X$ is exponential, written $X\sim \text{Exp}(\lambda )$, when $X$ measures the wait time until first arrival in a Poisson process with rate $\lambda$.

Exponential PDF:

$$f_X(t)=\{\begin{array}{cc}\lambda e^{-\lambda t}& t\ge 0\\ 0& t<0\end{array}$$

• Poisson is continuous analog of binomial
• Exponential is continuous analog of geometric

---

Notice the coefficient $\lambda$ in $f_X$. This ensures $P[-\infty \le X\le \infty ]=1$:

$${\int }_0^{\infty }{e}^{-\lambda t}dt\gg \gg -{\lambda }^{-1}(e^{-\lambda \cdot \infty }-1)\gg \gg {\lambda }^{-1}$$

---

Notice the “tail probability” is a simple exponential decay:

$$P[X>t]=e^{-\lambda t}$$

(Compute an improper integral to verify this.)

---

Erlang variable

A random variable $X$ is Erlang, written $X\sim \mathrm{Erlang}(\ell ,\lambda )$, when $X$ measures the wait time until ${\ell }^{\text{th}}$ arrival in a Poisson process with rate $\lambda$.

Erlang PDF:

$$f_X(t)=\frac{{\lambda }^{\ell }}{(\ell -1)!}{t}^{\ell -1}{e}^{-\lambda t}$$

• Erlang is continuous analog of Pascal

Theory 2

The memoryless distribution is exponential

The exponential distribution is memoryless. This means that knowledge that an event has not yet occurred does not affect the probability of its occurring in future time intervals:

$$P[X>t+s|X>t]=P[X>s].$$

This is easily checked using the PDF:

$$e^{-\lambda (t+s)}/e^{-\lambda t}=e^{-\lambda s}$$

No other continuous distribution is memoryless. This means any other (continuous) memoryless distribution agrees in probability with the exponential distribution. The reason is that the memoryless property can be rewritten as $P[X>t+s]=P[X>t]P[X>s]$. Consider $P[X>x]$ as a function of $x$, and notice that this function converts sums into products. Only the exponential function can do this.

The geometric distribution is the discrete memoryless distribution.

$$\begin{array}{c}P[X>n]\gg \gg \sum _{k=n+1}^{\infty }{q}^{k-1}p\gg \gg q^{n}p(1+q+q^2+\dots )\\ \\ \gg \gg q^n\frac{p}{1-q}\gg \gg q^n\end{array}$$

and by substituting $n+k$, we also know $P[X>n+k]=q^{n+k}$.

Then:

$$\begin{array}{c}P[X=n+k|X>n]\gg \gg \frac{P[X=n+k]}{P[X>n]}\gg \gg \frac{q^{n+k-1}p}{q^n}\\ \\ \gg \gg q^{k-1}p\gg \gg P[X=k]\end{array}$$

Theory 3

Extra - Inversion of decay rate factor in exponential

For constants $a$ and $\lambda$:

$$\mathrm{Exp}(a\lambda )\sim \frac{1}{a}\mathrm{Exp}(\lambda )$$

Derivation: Let $X\sim \mathrm{Exp}(\lambda )$ and observe that $P[X>t]=e^{-\lambda t}$ (the “tail probability”).

Now observe that:

$$P[a^{-1}X>t]=P[X>at]\gg \gg e^{-\lambda at}$$

Let $Y\sim \mathrm{Exp}(a\lambda )$. So we see that:

$$P[a^{-1}X>t]=P[Y>t]$$

Since the tail event is complementary to the cumulative event, these two distributions have the same CDF, and therefore they are equal.

Theory 4

Extra - Geometric limit to exponential

Divide the waiting time into small intervals. Let $p=\frac{\lambda }{n}$ be the probability of at least one success in the time interval $[a,a+\frac{1}{n}]$ for any $a$. Assume these events are independent.

A random variable $T_n$ measuring the end time of the first interval $[\frac{k-1}{n},\frac{k}{n}]$ containing a success would have a geometric distribution with $\frac{k}{n}$ in place of $k$:

$$P[T_n=\frac{k}{n}]={(1-\frac{\lambda }{n})}^{k-1}\frac{\lambda }{n}$$

By taking the sum of a geometric series, one finds:

$$P[T_n>x]={(1-\frac{\lambda }{n})}^{\lfloor nx\rfloor }$$

Thus $P[T_n>x]\to e^{-\lambda x}$ as $n\to \infty$.