Theory

Theory 1 - Bernoulli, binomial, geometric, Pascal, uniform

In a Bernoulli process, an experiment with binary outcomes is repeated; for example flipping a coin repeatedly. Several discrete random variables may be defined in the context of some Bernoulli process.

Notice that the sample space of a Bernoulli process is infinite: an outcome is any sequence of trial outcomes, e.g. $HTHHTTHHHTTTHHHHTTTT\cdots$

Bernoulli variable

A random variable $X_i$ is a Bernoulli indicator, written $X_i\sim \mathrm{Ber}(p)$, when $X_i$ indicates whether a success event, having probability $p$, took place in trial number $i$ of a Bernoulli process.

Bernoulli PMF:

$$P_{X_i}(k)=\{\begin{array}{cc}p& k=1\\ q& k=0\\ 0& \text{else}\end{array}$$

Here $q=1-p$.

An RV that always gives either $0$ or $1$ for every outcome is called an indicator variable.

Binomial variable

A random variable $S$ is binomial, written $S\sim \text{Bin}(n,p)$, when $S$ counts the number of successes in a Bernoulli process, each having probability $p$, over a specified number $n$ of trials.

Binomial PMF:

$$P_S(k)=\left(\genfrac{}{}{0px}{}{n}{k}\right)p^k{(1-p)}^{n-k}\text{for }k=\mathrm{0,1,2},\dots ,n$$

• For example, if $S\sim \mathrm{Bin}(10,0.2)$, then $P_S(5)$ gives the odds that success happens exactly 5 times over 10 trials, with probability $0.2$ of success for each trial.
• In terms of the Bernoulli indicators, we have: $S=X_1+X_2+\cdots +X_n$
• If $A$ is the success event, then $p=P[A]$ is the success probability, and $q=1-p$ is the failure probability.

---

Geometric variable

A random variable $N$ is geometric, written $N\sim \mathrm{Geom}(p)$, when $N$ counts the discrete wait time in a Bernoulli process until the first success takes place, given that success has probability $p$ in each trial.

Geometric PMF:

$$P_N(k)=q^{k-1}p\text{for }k=\mathrm{1,2,3},\dots$$

Here $q=1-p$.

• For example, if $N\sim \mathrm{Geom}(30\%)$, then $P_N(7)$ gives the probability of getting: failure on the first $6$ trials AND success on the $7^{\text{th}}$ trial.

---

Pascal variable

A random variable $L$ is Pascal, written $L\sim \mathrm{Pasc}(\ell ,p)$, when $L$ counts the discrete wait time in a Bernoulli process until success happens $\ell$ times, given that success has probability $p$ in each trial.

Pascal PMF:

$$P_L(k)=\left(\genfrac{}{}{0px}{}{k-1}{\ell -1}\right){(1-p)}^{k-\ell }{p}^{\ell }\text{for }k=\ell ,\ell +1,\ell +2,\dots$$

• For example, if $L\sim \mathrm{Pasc}(3,0.25)$, then $P_L(8)$ gives the probability of getting: the $3^{\text{rd}}$ success on (precisely) the $8^{\text{th}}$ trial.
• Interpret the formula: $\#$ ways to arrange $2$ successes among $7$ ‘prior’ trials, times the probability of exactly $3$ successes and $5$ failures in one specific sequence.
• The Pascal distribution is also called the negative binomial distribution, e.g. $L\sim \mathrm{Negbin}(\ell ,p)$.

---

Uniform variable

A discrete random variable $X$ is uniform on a finite set $A\subset S$, written $X\sim \mathrm{U}\mathrm{n}\mathrm{i}\mathrm{f}(\mathrm{A})$, when the probability is a fixed constant for outcomes in $A$ and zero for outcomes outside $A$.

Discrete uniform PMF:

$$P_X(k)=\{\begin{array}{cc}{\displaystyle \frac{1}{|A|}}& \text{when }k\in A\\ 0& \text{when }k\notin A\end{array}$$

Continuous uniform PDF:

$$f_X(x)=\{\begin{array}{cc}{\displaystyle \frac{1}{P[A]}}& \text{when }x\in A\\ 0& \text{when }x\notin A\end{array}$$