Examples

Binomial expectation and variance

Suppose we have repeated Bernoulli trials $X_1,\dots ,X_n$ with $X_i\sim \mathrm{Ber}(p)$.

The sum is a binomial variable: $S_n={\displaystyle \sum _{i=1}^n{X}_i}$.

We know $E[X_i]=p$ and $\mathrm{Var}[X_i]=pq$.

The summation rule for expectation:

$$E[S_n]=\sum _{i=1}^{n}E[X_i]\gg \gg \sum _{i=1}^{n}p\gg \gg np$$

The summation rule for variance:

$$\begin{array}{c}\mathrm{Var}[S_n]=\sum _{i=1}^n\mathrm{Var}[X_i]+2\sum _{i<j}\mathrm{Cov}[X_i,X_j]\\ \\ \gg \gg \sum _{i=1}^{n}pq+2\cdot 0\gg \gg npq\end{array}$$

Multinomial covariances

Each trial of an experiment has possible outcomes labeled $1,\dots ,r$ with probabilities of occurrence $p_1,\dots ,p_r$. The experiment is run $n$ times.

Let $X_i$ count the number of occurrences of outcome $i$. So $X_i\sim \mathrm{Bin}(n,p_i)$.

Find $\mathrm{Cov}[X_i,X_j]$.

Solution

Notice that $X_i+X_j$ is also a binomial variable with success probability $p=p_i+p_j$. (‘Success’ is an outcome of either $i$ or $j$. ‘Failure’ is any other value.)

The variance of a binomial is known to be $npq$.

Compute $\mathrm{Cov}[X_i,X_j]$ by solving:

$$\begin{array}{cc}\mathrm{Var}[X_i+X_j]& =\mathrm{Var}[X_i]+\mathrm{Var}[X_j]+2\mathrm{Cov}[X_i,X_j]\\ & \\ n(p_i+p_j)(1-(p_i+p_j))& =n{p}_i(1-p_i)+n{p}_j(1-p_j)+2\mathrm{Cov}[X_i,X_j]\\ & \\ \gg \gg \mathrm{Cov}[X_i,X_j]& =-n{p}_i{p}_j\end{array}$$

Hats in the air

All $n$ sailors throws their hats in the air, and catch a random hat when they fall back down.

(a) How many sailors do you expect will catch the hat they own?

(b) What is the variance of this number?

Solution

Strangely, the answers are both 1, regardless of the number of sailors. Here is the reasoning:

(a) Let $X_i=1$ when sailor $i$ catches their own hat, and $X_i=0$ otherwise. Thus $X_i$ is Bernoulli with $p=1/n$.

Now $X={\sum }_{i=1}^n{X}_i$ counts the total number of hats caught by their owners.

Note that $E[X_i]=1/n$. Therefore:

$$\begin{array}{c}E[X]\gg \gg E\left[\sum _{i=1}^n{X}_i\right]\\ \\ \gg \gg \sum _{i=1}^{n}E[X_i]\gg \gg \sum _{i=1}^n\frac{1}{n}\gg \gg 1\end{array}$$

---

(b) We know:

$$\mathrm{Var}[X]\gg \gg \sum _{i=1}^n\mathrm{Var}[X_i]+2\sum _{i<j}\mathrm{Cov}[X_i,X_j]$$

---

Now calculate $\mathrm{Var}[X_i]$:

Use $\mathrm{Var}[X_i]=E[X_i^2]-E{[X_i]}^2$. Observe that $X_i^2=X_i$. Therefore:

$$\mathrm{Var}[X_i]\gg \gg \frac{1}{n}-\frac{1}{n^2}\gg \gg \frac{n-1}{n^2}$$

---

Now calculate $\mathrm{Cov}[X_i,X_j]$:

$$\mathrm{Cov}[X_i,X_j]=E[X_i{X}_j]-E[X_i]E[X_j]$$

We need to compute $E[X_i{X}_j]$.

Notice that $X_i{X}_j=1$ when $i$ and $j$ both catch their own hats, and 0 otherwise. So it is Bernoulli. Then:

$$\begin{array}{c}P[X_i=1\text{and}{X}_j=1]\gg \gg \frac{1}{n(n-1)}\\ \\ \gg \gg E[X_i{X}_j]=\frac{1}{n(n-1)}\end{array}$$

Therefore:

$$\begin{array}{c}\mathrm{Cov}[X_i,X_j]\gg \gg \frac{1}{n(n-1)}-\frac{1}{n}\cdot \frac{1}{n}\\ \\ \gg \gg \frac{1}{n^2(n-1)}\end{array}$$

---

Putting everything together:

$$\begin{array}{c}\mathrm{Var}[X]\gg \gg \sum _{i=1}^n\mathrm{Var}[X_i]+2\sum _{i<j}\mathrm{Cov}[X_i,X_j]\\ \\ \gg \gg \sum _{i=1}^n\frac{n-1}{n^2}+2\sum _{i<j}\frac{1}{n^2(n-1)}\\ \\ \gg \gg \frac{n-1}{n}+n(n-1)\frac{1}{n^2(n-1)}\gg \gg 1\end{array}$$

Months with a birthday

Suppose study groups of 10 are formed from a large population.

For a typical study group, how many months out of the year contain a birthday of a member of the group? (Assume all 12 months have equal duration.)

Solution

(1) Let $X_i$ be 1 if month $i$ contains a birthday, and 0 otherwise.

So we seek $E[X_1+\cdots +X_{12}]$. This equals $E[X_1]+\cdots +E[X_{12}]$.

The answer will be $12E[X_i]$ because all terms are equal.

---

(2) For a given $i$:

$$P[\text{no birthday in month }i]={\left(\frac{11}{12}\right)}^{10}$$

The complement event:

$$P[\text{at least one birthday in month }i]=1-{\left(\frac{11}{12}\right)}^{10}$$

---

(3) Therefore:

$$12E[X_i]=12(1-{\left(\frac{11}{12}\right)}^{10})\gg \gg 6.97$$

Pascal expectation and variance

(1) Let $X\sim \mathrm{Pasc}(\ell ,p)$.

Let $X_1,X_2,\dots$ be independent random variables, where:

• $X_1$ counts the trials until the first success
• $X_2$ counts the trials after the first success until the second success
• $X_i$ counts the trials after the ${(i-1)}^{th}$ success until the $i^{th}$ success

Observe that $X={\sum }_{i=1}^{\ell }{X}_i$.

---

(2) Notice that $X_i\sim \mathrm{Geom}(p)$ for every $i$. Therefore:

$$E[X_i]=\frac{1}{p}\mathrm{Var}[X_i]=\frac{1-p}{p^2}$$

---

(3) Using the summation rule, conclude:

$$\begin{array}{c}E[X]\gg \gg \sum _{i=1}^{\ell }\frac{1}{p}\gg \gg \frac{\ell }{p}\\ \\ \mathrm{Var}[X]\gg \gg \sum _{i=1}^{\ell }\frac{q}{p^2}\gg \gg \frac{\ell q}{p^2}\end{array}$$