Theory

Theory 1

In many contexts it is useful to consider random variables that are summations of a large number of variables.

Summation formulas: $E[X]$ and $\mathrm{Var}[X]$

Suppose $X$ is a large sum of random variables:

$$X=X_1+X_2+\cdots +X_n$$

Then:

$$\begin{array}{c}E[X]=E[X_1]+E[X_2]+\cdots +E[X_n]\\ \\ \mathrm{Var}[X]=\mathrm{Var}[X_1]+\cdots +\mathrm{Var}[X_n]+2\sum _{i<j}\mathrm{Cov}[X_i,X_j]\end{array}$$

If $X_i$ and $X_j$ are uncorrelated (e.g. if they are independent):

$$\mathrm{Var}[X]=\mathrm{Var}[X_1]+\cdots +\mathrm{Var}[X_n]$$

Extra - Derivation of variance of a sum

Using the definition:

$$\begin{array}{cc}\mathrm{Var}[X_1+\cdots +X_n]& =E[(X_1+\cdots +X_n-({\mu }_{X_1}+\cdots +{\mu }_{X_n}){)}^2]\\ & \\ & =E\left[\right((X_1-{\mu }_{X_1})+\cdots +(X_n-{\mu }_{X_n}){)}^2]\\ & \\ & =E[\sum _{i,j}(X_i-{\mu }_{X_i})(X_j-{\mu }_{X_j})]\\ & \\ & =\sum _{i,j}\mathrm{Cov}(X_i,X_j)\\ & \\ & =\sum _i\mathrm{Var}[X_i]+2\sum _{i<j}\mathrm{Cov}[X_i,X_j]\end{array}$$

In the last line we use the fact that $\mathrm{Cov}[X,X]=\mathrm{Var}[X]$ for the first term, and the symmetry property of covariance for the second term with the factor of 2.