Theory

Theory 1

Expectation for a function on two variables

Discrete case:

Continuous case:

These formulas are not trivial to prove, and we omit the proofs. (Recall the technical nature of the proof we gave for in the discrete case.)

Expectation sum rule

Suppose and are any two random variables on the same probability model.

Then:

We already know that expectation is linear in a single variable: .

Therefore this two-variable formula implies:

Expectation product rule: independence

Suppose that and are independent.

Then we have:

Extra - Proof: Expectation sum rule, continuous case

Suppose and give marginal PDFs for and , and gives their joint PDF.

Then:

Observe that this calculation relies on the formula for , specifically with .

Extra - Proof: Expectation product rule