Week 11 notes

Recall some items related to conditional probability.

Conditioning definition: Multiplication rule: Division into Cases / Total Probability:

Conditional distribution

01 Theory

Conditional distribution, by a fixed event

Suppose is a random variable and . The distribution of conditioned on describes the probabilities of values of given the hypothesis that is known.

Discrete case:

Continuous case:

We can also condition the CDF directly and derive the PDF from the CDF:

We can also translate Division into Cases / Total Probability into distributional terms:

---

Conditional distribution, by a variable event

Suppose and are any two random variables. The distribution of conditioned on describes the probabilities of values of in terms of , given the hypothesis that is known.

Discrete case:

Continuous case:

Notice:

• is the probability of “ and .”
• is the probability of , given the hypothesis that is known.

Sometimes it is useful to rewrite the formulas this way, for example to describe a “continuous probability tree:”

Extra - Deriving

The density ought to be such that gives the probability of , on the hypothesis that is known. Calculate this probability:

Conditional expectation

02 Theory

Expectation conditioned by a fixed event

Suppose is a random variable and . The expectation of conditioned on describes the typical value of given the hypothesis that is known.

Discrete case:

Continuous case:

Conditional variance:

Division into Cases / Total Probability applied to expectation:

Linearity of conditional expectation:

Extra - Proof: Division of Expectation into Cases

We prove the discrete case only.

1. Expectation formula:
2. Division into Cases for the PMF:
3. Substitute in the formula for :

---

Expectation conditioned by a variable event

Suppose and are any two random variables. The expectation of conditioned on describes the typical of value of in terms of , given the hypothesis that is known.

Discrete case:

Continuous case:

03 Illustration

Example - Conditioning on a fixed event

Example - Conditioning on variable events, discrete PMF function

04 Theory

Expectation conditioned by a random variable

Suppose and are any two random variables. The expectation of conditioned on is a random variable giving the typical value of on the assumption that has value determined by an outcome of the experiment.

In other words, start by defining a function :

Now is defined as the composite random variable .

Considered as a random variable, takes an outcome , computes , sets , then returns the expectation of conditioned on .

Notice that is not evaluated at , only is.

Because the value of depends only on , and not on any additional information about , it is common to represent a conditional expectation using only the function .

---

Iterated Expectation

Proof of Iterated Expectation, discrete case

05 Illustration

Exercise - Proof of Iterated Expectation, continuous case

Example - Conditional expectations from joint density

Example - Flip coin, choose RV

Example - Sum of random number of RVs