Week 05 notes

Discrete families: summary

01 Theory

Memorize this info!

Bernoulli:

• Indicates a win.

Binomial:

• Counts number of wins.
• These are times the Bernoulli numbers.

Geometric:

• Counts discrete wait time until first win.

Pascal:

• Counts discrete wait time until win.
• These are times the Geometric numbers.

Poisson:

• Counts “arrivals” during time interval.

Function on a random variable

02 Theory

By composing any function with a random variable we obtain a new random variable . The new one is called a derived random variable.

• %& Write for this derived random variable .

Expectation of derived variables

Discrete case: (Here the sum is over all possible values of .)

Continuous case:

• ! Notice: when applied to outcome :
  • is the output of
  • is the output of

The proofs of these formulas are not trivial, since one must relate the PDF or PMF of to that of .

Proof - Discrete case - Expectation of derived variable

---

Linearity of expectation

For constants and :

For any and on the same probability model:

Exercise - Linearity of expectation

Using the definition of expectation, verify both linearity formulas for the discrete case.

Be careful!

Usually .

For example, usually . We distribute over sums but not products.

---

Variance squares the scale factor

For constants and :

Thus variance ignores the offset and squares the scale factor. It is not linear!

Proof - Variance squares the scale factor

---

Extra - Moments

The moment of is defined as the expectation of :

Discrete case: Continuous case:

A central moment of is a moment of the variable :

The data of all the moments collectively determines the probability distribution. This fact can be very useful! In this way moments give an analogue of a series representation, and are sometimes more useful than the PDF or CDF for encoding the distribution.

03 Illustration

Example - Function given by chart

Variance of uniform random variable

Exercise - Probabilities via CDF

04 Theory

Suppose we are given the PDF of , a continuous RV.

What is the PDF , the PDF of the derived variable given by composing with ?

PDF of derived

The PDF of is not (usually) equal to .

Relating PDF and CDF

When the CDF of is differentiable, we have:

Therefore, if we know , we can find using a 3-step process:

1. & Find , the CDF of , by integration.
   • Compute .
   • Now remember that .
2. && Find , the CDF of , by direct comparison to .
   • When is monotone increasing, we have equivalent conditions:
   • Therefore:
   • By definition of CDFs:
3. & Find , the PDF of , by differentiation.
   • Use .

05 Illustration

Example - PDF of derived from CDF

Continuous wait times

06 Theory

Exponential variable

A random variable is exponential, written , when measures the wait time until first arrival in a Poisson process with rate .

Exponential PDF:

• The exponential distribution is the continuous counterpart of the geometric distribution.

  • Analogous to how the Poisson distribution is a like a continuous binomial.

• Notice that: so the coefficient of in is there to ensure that .

• ! Notice also that the “tail probability” is given by , an exponential decay.

  • Compute the improper integral to find this.

---

Erlang variable

A random variable is Erlang, written , when measures the wait time until arrival in a Poisson process with rate .

Erlang PDF:

• The Erlang distribution is the continuous counterpart of the Pascal distribution.

07 Illustration

Example - Earthquake wait time

08 Theory

The memoryless distribution is exponential

The exponential distribution is memoryless. This means that knowledge that an event has not yet occurred does not affect the probability of its occurring in future time intervals:

This is easily checked using the PDF: .

No other continuous distribution is memoryless. This means any other (continuous) memoryless distribution agrees in probability with the exponential distribution. The reason is that the memoryless property can be rewritten as . Consider as a function of , and notice that this function converts sums into products. Only the exponential function can do this.

The geometric distribution is the discrete memoryless one.

and by substituting , we also know .

Then:

Extra - Inversion of decay rate factor in exponential

For constants and : Derivation Let and observe that (the “tail probability”).

Now observe that: Let . So we see that:

Since the tail event is complementary to the cumulative event, these two distributions have the same CDF, and therefore they are equal.

Extra - Geometric limit to exponential

Divide the waiting time into small intervals. Let be the probability of at least one success in the time interval for any . Assume these events are independent.

A random variable measuring the end time of the first interval containing a success would have a geometric distribution with in place of :

By taking the sum of a geometric series, one finds:

Thus as .