Discrete families: summary
01 Theory
Theory 1
Bernoulli:
- Indicates a win.
Binomial:
- Counts number of wins.
- These are
times the Bernoulli numbers. Geometric:
- Counts discrete wait time until first win.
Pascal:
- Counts discrete wait time until
win. - These are
times the Geometric numbers. Poisson:
Link to original
- Counts “arrivals” during time interval.
Function on a random variable
02 Theory
Theory 1
By composing any function
with a random variable we obtain a new random variable . The new one is called a derived random variable. Notation
The derived random variable
may be written “ ”. Expectation of derived variables
Discrete case:
(Here the sum is over all possible values
of , i.e. where .) Continuous case:
Notice: when applied to outcome
:
is the output of is the output of The proofs of these formulas are tricky because we must relate the PDF or PMF of
to that of . Proof - Discrete case - Expectation of derived variable
Linearity of expectation
For constants
and : For any
and on the same probability model: Exercise - Linearity of expectation
Using the definition of expectation, verify both linearity formulas for the discrete case.
Be careful!
Usually
. For example, usually
. We distribute
over sums but not products (unless the factors are independent).
Variance squares the scale factor
For constants
and : Thus variance ignores the offset and squares the scale factor. It is not linear!
Proof - Variance squares the scale factor
Extra - Moments
The
moment of is defined as the expectation of : Discrete case:
Continuous case:
A central moment of
is a moment of the variable : The data of all the moments collectively determines the probability distribution. This fact can be very useful! In this way moments give an analogue of a series representation, and are sometimes more useful than the PDF or CDF for encoding the distribution.
Link to original
03 Illustration
Example - Function given by chart
Expectation of function on RV given by chart
Suppose that
in such a way that and and and no other values are mapped to .
1 2 3 4 1 87 Then:
And:
Therefore:
Link to original
Variance of uniform random variable
Variance of uniform random variable
The uniform random variable
on has distribution given by when . (a) Find
using the shorter formula. (b) Find
using “squaring the scale factor.” (c) Find
directly. Solution (a)
(1) Compute density.
The density for
is:
(2) Compute
and directly using integral formulas. Compute
: Now compute
:
(3) Find variance using short formula.
Plug in:
(b)
(1) “Squaring the scale factor” formula:
(2) Plugging in:
(c)
(1) Density.
The variable
will have the density spread over the interval . Density is then:
(2) Plug into prior variance formula.
Use
and . Get variance:
Simplify:
Link to original
Exercise - Probabilities via CDF
Probabilities via CDF
Suppose the CDF of
is given by . Compute: (a)
Link to original(b) (c) (d)
04 Theory
Theory 2
Suppose we are given the PDF
of , a continuous RV. What is the PDF
, the derived variable given by composing with ? PDF of derived
The PDF of
is not (usually) equal to . Relating PDF and CDF
When the CDF of
is differentiable, we have: Therefore, if we know
, we can find using a 3-step process:
(1) Find
, the CDF of , by integration: Compute
. Now remember that
.
(2) Find
, the CDF of , by comparing conditions: When
is monotone increasing, we have equivalent conditions:
(3) Find
by differentiating : Method of differentials
Change variables: The measure for integration is
. Set so and . Thus . So the measure of integration in terms of is .
05 Illustration
Example - PDF of derived from CDF
PDF of derived from CDF
Suppose that
. (a) Find the PDF of
. (b) Find the PDF of . Solution
(a)
Formula:
Plug in:
(b)
By definition:
Since
is increasing, we know: Therefore:
Then using differentiation:
Link to original
Continuous wait times
06 Theory
Theory 1
Exponential variable
A random variable
is exponential, written , when measures the wait time until first arrival in a Poisson process with rate . Exponential PDF:
- Poisson is continuous analog of binomial
- Exponential is continuous analog of geometric
Notice the coefficient
in . This ensures :
Notice the “tail probability” is a simple exponential decay:
(Compute an improper integral to verify this.)
Erlang variable
A random variable
is Erlang, written , when measures the wait time until arrival in a Poisson process with rate . Erlang PDF:
Link to original
- Erlang is continuous analog of Pascal
07 Illustration
Example - Earthquake wait time
Earthquake wait time
Suppose the San Andreas fault produces major earthquakes modeled by a Poisson process, with an average of 1 major earthquake every 100 years.
(a) What is the probability that there will not be a major earthquake in the next 20 years?
(b) What is the probability that three earthquakes will strike within the next 20 years?
Solution
(a)
Since the average wait time is 100 years, we set
earthquakes per year. Set and compute: (b)
The same Poisson process has the same
earthquakes per year. Set , so: and compute:
Link to original
08 Theory
Theory 2
The memoryless distribution is exponential
The exponential distribution is memoryless. This means that knowledge that an event has not yet occurred does not affect the probability of its occurring in future time intervals:
This is easily checked using the PDF:
No other continuous distribution is memoryless. This means any other (continuous) memoryless distribution agrees in probability with the exponential distribution. The reason is that the memoryless property can be rewritten as
. Consider as a function of , and notice that this function converts sums into products. Only the exponential function can do this. The geometric distribution is the discrete memoryless distribution.
and by substituting
, we also know . Then:
Extra - Inversion of decay rate factor in exponential
For constants
and : Derivation: Let
and observe that (the “tail probability”). Now observe that:
Let
. So we see that: Since the tail event is complementary to the cumulative event, these two distributions have the same CDF, and therefore they are equal.
Link to originalExtra - Geometric limit to exponential
Divide the waiting time into small intervals. Let
be the probability of at least one success in the time interval for any . Assume these events are independent. A random variable
measuring the end time of the first interval containing a success would have a geometric distribution with in place of : By taking the sum of a geometric series, one finds:
Thus
as .