Events and outcomes
01 Theory
Theory 1
Events and outcomes – informally
- An event is a description of something that can happen.
- An outcome is a complete description of something that can happen.
All outcomes are events. An event is usually a partial description. Outcomes are events given with a complete description.
Here ‘complete’ and ‘partial’ are within the context of the probability model.
It can be misleading to say that an ‘outcome’ is an ‘observation’.
- ‘Observations’ occur in the real world, while ‘outcomes’ occur in the model.
- To the extent the model is a good one, and the observation conveys complete information, we can say ‘outcome’ for the observation.
Notice: Because outcomes are complete, no two distinct outcomes could actually happen in a run of the experiment being modeled.
When an event happens, the fact that it has happened constitutes information.
Events and outcomes – mathematically
- The sample space is the set of possible outcomes, so it is the set of the complete descriptions of everything that can happen.
- An event is a subset of the sample space, so it is a collection of outcomes.
For mathematicians: some “wild” subsets are not valid events. Problems with infinity and the continuum...
Notation
Write
for the set of possible outcomes, for a single outcome in . Write
or for some events, subsets of . Write
for the collection of all events. This is frequently a huge set! Write
for the cardinality or size of a set , i.e. the number of elements it contains. Using this notation, we can consider an outcome itself as an event by considering the “singleton” subset
Link to originalwhich contains that outcome alone.
02 Illustration
Example - Coin flipping
Coin flipping
Flip a fair coin two times and record both results.
Outcomes: sequences, like
or . Sample space: all possible sequences, i.e. the set
. Events: for example:
With this setup, we may combine events in various ways to generate other events:
Complex events: for example:
, or in words: Notice that the last one is a complete description, namely the outcome
. , or in words: Link to original
Exercise - Coin flipping: counting subsets
Coin flipping: counting subsets
Flip a fair coin five times and record the results.
How many elements are in the sample space? (How big is
Link to original?) How many events are there? (How big is ?)
03 Theory
Theory 2
New events from old
Given two events
and , we can form new events using set operations: We also use these terms for events
and :
They are mutually exclusive when
, that is, they have no elements in common. They are collectively exhaustive
, that is, when they jointly cover all possible outcomes. In probability texts, sometimes
is written “ ” or even (frequently!) “ ”. Link to originalRules for sets
Algebraic rules
Associativity:
. Analogous to . Distributivity:
. Analogous to . De Morgan’s Laws
In other words: you can distribute “ ” but must simultaneously do a switch .
Probability models
04 Theory
Theory 1
Axioms of probability
A probability measure is a function
satisfying: Kolmogorov Axioms:
Axiom 1:
for every event (probabilities are not negative!) Axiom 2:
(probability of “anything” happening is 1) Axiom 3: additivity for any countable collection of mutually exclusive events:
Notation: we write
instead of , even though is a function, to emphasize the fact that is a set. Probability model
A probability model or probability space consists of a triple
:
the sample space
the set of valid events, where every satisfies
a probability measure satisfying the Kolmogorov Axioms Finitely many exclusive events
It is a consequence of the Kolmogorov Axioms that additivity also works for finite collections of mutually exclusive events:
Inferences from Kolmogorov
A probability measure satisfies these rules. They can be deduced from the Kolmogorov Axioms.
- Negation: Can you find
but not ? Use negation:
- Monotonicity: Probabilities grow when outcomes are added:
- Inclusion-Exclusion: A trick for resolving unions:
(even when
and are not exclusive!) Link to originalInclusion-Exclusion
The principle of inclusion-exclusion generalizes to three events:
The same pattern works for any number of events!
The pattern goes: “include singles” then “exclude doubles” then “include triples” then …
Include, exclude, include, exclude, include, …
05 Illustration
Example - Lucia is Host or Player
Lucia is Host or Player
The professor chooses three students at random for a game in a class of 40, one to be Host, one to be Player, one to be Judge. What is the probability that Lucia is either Host or Player?
Solution
(1) Set up the probability model.
Label the students
to . Write for Lucia’s number. Outcomes: assignments such as
These are ordered triples with distinct entries in
. Sample space:
is the collection of all such distinct triples Events: any subset of
Probability measure: assume all outcomes are equally likely, so
for all In total there are
triples of distinct numbers. Therefore
for any specific outcome . Therefore
for any event . (Recall is the number of outcomes in .)
(2) Define the desired event.
Want to find
Define
and . Thus: So we seek
.
(3) Compute the desired probability.
Importantly,
(mutually exclusive). There are no outcomes in
in which Lucia is both Host and Player. By additivity, we infer
. Now compute
. There are
ways to choose and from the students besides Lucia. Therefore
. Therefore:
Now compute
. It is similar: . Finally compute that
, so the answer is: Link to original
Example - iPhones and iPads
iPhones and iPads
At Mr. Jefferson’s University, 25% of students have an iPhone, 30% have an iPad, and 60% have neither.
What is the probability that a randomly chosen student has some iProduct? (Q1)
What about both? (Q2)
Solution
(1) Set up the probability model.
A student is chosen at random: an outcome is the chosen student.
Sample space
is the set of all students. Write
and concerning the chosen student. All students are equally likely to be chosen: therefore
for any event . Therefore
and . Furthermore,
. This means 60% have “not iPhone AND not iPad”.
(2) Define the desired event.
Q1:
Q2:
(3) Compute the probabilities.
We do not believe
and are exclusive. Try: apply inclusion-exclusion:
We know
and . So this formula, with given data, RELATES Q1 and Q2. Notice the complements in
and try Negation. Negation:
DOESN’T HELP.
Try again: Negation:
And De Morgan (or a Venn diagram!):
Therefore:
We have found Q1:
. Applying the RELATION from inclusion-exclusion, we get Q2:
Link to original
Conditional probability
06 Theory
Theory 1
Conditional probability
The conditional probability of “
given ” is defined by: This conditional probability
represents the probability of event taking place given the assumption that took place. (All within the given probability model.) By letting the actuality of event
be taken as a fixed hypothesis, we can define a conditional probability measure by plugging events into the slot of : It is possible to verify each of the Kolmogorov axioms for this function, and therefore
itself defines a bona fide probability measure. Conditioning
What does it really mean?
Conceptually,
corresponds to creating a new experiment in which we run the old experiment and record data only those times that happened. Or, it corresponds to finding ourselves with knowledge or data that happened, and we seek our best estimates of the likelihoods of other events, based on our existing model and the actuality of . Mathematically,
corresponds to restricting the probability function to outcomes in , and renormalizing the values (dividing by ) so that the total probability of all the outcomes (in ) is now . The definition of conditional probability can also be turned around and reinterpreted:
Multiplication rule
“The probability of
AND equals the probability of times the probability of -given- .” This principle generalizes to any events in sequence:
Link to originalGeneralized multiplication rule
The generalized rule can be verified like this. First substitute
for and for in the original rule. Now repeat, substituting for and for in the original rule, and combine with the first one, and you find the rule for triples. Repeat again with and , combine with the triples, and you get quadruples.
07 Illustration
Exercise - Simplifying conditionals
Simplifying conditionals inclusion
Let
. Simplify the following values: Link to original
Example - Coin flipping: at least 2 heads
Coin flipping: at least 2 heads
Flip a fair coin 4 times and record the outcomes as sequences, like HHTH.
Let
be the event that there are at least two heads, and the event that there is at least one heads. First let’s calculate
. Define
, the event that there were exactly 2 heads, and , the event of exactly 3, and the event of exactly 4. These events are exclusive, so: Each term on the right can be calculated by counting:
Therefore,
. Now suppose we find out that “at least one heads definitely came up”. (Meaning that we know
.) For example, our friend is running the experiment and tells us this fact about the outcome. Now what is our estimate of likelihood of
? The formula for conditioning gives:
Now
. (Any outcome with at least two heads automatically has at least one heads.) We already found that . To compute we simply add the probability , which is , to get . Therefore:
Link to original
Example - Flip a coin, then roll dice
Multiplication: flip a coin, then roll dice
Flip a coin. If the outcome is heads, roll two dice and add the numbers. If the outcome is tails, roll a single die and take that number. What is the probability of getting a tails AND a number at least 3?
Solution
(1) This “two-stage” experiment lends itself to a solution using the multiplication rule for conditional probability.
Label the events of interest.
Let
and be the events that the coin showed heads and tails, respectively. Let
be the events that the final number is , respectively. The value we seek is
.
(2) Observe known (conditional) probabilities.
We know that
and . We know that
, for example, or that .
(3) Apply “multiplication” rule.
This rule gives:
We know
and can see by counting that . Therefore
Link to original.
Example - Multiplication: draw two cards
Multiplication: draw two cards
Two cards are drawn from a standard deck (without replacement).
What is the probability that the first is a 3, and the second is a 4?
Solution
(1) This “two-stage” experiment lends itself to a solution using the multiplication rule for conditional probability. Label events.
Write
for the event that the first card is a 3 Write
for the event that the second card is a 4. We seek
.
(2) Write down knowns.
We know
. (It does not depend on the second draw.) Easily find
.
(3) If the first is a 3, then there are four 4s remaining and 51 cards.
So
.
(4) Apply multiplication rule.
Multiplication rule:
Therefore
Link to original
08 Theory
Theory 2
Division into Cases
For any events
and : Interpretation: event
may be divided along the lines of , with some of coming from the part in and the rest from the part in . Total Probability - Explanation
- First divide
itself into parts in and out of :
- These parts are exclusive, so in probability we have:
- Use the Multiplication rule to break up
and :
- Now substitute in the prior formula:
This law can be generalized to any partition of the sample space
. A partition is a collection of events which are mutually exclusive and jointly exhaustive: The generalized formulation of Total Probability for a partition is:
Law of Total Probability
For a partition
of the sample space :
Division into Cases is just the Law of Total Probability after setting
Link to originaland .
09 Illustration
Exercise - Marble transferred, marble drawn
Marble transferred, marble drawn
Setup:
- Bin 1 holds five red and four green marbles.
- Bin 2 holds four red and five green marbles.
Experiment:
- You take a random marble from Bin 1 and put it in Bin 2 and shake Bin 2.
- Then you draw a random marble from Bin 2 and look at it.
What is the probability that the marble you look at is red?
Link to original