Events and outcomes
01 Theory
Events and outcomes – informally
- An event is a description of something that can happen.
- An outcome is a complete description of something that can happen.
All outcomes are events. An event is usually a partial description. Outcomes are events given with a complete description.
Here ‘complete’ and ‘partial’ are within the context of the probability model.
- !! It can be misleading to say that an ‘outcome’ is an ‘observation’.
- ‘Observations’ occur in the real world, while ‘outcomes’ occur in the model.
- To the extent the model is a good one, and the observation conveys complete information, we can say ‘outcome’ for the observation.
Notice:
- ! Because outcomes are complete, no two distinct outcomes could actually happen in a run of the experiment being modeled.
When an event happens, the fact that it has happened constitutes information.
Events and outcomes – mathematically
- The sample space is the set of possible outcomes, so it is the set of the complete descriptions of everything that can happen.
- An event is a subset of the sample space, so it is a collection of outcomes.
- % For mathematicians: some “wild” subsets are not valid events. Problems with infinity and the continuum…
Notation
Write
for the set of possible outcomes, for a single outcome in . Write
or for some events, subsets of . Write
for the collection of all events. This is frequently a huge set! Write
for the cardinality or size of a set , i.e. the number of elements it contains.
Using this notation, we can consider an outcome itself as an event by considering the “singleton” subset
02 Illustration
Example - Coin flipping
Exercise - Coin flipping: counting subsets
03 Theory
New events from old
Given two events
and , we can form new events using set operations: We also use these terms for events and :
They are mutually exclusive when
, that is, they have no elements in common. They are collectively exhaustive
, that is, when they jointly cover all possible outcomes.
- ! In probability texts, sometimes
is written “ ” or even (frequently!) “ ”.
Rules for sets
Algebraic rules
Associativity:
. Analogous to . Distributivity:
. Analogous to . De Morgan’s Laws
In other words: you can distribute “ ” but must simultaneously do a switch .
Probability models
04 Theory
Axioms of probability
A probability measure is a function
satisfying: Kolmogorov Axioms:
Axiom 1:
for every event (probabilities are not negative!) Axiom 2:
(probability of “anything” happening is 1) Axiom 3: additivity for any countable collection of mutually exclusive events:
- %& Notation: we write
instead of , even though is a function, to emphasize the fact that is a set.
Probability model
A probability model or probability space consists of a triple
:
the sample space
the set of valid events, where every satisfies
a probability measure satisfying the Kolmogorov Axioms
Finitely many exclusive events
It is a consequence of the Kolmogorov Axioms that additivity also works for finite collections of mutually exclusive events:
Inferences from Kolmogorov
A probability measure satisfies these rules. They can be deduced from the Kolmogorov Axioms.
Negation: Can you find
but not ? Use negation: Monotonicity: Probabilities grow when outcomes are added:
Inclusion-Exclusion: A trick for resolving unions:
(even when and are not exclusive!)
Inclusion-Exclusion
The principle of inclusion-exclusion generalizes to three events:
The same pattern works for any number of events!
The pattern goes: “include singles” then “exclude doubles” then “include triples” then …
Include, exclude, include, exclude, include, …
05 Illustration
Example - Lucia is Host or Player
Example - iPhones and iPads
Conditional probability
06 Theory
Conditional probability
The conditional probability of “
given ” is defined by:
This conditional probability
By letting the actuality of event
It is possible to verify each of the Kolmogorov axioms for this function, and therefore
Conditioning
What does it really mean?
Conceptually,
corresponds to creating a new experiment in which we run the old experiment and record data only those times that happened. Or, it corresponds to finding ourselves with knowledge or data that happened, and we seek our best estimates of the likelihoods of other events, based on our existing model and the actuality of . Mathematically,
corresponds to restricting the probability function to outcomes in , and renormalizing the values (dividing by ) so that the total probability of all the outcomes (in ) is now .
The definition of conditional probability can also be turned around and reinterpreted:
Multiplication rule
“The probability of
AND equals the probability of times the probability of -given- .”
This principle generalizes to any events in sequence:
Generalized multiplication rule
The generalized rule can be verified like this. First substitute
for and for in the original rule. Now repeat, substituting for and for in the original rule, and combine with the first one, and you find the rule for triples. Repeat again with and , combine with the triples, and you get quadruples.
07 Illustration
Exercise - Simplifying conditionals
Example - Coin flipping: at least 2 heads
Example: Flip a coin, then roll dice
Multiplication: draw two cards
08 Theory
Division into Cases
For any events
and :
Interpretation: event
Total Probability - Explanation
- First divide
itself into parts in and out of : - These parts are exclusive, so in probability we have:
- Use the Multiplication rule to break up
and : - Now substitute in the prior formula:
This law can be generalized to any partition of the sample space
Law of Total Probability
For a partition
of the sample space :
Division into Cases is just the Law of Total Probability after setting
09 Illustration
Exercise - Marble transferred, marble drawn