Theory

Theory 1

Conditional probability

The conditional probability of “ given ” is defined by:

This conditional probability represents the probability of event taking place given the assumption that took place. (All within the given probability model.)

By letting the actuality of event be taken as a fixed hypothesis, we can define a conditional probability measure by plugging events into the slot of :

It is possible to verify each of the Kolmogorov axioms for this function, and therefore itself defines a bona fide probability measure.

Conditioning

What does it really mean?

Conceptually, corresponds to creating a new experiment in which we run the old experiment and record data only those times that happened. Or, it corresponds to finding ourselves with knowledge or data that happened, and we seek our best estimates of the likelihoods of other events, based on our existing model and the actuality of .

Mathematically, corresponds to restricting the probability function to outcomes in , and renormalizing the values (dividing by ) so that the total probability of all the outcomes (in ) is now .

The definition of conditional probability can also be turned around and reinterpreted:

Multiplication rule

“The probability of AND equals the probability of times the probability of -given-.”

This principle generalizes to any events in sequence:

Generalized multiplication rule

The generalized rule can be verified like this. First substitute for and for in the original rule. Now repeat, substituting for and for in the original rule, and combine with the first one, and you find the rule for triples. Repeat again with and , combine with the triples, and you get quadruples.

Theory 2

Division into Cases

For any events and :

Interpretation: event may be divided along the lines of , with some of coming from the part in and the rest from the part in .

Total Probability - Explanation

• First divide itself into parts in and out of :

• These parts are exclusive, so in probability we have:

• Use the Multiplication rule to break up and :

• Now substitute in the prior formula:

This law can be generalized to any partition of the sample space . A partition is a collection of events which are mutually exclusive and jointly exhaustive:

The generalized formulation of Total Probability for a partition is:

Law of Total Probability

For a partition of the sample space :

Division into Cases is just the Law of Total Probability after setting and .