Theory

Theory 1

Joint distributions describe the probabilities of events associated with multiple random variables simultaneously.

In this course we consider only two variables at a time, typically called and . It is easy to extend this theory to vectors of random variables.

Joint PMF and joint PDF

Discrete joint PMF:

Continuous joint PDF:

Probabilities of events: Discrete case If is a set of points in the plane, then an event is formed by the set of all outcomes mapped by and to points in :

The probabilities of such events can be measured using the joint PMF:

Probabilities of events: Continuous case Let be the rectangular region defined by such that and . Then:

For more general regions :

---

The existence of a variable does not change the theory for a variable considered by itself.

However, it is possible to relate the theory for to the theory for , in various ways.

The simplest relationship is the marginal distribution for , which is merely the distribution of itself, considered as a single random variable, but in a context where it is derived from the joint distribution for .

Marginal PMF, marginal PDF

Marginal distributions are obtained from joint distributions by summing the probabilities over all possibilities of the other variable.

Discrete marginal PMF:

Continuous marginal PMF:

---

Infinitesimal method

Suppose has density that is continuous at . Then for infinitesimal :

Suppose and have joint density that is continuous at . Then for infinitesimal :

Joint densities depend on coordinates

The density in these integration formulas depends on the way and act as Cartesian coordinates and determine differential areas as little rectangles.

To find a density in polar coordinates, for example, it is not enough to solve for and and plug into !

Instead, we must consider the differential area vs. . We find that .

As an example, the density of the uniform distribution on the unit disk is , which is not constant as a function of and .

Extra - Joint densities may not exist

It is not always possible to form a joint PDF from any two continuous RVs and .

For example, if , then cannot have a joint PDF, since but the integral over the region will always be 0. (The area of a line is zero.)

Theory 2

Joint CDF

The joint CDF of and is defined by:

We can relate the joint CDF to the joint PDF using integration:

Conversely, if and have a continuous joint PDF that is also differentiable, we can obtain the PDF from the CDF using partial derivatives:

---

There is also a marginal CDF that is computed using a limit:

This could also be written, somewhat abusing notation, as .