Raising the bar

Marginal probabilities and densities

This is to help everybody, from those who study Basic Statistics up to Advanced Statistics ST2133.

Discrete case

Suppose in a box we have coins and banknotes of only two denominations: $1 and $5 (see Figure 1).

Figure 1. Illustration of two variables

We pull one out randomly. The division of cash by type (coin or banknote) divides the sample space (shown as a square, lower left picture) with probabilities $latex p_{c}$ and $latex p_{b}$ (they sum to one). The division by denomination ($1 or $5) divides the same sample space differently, see the lower right picture, with the probabilities to pull out $1 and $5 equal to $latex p_{1}$ and $latex p_{5}$, resp. (they also sum to one). This is summarized in the tables

Now we can consider joint events and probabilities (see Figure 2, where the two divisions are combined).

Figure 2. Joint probabilities

For example, if we pull out a random $latex item$ it can be a $latex coin$ and $1 and the corresponding probability is $latex P\left(item=coin,\ item\ value=\$1\right) =p_{c1}.$ The two divisions of the sample space generate a new division into four parts. Then geometrically it is obvious that we have four identities:

Adding over denominations: $latex p_{c1}+p_{c5}=p_{c},$ $latex p_{b1}+p_{b5}=p_{b},$

Adding over cash types: $latex p_{c1}+p_{b1}=p_{1},$ $latex p_{c5}+p_{b5}=p_{5}.$

Formally, here we use additivity of probability for disjoint events

$latex P\left( A\cup B\right) =P\left( A\right) +P\left( B\right) .$

In words: we can recover own probabilities of variables 1,2 from joint probabilities.

Generalization

Suppose we have two discrete random variables $latex X,Y$ taking values $latex x_{1},...,x_{n}$ and $latex y_{1},...,y_{m},$ resp., and their own probabilities are $latex P\left( X=x_{i}\right) =p_{i}^{X},$ $latex P\left(Y=y_{j}\right) =p_{j}^{Y}.$ Denote the joint probabilities $latex P\left(X=x_{i},Y=y_{j}\right) =p_{ij}.$ Then we have the identities

(1) $latex \sum_{j=1}^mp_{ij}=p_{i}^{X},$ $latex \sum_{i=1}^np_{ij}=p_{j}^{Y}$ ($latex n+m$ equations).

In words: to obtain the marginal probability of one variable (say, $latex Y$) sum over the values of the other variable (in this case, $latex X$).

The name marginal probabilities is used for $latex p_{i}^{X},p_{j}^{Y}$ because in the two-dimensional table they arise as a result of summing table entries along columns or rows and are displayed in the margins.

Analogs for continuous variables with densities

Suppose we have two continuous random variables $latex X,Y$ and their own densities are $latex f_{X}$ and $latex f_{Y}.$ Denote the joint density $latex f_{X,Y}$. Then replacing in (1) sums by integrals and probabilities by densities we get the marginal rule (see Example 6)

(2) $latex \int_R f_{X,Y}\left( x,y\right) dy=f_{X}\left( x\right) ,\ \int_R f_{X,Y}\left( x,y\right) dx=f_{Y}\left( y\right) .$

In words: to obtain one marginal density (say, $latex f_{Y}$) integrate out the other variable (in this case, $latex x$).