Oct 22

Marginal probabilities and densities

Marginal probabilities and densities

This is to help everybody, from those who study Basic Statistics up to Advanced Statistics ST2133.

Discrete case

Suppose in a box we have coins and banknotes of only two denominations: $1 and $5 (see Figure 1).

Box with cash

Figure 1. Illustration of two variables

We pull one out randomly. The division of cash by type (coin or banknote) divides the sample space (shown as a square, lower left picture) with probabilities p_{c} and p_{b} (they sum to one). The division by denomination ($1 or $5) divides the same sample space differently, see the lower right picture, with the probabilities to pull out $1 and $5 equal to p_{1} and p_{5}, resp. (they also sum to one). This is summarized in the tables

Variable 1: Cash type Prob
coin p_{c}
banknote p_{b}
Variable 2: Denomination Prob
$1 p_{1}
$5 p_{5}

Now we can consider joint events and probabilities (see Figure 2, where the two divisions are combined).

Box with cash

Figure 2. Joint probabilities

For example, if we pull out a random item it can be a coin and $1 and the corresponding probability is P\left(item=coin,\ item\ value=\$1\right) =p_{c1}. The two divisions of the sample space generate a new division into four parts. Then geometrically it is obvious that we have four identities:

Adding over denominations: p_{c1}+p_{c5}=p_{c}, p_{b1}+p_{b5}=p_{b},

Adding over cash types: p_{c1}+p_{b1}=p_{1}, p_{c5}+p_{b5}=p_{5}.

Formally, here we use additivity of probability for disjoint events

P\left( A\cup B\right) =P\left( A\right) +P\left( B\right) .

In words: we can recover own probabilities of variables 1,2 from joint probabilities.


Suppose we have two discrete random variables X,Y taking values x_{1},...,x_{n} and y_{1},...,y_{m}, resp., and their own probabilities are P\left( X=x_{i}\right) =p_{i}^{X}, P\left(Y=y_{j}\right) =p_{j}^{Y}. Denote the joint probabilities P\left(X=x_{i},Y=y_{j}\right) =p_{ij}. Then we have the identities

(1) \sum_{j=1}^mp_{ij}=p_{i}^{X}, \sum_{i=1}^np_{ij}=p_{j}^{Y} (n+m equations).

In words: to obtain the marginal probability of one variable (say, Y) sum over the values of the other variable (in this case, X).

The name marginal probabilities is used for p_{i}^{X},p_{j}^{Y} because in the two-dimensional table they arise as a result of summing table entries along columns or rows and are displayed in the margins.

Analogs for continuous variables with densities

Suppose we have two continuous random variables X,Y and their own densities are f_{X} and f_{Y}. Denote the joint density f_{X,Y}. Then replacing in (1) sums by integrals and probabilities by densities we get

(2) \int_R f_{X,Y}\left( x,y\right) dy=f_{X}\left( x\right) ,\ \int_R f_{X,Y}\left( x,y\right) dx=f_{Y}\left( y\right) .

In words: to obtain one marginal density (say, f_{Y}) integrate out the other variable (in this case, x).


Leave a Reply

You must be logged in to post a comment.