24
Oct 22

## Marginal probabilities and densities

This is to help everybody, from those who study Basic Statistics up to Advanced Statistics ST2133.

### Discrete case

Suppose in a box we have coins and banknotes of only two denominations: $1 and$5 (see Figure 1).

Figure 1. Illustration of two variables

We pull one out randomly. The division of cash by type (coin or banknote) divides the sample space (shown as a square, lower left picture) with probabilities $p_{c}$ and $p_{b}$ (they sum to one). The division by denomination ($1 or$5) divides the same sample space differently, see the lower right picture, with the probabilities to pull out $1 and$5 equal to $p_{1}$ and $p_{5}$, resp. (they also sum to one). This is summarized in the tables

 Variable 1: Cash type Prob coin $p_{c}$$p_{c}$ banknote $p_{b}$$p_{b}$
 Variable 2: Denomination Prob $1 $p_{1}$$p_{1}$$5 $p_{5}$$p_{5}$

Now we can consider joint events and probabilities (see Figure 2, where the two divisions are combined).

Figure 2. Joint probabilities

For example, if we pull out a random $item$ it can be a $coin$ and \$1 and the corresponding probability is $P\left(item=coin,\ item\ value=\1\right) =p_{c1}.$ The two divisions of the sample space generate a new division into four parts. Then geometrically it is obvious that we have four identities:

Adding over denominations: $p_{c1}+p_{c5}=p_{c},$ $p_{b1}+p_{b5}=p_{b},$

Adding over cash types: $p_{c1}+p_{b1}=p_{1},$ $p_{c5}+p_{b5}=p_{5}.$

Formally, here we use additivity of probability for disjoint events

$P\left( A\cup B\right) =P\left( A\right) +P\left( B\right) .$

In words: we can recover own probabilities of variables 1,2 from joint probabilities.

### Generalization

Suppose we have two discrete random variables $X,Y$ taking values $x_{1},...,x_{n}$ and $y_{1},...,y_{m},$ resp., and their own probabilities are $P\left( X=x_{i}\right) =p_{i}^{X},$ $P\left(Y=y_{j}\right) =p_{j}^{Y}.$ Denote the joint probabilities $P\left(X=x_{i},Y=y_{j}\right) =p_{ij}.$ Then we have the identities

(1) $\sum_{j=1}^mp_{ij}=p_{i}^{X},$ $\sum_{i=1}^np_{ij}=p_{j}^{Y}$ ($n+m$ equations).

In words: to obtain the marginal probability of one variable (say, $Y$) sum over the values of the other variable (in this case, $X$).

The name marginal probabilities is used for $p_{i}^{X},p_{j}^{Y}$ because in the two-dimensional table they arise as a result of summing table entries along columns or rows and are displayed in the margins.

### Analogs for continuous variables with densities

Suppose we have two continuous random variables $X,Y$ and their own densities are $f_{X}$ and $f_{Y}.$ Denote the joint density $f_{X,Y}$. Then replacing in (1) sums by integrals and probabilities by densities we get

(2) $\int_R f_{X,Y}\left( x,y\right) dy=f_{X}\left( x\right) ,\ \int_R f_{X,Y}\left( x,y\right) dx=f_{Y}\left( y\right) .$

In words: to obtain one marginal density (say, $f_{Y}$) integrate out the other variable (in this case, $x$).