Raising the bar

Density of a sum of independent variables is given by convolution

This topic is pretty complex because it involves properties of integrals that economists usually don't study. I provide this result to be able to solve one of UoL problems.

General relationship between densities

Let $latex X,Y$ be two independent variables with densities $latex f_X,f_Y$. Denote $latex f_{X,Y}$ the joint density of the pair $latex (X,Y)$.

By independence we have

(1) $latex f_{X,Y}(x,y)=f_X(x)f_Y(y).$

Let $latex Z=X+Y$ be the sum and let $latex f_Z,\ F_Z$ be its density and distribution function, respectively. Then

(2) $latex f_Z(z)=\frac{d}{dz}F_Z(z)$.

These are the only simple facts in this derivation. By definition,

(3) $latex F_Z(z)=P(Z\le z)=P(X+Y\le z)$.

For the last probability in (3) we have a double integral

$latex P(X+Y\le z)=\int\int_{x+y\le z}f_{X,Y}(x,y)dxdy$.

Using (1), we replace the joint probability by the product of individual probabilities and the double integral by the repeated one:

(4) $latex P(X+Y\le z)=\int\int_{x+y\le z}f_X(x)f_Y(y)dxdy=\int_R\int_{-\infty}^{z-x}f_X(x)f_Y(y)dxdy$

$latex =\int_Rf_X(x)\left(\int_{-\infty}^{z-x}f_Y(y)dy\right)dx$.

The geometry is explained in Figure 1. The area $latex x+y\le z$ is limited by the line $latex y=z-x$. In the repeated integral, we integrate first over red lines from $latex -\infty$ to $latex z-x$ and then in the outer integral over all $latex x\in R$.

Figure 1. Area of integration

(3) and (4) imply

$latex F_Z(z)=\int_Rf_X(x)\left(\int_{-\infty}^{z-x}f_Y(y)dy\right)dx$.

Finally, using (2) we differentiate both sides to get

(5) $latex f_Z(z)=\int_Rf_X(x)f_Y(z-x)dx$.

This is the result. The integral on the right is called a convolution of functions $latex f_X,f_Y$.

Remark. Existence of density (2) follows from existence of $latex f_X,f_Y$, although we don't prove this fact.

Exercise. Convolution is usually denoted by $latex (f*g)(z)=\int_Rf(x)g(z-x)dx$. Prove that

1. $latex (f*g)(z)=(g*f)(z)$.

2. $latex \int_R|(f*g)(z)|dz\le \int_R|f(x)|dx\int_R|g(x)|dx$.

3. If $latex X$ is uniformly distributed on some segment, then $latex (f_X*f_X)(z)$ is zero for large $latex z$.