Sum of random variables and convolution
Link between double and iterated integrals
Why do we need this link? For simplicity consider the rectangle The integrals
both are taken over the rectangle but they are not the same. is a double (two-dimensional) integral, meaning that its definition uses elementary areas, while is an iterated integral, where each of the one-dimensional integrals uses elementary segments. To make sense of this, you need to consult an advanced text in calculus. The difference notwithstanding, in good cases their values are the same. Putting aside the question of what is a "good case", we concentrate on geometry: how a double integral can be expressed as an iterated integral.
It is enough to understand the idea in case of an oval on the plane. Let be the function that describes the lower boundary of the oval and let be the function that describes the upper part. Further, let the vertical lines and be the minimum and maximum values of in the oval (see Chart 1).
We can paint the oval with strokes along red lines from to If we do this for all we'll have painted the whole oval. This corresponds to the representation of as the union of segments with
and to the equality of integrals
(double integral) (iterated integral)
Density of a sum of two variables
Assumption 1 Suppose the random vector has a density and define (unlike the convolution theorem below, here don't have to be independent).
From the definitions of the distribution function and probability
The integral on the right is a double integral. The painting analogy (see Chart 2)
Differentiating both sides with respect to we get
If we start with the inner integral that is with respect to and the outer integral with respect to then similarly
Exercise. Suppose the random vector has a density and define Find Hint: review my post on Leibniz integral rule.
In addition to Assumption 1, let be independent. Then and the above formula gives
This is denoted as and called a convolution.
The following may help to understand this formula. The function is a density (it is non-negative and integrates to 1). Its graph is a mirror image of that of with respect to the vertical axis. The function is a shift of by along the horizontal axis. For fixed it is also a density. Thus in the definition of convolution we integrate the product of two densities Further, to understand the asymptotic behavior of when imagine two bell-shaped densities and When goes to, say, infinity, the humps of those densities are spread apart more and more. The hump of one of them gets multiplied by small values of the other. That's why goes to zero, in a certain sense.
The convolution of two densities is always a density because it is non-negative and integrates to one:
Replacing in the inner integral we see that this is