Sampling from uniform distribution - example of convolution
For the intuition behind the uniform distribution see this post. More formally, let us fix an interval . We say that a random variable is uniformly distributed on if any value in this segment is equally likely and any value outside is impossible. The "equally likely" part gives a constant density value in the segment and the "impossible" part gives zero density outside the segment. Thus the density is of the form:
This is a step function divided by to make the total area under the graph equal to one. It is easy to find the distribution function of by considering three ranges of :
What is the density of a sum?
Suppose we have a sample of two independent observations from the same uniform distribution. This means that they have the same density:
By independence we can apply the convolution formula (see equation (5)). The sum has the density
Here is different from zero only for
Similarly, is different from zero only for or, equivalently, only for
(2) and (3) show that the integrand in (1) is different from zero only for So (1) becomes
Next we consider several cases depending on the relationship between the segments
a) The two segments do not intersect: or (see Cases 1 and 2 in Figure 1). Equivalently, either or . In this case the integral in (4) is zero.
b) or, equivalently, , see Case 3 in Figure 1. In this case the integral in (4) is equal to the length of the intersection: .
c) Finally, or, equivalently, (Case 4). Again, the integral in (4) equals the length of the intersection: .
This is a triangular distribution, shown in Figure 2 from Mathematica with a=1, b=2. Note that it is smoother than the density of the uniform distribution: the density in Figure 2 is continuous and doesn't have jumps.
The density of the sum of three independent observations from the same uniform distribution is even smoother, see Figure 3. This is a general fact: as the number of independent observations (from the same population) increases, the density of the sum more and more reminds that of the normal distribution.
It is convenient to denote the uniform distribution on the segment and the triangular distribution defined in (5). We have proved that if are independent and distributed as , then their sum is distributed as .