The law of large numbers overview
I have already several posts about the law of large numbers:
- start with the intuition, which is illustrated using Excel;
- simulations in Excel show that convergence is not as fast as some textbooks claim;
- to distinguish the law of large numbers from the central limit theorem read this;
- the ultimate purpose is the application to simple regression with a stochastic regressor.
Here we busy ourselves with the proof.
Measuring deviation of a random variable from a constant
Let be a random variable and
some constant. We want a measure of
differing from the constant by a given number
or more. The set where
differs from
by
or more is the outside of the segment
, that is,
.

Figure 1. Measuring the outside of interval
Now suppose
Convergence to a spike formalized


Once again, check out the idea. Consider a sequence of random variables
Definition. Let
The law of large numbers in its simplest form
Let
(1)
and its variance tends to zero
(2)
Now
Since this is true for any
Final remarks
The above proof applies in the next more general situation.
Theorem. Let
This statement is often used on the Econometrics exams of the University of London.
In the unbiasedness definition the sample size is fixed. In the consistency definition it tends to infinity. The above theorem says that unbiasedness for all
Leave a Reply
You must be logged in to post a comment.