All you need to know about the law of large numbers

Sep 16

All you need to know about the law of large numbers

category: AP Stats and Business Stats, Econometrics, ST104A, ST104B, Statistics 1, Statistics 2, The College Board 3 Comments

All about the law of large numbers: properties and applications

Level 1: estimation of population parameters

The law of large numbers is a statement about convergence which is called convergence in probability and denoted $\text{plim}$ . The precise definition is rather complex but the intuition is simple: it is convergence to a spike at the parameter being estimated. Usually, any unbiasedness statement has its analog in terms of the corresponding law of large numbers.

Example 1. The sample mean unbiasedly estimates the population mean: $E\bar{X}=EX$ . Its analog: the sample mean converges to a spike at the population mean: $\text{plim}\bar{X}=EX$ . See the proof based on the Chebyshev inequality.

Example 2. The sample variance unbiasedly estimates the population variance: $E\overline{s^2}=Var(X)$ where $s^2=\frac{\sum(X_i-\bar{X})^2}{n-1}$ . Its analog: the sample variance converges to a spike at the population variance:

(1) $\text{plim}\overline{s^2}=Var(X)$ .

Example 3. The sample covariance $s_{X,Y}=\frac{\sum(X_i-\bar{X})(Y_i-\bar{Y})}{n-1}$ unbiasedly estimates the population covariance: $E\overline{s_{X,Y}}=Cov(X,Y)$ . Its analog: the sample covariance converges to a spike at the population covariance:

(2) $\text{plim}\overline{s_{X,Y}}=Cov(X,Y)$ .

Up one level: convergence in probability is just convenient

Using or not convergence in probability is a matter of expedience. For usual limits of sequences we know the properties which I call preservation of arithmetic operations:

$\lim(a_n\pm b_n)=\lim a_n\pm \lim b_n,$

$\lim(a_n\times b_n)=\lim a_n\times\lim b_n,$

$\lim(a_n/ b_n)=\lim a_n/\lim b_n.$

Convergence in probability has exact same properties, just replace $\lim$ with $\text{plim}$ .

Next level: making regression estimation more plausible

Using convergence in probability allows us to handle stochastic regressors and avoid the unrealistic assumption that regressors are deterministic.

Convergence in probability and in distribution are two types of convergence of random variables that are widely used in the Econometrics course of the University of London.

Raising the bar

Ins and outs of quantitative courses of University of London