Raising the bar

Chapter 6. Continuous Random Variables and Probability Distributions

Unit 6.7. Normal approximation to the binomial

Example 6.3. Compare two statements:

1. The expected price of Microsoft stock tomorrow is $latex \$43$.
2. With probability 0.95 the price of Microsoft stock tomorrow will be between $latex \$40$ and $latex \$46$, $latex P(40 \le X \le 46) = 0.95$.

The first statement, called a point estimate, entails too much uncertainty. The second estimate, called an interval estimate, is preferable.

Confidence interval and related definitions. An interval which contains the values of a random variable $latex X$ with high probability, $latex P(a \le X \le b) = 1 - \alpha$, where $latex \alpha$ is close to zero, is called a confidence interval and the number $latex 1 - \alpha$ is called a confidence level*. The areas $latex { X < a}$ and $latex { X > b}$ are called a* left tail and right tail*, respectively. Confidence intervals can be two-tail and one-tail. If both probabilities $latex P(X < a)$ and $latex P(X > b)$ are positive, it is a two-tail interval; if one of them is zero, we deal with a one-tail interval. The number $latex \alpha$ is called a* significance level.

The interest in confidence intervals leads us to the next definition.

Convergence in distribution. Let $latex {X_1},\,\,{X_2},\,\,...$ be a sequence of random variables with distribution functions $latex F_1(x),F_2(x),...$ We say that it converges to a random variable $latex X$ in distribution if distribution functions $latex {F_n}(x)$ approach $latex F(x)$ for all real $latex x$ ($latex F_n(x)\rightarrow F(x)$), as $latex n \to \infty$.

See the illustration in Figure 6.4, where the dashed curves are distribution functions of $latex X_1,...,\ X_4$ and the continuous line is the distribution function of $latex X$.

Figure 6.4. Convergence in distribution.

Corollary 6.1. If $latex {X_n}$ converges to $latex X$ in distribution, then for all large $latex n$ by the interval formula confidence intervals for $latex {X_n}$ are close to the confidence interval for $latex X$:

$latex P(a \le {X_n} \le b) = {F_{{X_n}}}(b) - {F_{{X_n}}}(a) \approx {F_X}(b) - {F_X}(a) = P(a \le X \le b). \qquad (\textrm{Eq. 6.20})$

Thus, to have an approximation for confidence intervals it is enough to have an approximation for distribution functions.

Theorem 6.1 (Central limit theorem). Let $latex {X_1},\,\,{X_2},\,\,...$ be a sequence of i.i.d. random variables obtained by sampling from an arbitrary population with $latex \sigma^2_X < \infty$. We want to obtain confidence intervals for the sums $latex {S_n} = {X_1} + ... + {X_n}$. Denote $latex {z_n} = \frac{{S_n - ES_n}}{{\sigma _{S_n}}}$ the standardized versions of $latex {S_n}$. $latex {z_n}$ in general are not standard normal but they converge in distribution to a standard normal distribution. In particular, by Eq. (6.20) for any $latex a < b$ one has

$latex P(a < {z_n} < b) \approx P(a < z < b). \qquad (\textrm{Eq. 6.21})$

Comments. (1) Show that the standardized versions of sample means coincide with those of $latex S_n$. Hence, they satisfy Eq. (6.21).

(2) For convergence of a sequence of random variables in distribution to a non-degenerate limit, it is necessary that their means converge to a real number and their variances to a positive number. As we know from Eq. (5.13) (see An obvious and handy result), means and variances of the sums $latex S_n$ do not converge. Population means of sample means do converge (they are just constant) but variances converge to zero. Thus neither the sums $latex S_n$ nor the sample means $latex S_n/n$ satisfy the conditions necessary for the central limit theorem (still it is possible to obtain approximate confidence intervals for $latex S_n$, see the next corollary). By standardizing the sums $latex S_n$ we obtain variables whose means are zero and variances are unities, which increases the chances for convergence in distribution. After standardization it suffices to impose only the condition $latex \sigma^2_X < \infty$ to achieve convergence. This is why Theorem 6.1 has so many applications.

Corollary 6.2. Let $latex {S_n}$ be a binomial distribution. Then for any numbers $latex a,b$ and large $latex n$ one has

$latex P(a \le {S_n} \le b) \approx P\left( \frac{{a - E{S_n}}}{{{\sigma _{{S_n}}}}} \le z \le \frac{{b - E{S_n}}}{{{\sigma _{{S_n}}}}} \right). \qquad (\textrm{Eq. 6.22})$

Proof. We need two properties of ordering of real numbers:

1. the inequality $latex u \le v$ is equivalent to the inequality $latex u + c \le v + c$ for any constant $latex c$,
2. the inequality $latex u \le v$ is equivalent to the inequality $latex uc \le vc$ for any positive constant $latex c$.

It follows that the next three inequalities are equivalent:

(i) $latex a \le {S_n} \le b$,

(ii) $latex a - E{S_n} \le {S_n} - E{S_n} \le b - E{S_n}$,

(iii) $latex \frac{{a - E{S_n}}}{{{\sigma _{{S_n}}}}} \le \frac{{{S_n} - E{S_n}}}{{{\sigma _{{S_n}}}}} \le \frac{{b - E{S_n}}}{{{\sigma _{{S_n}}}}}.$

Hence, probabilities of these events are the same. Now by Eq. (6.21)

$latex P(a \le {S_n} \le b) = P\left( \frac{{a - E{S_n}}}{{{\sigma _{{S_n}}}}} \le {z_n} \le \frac{{b - E{S_n}}}{{{\sigma _{{S_n}}}}} \right)$

$latex \approx P\left( \frac{{a - E{S_n}}}{{{\sigma _{{S_n}}}}} \le z \le \frac{{b - E{S_n}}}{{{\sigma _{{S_n}}}}} \right).$

Exercise 6.3. Obtain an approximate confidence interval for $latex \hat p = \frac{{{S_n}}}{n}$.

Previous: The normal distribution | Next: Joint distribution and independence

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.