Raising the bar

Chapter 5. Discrete Random Variables and Probability Distributions

Unit 5.12. An obvious and handy result

It is possible to calculate means and variances of $latex {S_n}$ and $latex \bar X$ by deriving their distributions and applying the definitions of the mean and variance directly (see Exercise 5.8 below). But there is a much easier way to do that, which should be obvious, if not now then at least after reading the proof below. It is as handy as it is indispensable in a number of applications.

The sum $latex {S_n} = {X_1} + ... + {X_n}$ and the sample mean $latex \bar X$ can be used when $latex {X_i}$ are drawn from an arbitrary population.

Number and proportion of successes. In the context of drawing from a Bernoulli population $latex {S_n}$ is called a number of successes (because only unities contribute to it). The sample mean $latex \bar X$ in this case is called a proportion of successes and denoted $latex \hat p$.

The mean and variance of $latex {S_n}$ and $latex \bar X$. (1) If $latex {X_1},...,{X_n}$ are i.i.d., then

$latex E{S_n} = n{\mu _X},\,\,V({S_n}) = n\sigma _X^2,\,\,E\bar X = {\mu _X},\,\,V(\bar X) = \frac{{\sigma _X^2}}{n}. \qquad (\textrm{Eq. 5.13})$

(2) If, additionally, $latex {S_n}$ is a binomial variable and $latex {p_X}$ denotes the parameter of the parent Bernoulli population, then Eq. (5.13) becomes

$latex E{S_n} = n{p_X},\,\,V({S_n}) = n{p_X}(1 - {p_X}),\,\,E\hat p = {p_X},\,\,V(\hat p) = \frac{{{p_X}(1 - {p_X})}}{n}. \qquad (\textrm{Eq. 5.14})$

Proof. (1) Here our derivation of algebraic properties of means and variances starts paying back.

$latex E{S_n} = E({X_1} + ... + {X_n})$ (by additivity)

$latex = E{X_1} + ... + E{X_n}$ ($latex {X_1},...,{X_n}$ are identically distributed)

$latex = nEX = n{\mu _X}.$

For variance we use also independence. Due to independence, all interaction terms vanish and variance is additive.

$latex V({S_n}) = V({X_1} + ... + {X_n})$ (by additivity)

$latex = V({X_1}) + ... + V({X_n})$ ($latex {X_1},...,{X_n}$ are identically distributed)

$latex = nV(X) = n\sigma _X^2.$

The results for the sample mean Eq. (5.11) follow by homogeneity:

$latex E\bar X = E\frac{1}{n}{S_n} = \frac{1}{n}E{S_n} = \frac{1}{n}n{\mu _X} = {\mu _X},$

$latex V(\bar X) = V\left( \frac{1}{n}{S_n} \right) = \frac{1}{{{n^2}}}V({S_n}) = \frac{1}{{{n^2}}}n\sigma _X^2 = \frac{{\sigma _X^2}}{n}.$

(2) In case of the binomial variable just replace $latex {\mu _X}$ by $latex {p_X}$ and $latex \sigma _X^2$ by $latex {p_X}(1 - {p_X})$.

Standard error. The standard deviation of the sample mean is called a standard error. From Eq. (5.13) it follows that the standard error is $latex \sigma _{\bar X} = \frac{{{\sigma _X}}}{{\sqrt n }}$.

Previous: Bernoulli variable | Next: Distribution of the binomial variable

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.