Raising the bar

Chapter 7. Sampling and Sampling Distribution

Unit 7.1. Sampling distribution

Example 7.1. Suppose we are interested in finding the average income of the population of Kazakhstan. The random variable is income of a person. Its mean is the parameter of the population we are interested in. Observing the whole population is costly. Therefore we want to obtain a sample and make inference (conclusions) about the population from the sample information. Review the theory we've studied so far and satisfy yourself that most theoretical facts critically depend on two assumptions: randomness and independence of observations.

Definition 7.1 (Simple random sample). A simple random sample satisfies two conditions:

a. Every object has an equal probability of being selected, and
b. The objects are selected independently.

Comments. (1) Condition (a) can be satisfied by shuffling in case of playing cards. If a list of objects is available, it can be satisfied using random numbers generated on the computer. Suppose we have a list of $latex N$ objects to choose from. The function RAND() (with empty parentheses) from Excel is uniformly distributed over the segment $latex (0,1)$. Then N*RAND() is uniformly distributed over $latex (0,N)$. Applying the function INT, which rounds a number down to the closest integer, we get INT(N*RAND()) that takes integer values from 0 to $latex N - 1$.

(2) It's good to know a couple of examples when the above conditions are violated. One example is sampling without replacement (see Bernoulli variable). Another is quota sampling. In quota sampling elements are sampled until certain quotas are filled. For instance, if the percentage of men in the country is 49 and the percentage of employed is 80, one might want to keep the same percentages in the sample. The problem with this type of sampling is dependence (inclusion of a new element in the sample depends on the previously selected ones) and impossibility to satisfy more than one quota simultaneously, unless by coincidence.

Definition 7.2 (Statistic and sampling distribution $latex -$ full definition). Let $latex \tau$ be a population parameter (for example, $latex \tau = \mu_X$ can be the population mean) and let $latex T$ be the sample characteristic designed to estimate $latex \tau$ (for example, the sample mean is a good estimator for the population mean because $latex E\bar X = \mu_X$). $latex T$ is constructed from sample observations $latex X_1, \ldots, X_n$ and in theory these observations are thought of as ex-ante random variables. Therefore $latex T$ is a random variable and, as any random variable, it has a distribution. In the situation just described, $latex T$ is called a statistic and its distribution is called a sampling distribution.

Definition 7.3 (Statistic and sampling distribution $latex -$ working definition). A statistic is a random variable constructed from sample observations and designed to estimate a parameter of interest. A sampling distribution is a table "values + probabilities" for the statistic.

Comments. (1) Compare this definition with the definition of the sampling distribution from Chapter 6 of NCT: "The sampling distribution of the sample mean is the probability distribution of the sample means obtained from all possible samples of the same number of observations drawn from the population". This is a good way to confuse people. Firstly, it is difficult to think in terms of "all possible samples". Secondly, such a definition does not help in finding the sampling distribution. We found the distribution of the binomial variable $latex S_3$ in Distribution of the binomial variable without appealing to "all possible samples". Thirdly, for i.i.d. variables one can calculate the means and variances of $latex S_n$ and $latex \bar X$ without knowing their sampling distributions, as we did in Eq. (5.13).

(2) The sampling distribution of any statistic depends on the sample size, and as the sample size increases, that distribution improves in very specific ways. This is the only reason why the notion of the sampling distribution matters. Below we summarize what we know about how the sampling distribution of the sample mean improves with the sample size.

Asymptotic properties of the sample mean. (1) As the sample size increases, the sampling distribution of the sample mean collapses to a spike around the population mean, so that the estimation precision improves as the sample grows.

(2) As the sample size increases, the sampling distribution of the standardized version of the sample mean approaches the distribution of the standard normal.

The first statement follows from Eq. (5.13): $latex V(\bar X) \to 0$ as $latex n \to \infty$. The second statement follows from the first comment to Theorem 6.1.

Previous: Questions for repetition (Chapter 6) | Next: Sampling distribution of sample variance

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.