Raising the bar

Chapter 2. Describing Data: Graphical

Unit 2.2. Frequencies and distributions

One of the implicit purposes of Chapter 2 is to show you that real-life data are huge and impossible to grasp without certain graphical tools or summary measures. I think at this point it is important for you to get an idea how the data arrive and how they are transformed for the purposes of statistical data processing. A general flaw of statistical texts is that they provide ready data and students don't see this transformation. Using an Excel spreadsheet will give you the feeling of the data generation process in real time. The explanation of all Excel functions we use can be found in Excel Help. The mathematical side of what we do will be explained in Unit 7.3 (Monte Carlo simulations).

Exercise 2.1. We are going to simulate a numerical two-valued variable (in Unit 5.11 it will be identified as the Bernoulli variable). In cell A1 type Bernoulli. In cell A2 enter the formula =IF(RAND()＞1/4,0,1) exactly as it is (no blanks). You can use lower-case letters but Excel will convert them to upper-case. On the Ribbon in "Formulas/Calculation Options" select "Manual". Now press F9 several times to see some realized values of the Bernoulli variable.

Exercise 2.2. This example models a variable with 4 integer values (we'll see in Unit 5.11 that it's a binomial variable). Copy the formula you entered in A2 to cells A3, A4 (you can select cell A2, pick with a mouse a small square in the lower right corner of the cell and pull it down for the selection to include cells A3, A4; the formula will be copied). In cell B1 type Binomial and in cell B2 enter the formula =SUM(A2:A4). By pressing F9 you can observe the realized values of the binomial variable. Many students consider it quite a headache. It will be, if you jump to the conclusions.

Exercise 2.3. This is the main exercise. We are going to collect a sample of observations on the binomial variable. By definition, the sample size, $latex n$, is the number of observations. Choose $latex n$, press F9 $latex n$ times and write down the values observed in cell B2: $latex {b_1}=...,\ {b_2}=...,\ ...,\ {b_n}=...$ Don't be lazy and use at least $latex n=30$.

1. After having pressed F9 $latex n$ times do you think you can predict what will show up next time? Intuitive definition of a random variable: a variable is called random (or stochastic) if its values cannot be predicted.
2. Alternatively, do you think you can tell what is NOT likely to show up next time? The answer I am pushing for is that any number except $latex 0,\,1,\,2,\,3$ is not likely to appear.
3. Denote $latex {x_i}$ distinct values in your sample. Most probably, they are $latex {x_1}=0,\,{x_2}=1,\,{x_3}=2,\,{x_4}=3$. Count the number of times $latex {x_i}$ appears in your sample and denote it $latex {n_i}$. These numbers are called absolute frequencies. Do you think their total is $latex n$:
   $latex {n_1}+{n_2}+{n_3}+{n_4}=n? \qquad (\textrm{Eq. 2.1})$
4. The numbers $latex {r_i}={n_i}/n$, $latex i=1,2,3,4$, are called relative frequencies. $latex {r_1}$, for example, shows the percentage of times the value $latex {x_1}$ appears in your sample. Do you think the relative frequencies sum to 1:
   $latex {r_1}+{r_2}+{r_3}+{r_4}=1? \qquad (\textrm{Eq. 2.2})$
   You can get this equation from Eq. (2.1) by dividing both sides by $latex n$.
5. Summarize your findings in Table 2.1.

Table 2.1. Absolute and relative frequencies (discrete numerical variable).

Conclusions. Suppose somebody else works with a different sample size. Your and that person's absolute frequencies will not be comparable because their totals are different. On the other hand, comparing relative frequencies will make sense because they sum to 1. Relative frequencies are better also because they are percentages. The columns of frequencies show how totals are distributed over the values of the variable. Therefore they are called absolute and relative frequency distributions, respectively. For convenience reasons, most of the time the last column of Table 2.1 is used and the adjective "relative" is omitted. Once you understand this basic situation, it is easy to understand possible variations, which we consider next.

Categorical variables. For example, suppose that at a given university there are four instructor ranks: lecturer, assistant professor, associate professor and full professor. Then instead of Table 2.1 we have Table 2.2.

Table 2.2. Absolute and relative frequencies (categorical variable).

Exercise 2.4. Here we model a continuous variable. See Unit 6.6 for the theory.

1. To model summer temperatures in my city, type Temperature in cell C1 and in cell C2 the formula =NORMINV(RAND(),25,8). The resulting numbers are real and may have many nonzero digits. Usually nobody reports temperatures with high precision. In "Home/Format/Format Cells/Number" select "Number" as the category and 1 decimal place in "Decimal places" and click OK.
2. Select the sample size and write down the observed values in form $latex t_1=...,\ t_2=...$ Since the elements in the sample are real numbers, none may be repeated and in that case all absolute frequencies are equal to 1. What is worse, if you obtain another sample, it may have very few common elements with the first one. To achieve stability over samples and reduce the table size, in such cases the observations are joined in ranges (groups, clusters). In case of temperatures you can use intervals of length 1:
   $latex {t_{\min}}=10\le t<11,\,\,\,...,\,\,\,39\le t<40={t_{\max}} \qquad (\textrm{Eq. 2.3})$
   (the values $latex {t_{\min}},\,{t_{\max}}$ will in fact depend on the sample and may be different from the ones you see here). In this approach the values in the first column of the frequency column will be intervals of Eq. (2.3).
3. For each interval, you calculate the number of observations falling into that interval to find the absolute frequency. Since the observed values are replaced by the intervals they fall into, under this approach a part of the information contained in the sample is lost. However, if the lengths of the intervals are reduced, the loss decreases and the resulting frequencies represent the variables very well. Denoting $latex k$ the number of intervals, we obtain a table of type Table 2.3.

Table 2.3. Absolute and relative frequencies (continuous variable).

Previous: Classification of variables | Next: Visualizing statistical data

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.