Raising the bar

Chapter 3. Describing Data: Numerical

Unit 3.5. Measures of relationships

This is another complex topic which will be fully explained in Chapter 5.

Sample covariance. A sample covariance is defined by

$latex s_{xy}^{} = \frac{1}{{n - 1}}\left[ {({x_1} - \bar x)({y_1} - \bar y) + ... + ({x_n} - \bar x)({y_n} - \bar y)} \right] \qquad (\textrm{Eq. 3.5})$

where $latex ({x_i},{y_i})$ are observations on a pair of variables. It is an auxiliary algebraic device that doesn't have much geometric or statistical meaning by itself.

To facilitate understanding of the definition, compare it to $latex s_x^2$ and note that $latex {s_{xx}} = s_x^2$. After Exercise 3.1 (see Measures of variability) you should know how to calculate $latex s_{xy}^{}$.

Sample correlation coefficient. A sample correlation coefficient is defined by

$latex {r_{xy}} = \frac{{{s_{xy}}}}{{{s_x}{s_y}}}$

where $latex {s_{xy}}$ is the sample covariance and $latex {s_x},\,{s_y}$ are sample standard deviations.

These definitions are rooted in the Euclidean geometry. In particular, in Unit 5.9 we shall see that

$latex {r_{xy}} = \textrm{cosine of the angle between vectors}\ x,y.$

For those who know properties of cosine this explains why the range of $latex {r_{xy}}$ is the segment $latex [-1,1]$.

Possibilities for correlation coefficient. We say that the variables $latex x,y$ are positively correlated if $latex {r_{xy}} > 0$ and negatively correlated if $latex {r_{xy}} < 0$. In case $latex {r_{xy}} = 0$ the variables are called uncorrelated. In the extreme cases $latex {r_{xy}} = \pm 1$ we talk about perfect correlation.

We use Excel to see what these names mean in terms of a scatterplot.

Exercise 3.3.

(1) From Exercise 3.1 (see Measures of variability) it should be clear that calculating the sample correlation coefficient is a long story, even in Excel. We are going to take a shortcut using a built-in function CORREL. In Exercise 2.5 (see Visualizing statistical data) we have plotted the scatterplot of the pair (Height, Weight). To find the correlation coefficient for the same pair, in cell J1 type Positive correlation and in cell J2 the formula =CORREL(E2:E21,F2:F21).

(2) To generate a variable that is negatively correlated with Height, in K1 type Negative, in K2 put the formula =-E2+NORMINV(RAND(),0,20) and copy it to K3-K21. In L1 type Negative correlation and in L2 put the formula =CORREL(E2:E21,K2:K21). You know how to produce a scatterplot.

(3) For Case 1 below you might want to arrange cases of perfect positive and negative correlation. Just use formulas like =2*E2 and =-E2. Simulation of uncorrelated variables can be done using Exercise 10.3.

Case 1: $latex {r_{xy}} = 1$. There is an exact linear relationship between $latex {y_i},\,{x_i}$ in the form $latex {y_i} = a + b{x_i}$ with a positive $latex b$.

Case 2: $latex 0 < {r_{xy}} < 1$. There is an approximate linear relationship between $latex {y_i},\,{x_i}$ in the form $latex {y_i} \approx a + b{x_i}$ with a positive $latex b$.

Table 3.3.2.

Case 3: $latex {r_{xy}} = 0$. There is no definite pattern in the dependence.

Table 3.3.3.

Case 4: $latex -1 < {r_{xy}} < 0$. There is an approximate linear relationship between $latex {y_i},\,{x_i}$ in the form $latex {y_i} \approx a + b{x_i}$ with a negative $latex b$.

Table 3.3.4.

[INSERT IMAGE: Table3.3.4.png]

Case 5: $latex {r_{xy}} = -1$. There is an exact linear relationship between $latex {y_i},\,{x_i}$ in the form $latex {y_i} = a + b{x_i}$ with a negative $latex b$. The scatterplot is the mirror image of the one in Case 1, reflected about a horizontal line.

Exercise 3.4. Take some pairs of the following variables: inflation, unemployment, aggregate income, investment, government expenditures, per capita income and birth rate. For each pair you take indicate the expected sign of the correlation coefficient.

Previous: Measures of variability | Next: Questions for repetition (Chapter 3)

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.