Correlation coefficient: the last block of statistical foundation
Correlation has already been mentioned in
The hierarchy of definitions
Suppose random variables are not constant. Then their standard deviations are not zero and we can define their correlation as in Chart 1.
Properties of correlation
Property 1. Range of the correlation coefficient: for any one has .
This follows from the Cauchy-Schwarz inequality, as explained here.
Recall from this post that correlation is cosine of the angle between and .
Property 2. Interpretation of extreme cases. (Part 1) If , then with
(Part 2) If , then with .
Proof. (Part 1) implies
which, in turn, implies that is a linear function of : (this is the second part of the Cauchy-Schwarz inequality). Further, we can establish the sign of the number . By the properties of variance and covariance
Plugging this in Eq. (1) we get and see that is positive.
The proof of Part 2 is left as an exercise.
Property 3. Suppose we want to measure correlation between weight and height of people. The measurements are either in kilos and centimeters or in pounds and feet . The correlation coefficient is unit-free in the sense that it does not depend on the units used: . Mathematically speaking, correlation is homogeneous of degree in both arguments.
Proof. One measurement is proportional to another, with some positive constants . By homogeneity