Raising the bar

Correlation coefficient: the last block of statistical foundation

Correlation has already been mentioned in

Statistical measures and their geometric roots

Properties of standard deviation

The pearls of AP Statistics 35

Properties of covariance

The pearls of AP Statistics 33

The hierarchy of definitions

Suppose random variables $latex X,Y$ are not constant. Then their standard deviations are not zero and we can define their correlation as in Chart 1.

Chart 1. Correlation definition

Properties of correlation

Property 1. Range of the correlation coefficient: for any $latex X,Y$ one has $latex - 1 \le \rho (X,Y) \le 1$.
This follows from the Cauchy-Schwarz inequality, as explained here.

Recall from this post that correlation is cosine of the angle between $latex X-EX$ and $latex Y-EY$.
Property 2. Interpretation of extreme cases. (Part 1) If $latex \rho (X,Y) = 1$, then $latex Y = aX + b$ with $latex a > 0.$

(Part 2) If $latex \rho (X,Y) = - 1$, then $latex Y = aX + b$ with $latex a < 0$.

Proof. (Part 1) $latex \rho (X,Y) = 1$ implies
(1) $latex Cov (X,Y) = \sigma (X)\sigma (Y)$
which, in turn, implies that $latex Y$ is a linear function of $latex X$: $latex Y = aX + b$ (this is the second part of the Cauchy-Schwarz inequality). Further, we can establish the sign of the number $latex a$. By the properties of variance and covariance
$latex Cov(X,Y)=Cov(X,aX+b)=aCov(X,X)+Cov(X,b)=aVar(X)$,

$latex \sigma (Y)=\sigma(aX + b)=\sigma (aX)=|a|\sigma (X)$.
Plugging this in Eq. (1) we get $latex aVar(X) = |a|\sigma^2(X)$ and see that $latex a$ is positive.

The proof of Part 2 is left as an exercise.

Property 3. Suppose we want to measure correlation between weight $latex W$ and height $latex H$ of people. The measurements are either in kilos and centimeters $latex {W_k},{H_c}$ or in pounds and feet $latex {W_p},{H_f}$. The correlation coefficient is unit-free in the sense that it does not depend on the units used: $latex \rho (W_k,H_c)=\rho (W_p,H_f)$. Mathematically speaking, correlation is homogeneous of degree $latex 0$ in both arguments.
Proof. One measurement is proportional to another, $latex W_k=aW_p,\ H_c=bH_f$ with some positive constants $latex a,b$. By homogeneity
$latex \rho (W_k,H_c)=\frac{Cov(W_k,H_c)}{\sigma(W_k)\sigma(H_c)}=\frac{Cov(aW_p,bH_f)}{\sigma(aW_p)\sigma(bH_f)}=\frac{abCov(W_p,H_f)}{ab\sigma(W_p)\sigma (H_f)}=\rho (W_p,H_f).$