Raising the bar

Chapter 5. Discrete Random Variables and Probability Distributions

Unit 5.8. Standard deviation

Standard deviation. For a random variable $latex X$, the quantity $latex \sigma (X) = \sqrt {V(X)}$ is called its standard deviation.

In general, there are two square roots of a positive number, one positive and the other negative. The positive one is called an arithmetic square root. An arithmetic root is applied here to $latex V(X) \ge 0$, so standard deviation is always nonnegative.

Absolute values. An absolute value of a real number $latex a$ is defined by

$latex |a| = a,\ \textrm{if } a \textrm{ is nonnegative}, |a|=-a\ \textrm{if } a \textrm{ is negative}. \qquad (\textrm{Eq. 5.7})$

It is introduced to measure the distance from point $latex a$ to the origin. For example, $latex \textrm{dist}(3,0) = |3| = 3$ and $latex \textrm{dist}(-3,0) = |-3| = 3$. More generally, for any points $latex a,b$ on the real line the distance between them is given by $latex \textrm{dist}(a,b) = |a - b|$.

The distance interpretation of the absolute value makes it clear that

1. the condition $latex a \in [-b,b]$ is equivalent to $latex |a| = \textrm{dist}(a,0) \le b$ and
2. the condition $latex a \notin [-b,b]$ is equivalent to $latex |a| = \textrm{dist}(a,0) > b$.

By squaring both sides in Eq. (5.7) we obtain $latex |a|^2 = {a^2}$. Application of the arithmetic square root gives $latex |a| = \sqrt {a^2}$. This is the equation we need right now.

Property 1. Standard deviation is homogeneous of degree 1:

$latex \sigma (aX) = \sqrt {V(aX)} = \sqrt {{a^2}V(X)} = |a|\sigma (X).$

Property 2. Cauchy-Schwarz inequality. (1) For any random variables $latex X,Y$ one has $latex |{\mathop{\rm cov}} (X,Y)| \le \sigma (X)\sigma (Y)$.

(2) If the inequality sign turns into equality, $latex |{\mathop{\rm cov}} (X,Y)| = \sigma (X)\sigma (Y)$, then $latex Y$ is a linear function of $latex X$: $latex Y = aX + b$.

Proof. (1) If at least one of the variables is constant, both sides of the inequality are 0 and there is nothing to prove. To exclude the trivial case, let $latex X,Y$ be nonconstant and, therefore, $latex V(X),\,\,V(Y)$ are positive. Consider a real-valued function of a real number $latex t$ defined by $latex f(t) = V(tX + Y)$. Here we have variance of a linear combination $latex f(t) = {t^2}V(X) + 2t{\mathop{\rm cov}} (X,Y) + V(Y)$.

We see that $latex f(t)$ is a parabola with branches looking upward (because the senior coefficient $latex V(X)$ is positive). By nonnegativity of variance, $latex f(t) \ge 0$ and the parabola lies above the horizontal axis in the $latex (f,t)$ plane. Hence, the quadratic equation $latex f(t) = 0$ may have at most one real root. This means that the discriminant of the equation is non-positive: $latex D = {[{\mathop{\rm cov}} (X,Y)]^2} - V(X)V(Y) \le 0.$ Applying square roots to both sides of $latex {[{\mathop{\rm cov}} (X,Y)]^2} \le V(X)V(Y)$ we finish the proof of the first part.

(2) In case of the equality sign the discriminant is 0. Therefore the parabola touches the horizontal axis where $latex f(t) = V(tX + Y) = 0$. But we know that this implies $latex tX + Y = c$ which is just another way of writing $latex Y = aX + b$.

Do you think this proof is tricky? During the long history of development of mathematics mathematicians have invented many tricks, small and large. No matter how smart you are, you cannot reinvent all of them. Studying the existing tricks is a necessity. By the way, the definition of what is tricky and what is not is strictly personal and time-dependent.

Previous: Variance | Next: Correlation coefficient

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.