Raising the bar

Chapter 5. Discrete Random Variables and Probability Distributions

Unit 5.10. Standardization

Questions. The motivation here is purely mathematical. (1) What can be done to a random variable $latex X$ to obtain a new random variable with mean zero?

(2) What can be done to $latex X$ to obtain a new variable with unit variance (and standard deviation)?

Digression on linear transformations. Let us look at what happens to the distribution of $latex X$ when a linear transformation $latex Y = aX + b$ is applied to it. From Linear combinations of random variables it is clear that $latex Y = aX + b$ takes values $latex a{x_i} + b$.

Consider a special case $latex a = 1$. When a constant $latex b$ is added to $latex X$, all values of $latex X$ move to the right ($latex b > 0$) or to the left ($latex b < 0$) by the same amount. Correspondingly, the histogram moves right or left as a rigid body (the distances between values do not change).

Consider another special case $latex b = 0$. If $latex X$ is multiplied by $latex a$, its values will be stretched ($latex |a| > 1$) or contracted towards the origin ($latex |a| < 1$). In case of stretching its variance increases and in case of contraction it decreases.

In the general case stretching/contraction is combined with movement along the x-axis. This digression explains the intuition behind the following answers.

Answers. (1) To obtain a variable with mean zero out of $latex X$ we subtract its mean: $latex E(X - EX) = EX - EX = 0$.

(2) To obtain a variable with unit variance out of $latex X$ we divide it by $latex \sigma (X)$ (provided it's not zero):

$latex V\left( {\frac{X}{{\sigma (X)}}} \right) = \textrm{(by homogeneity)} = \frac{1}{{{\sigma ^2}(X)}}V(X) = 1.$

To satisfy both requirements we combine the two transformations:

$latex Y = \frac{{X - EX}}{{\sigma (X)}}. \qquad (\textrm{Eq. 5.9})$

Standardization. The transformation of $latex X$ defined in Eq. (5.9) is called standardization and $latex Y$ is called a standardized version of $latex X$.

Exercise 5.7. Check that Eq. (5.9) has mean zero and unit variance.

In NCT the mean, variance and standard deviation are denoted by $latex \mu$, $latex {\sigma ^2}$ and $latex \sigma$, respectively. Always put a subscript to reflect the variable in question in this notation:

$latex \mu _X = EX,\ \sigma _X^2 = V(X),\ \sigma_X = \sqrt {V(X)} = \sigma (X).$

Previous: Correlation coefficient | Next: Bernoulli variable

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.