Raising the bar

Properties of means, covariances and variances are bread and butter of professionals. Here we consider the bread - the means

Properties of means: as simple as playing with tables

Definition of a random variable. When my Brazilian students asked for an intuitive definition of a random variable, I said: It is a function whose values are unpredictable. Therefore it is prohibited to work with their values and allowed to work only with their various means. For proofs we need a more technical definition: it is a table values+probabilities of type Table 1.

Note: The complete form of writing $latex {p_i}$ is $latex P(X = {x_i})$.

Mean (or expected value) value definition. $latex EX = x_1p_1 + ... + x_np_n = \sum\limits_{i = 1}^nx_ip_i.$ In words, this is a weighted sum of values, where the weights $latex p_i$ reflect the importance of corresponding $latex x_i$.

Note: The expected value is a function whose argument is a complex object (it is described by Table 1) and the value is simple: $latex EX$ is just a number. And it is not a product of $latex E$ and $latex X$! See how different means fit this definition.

Definition of a linear combination. See here the financial motivation. Suppose that $latex X,Y$ are two discrete random variables with the same probability distribution $latex {p_1},...,{p_n}$. Let $latex a,b$ be real numbers. The random variable $latex aX + bY$ is called a linear combination of $latex X,Y$ with coefficients $latex a,b$. Its special cases are $latex aX$ ($latex X$ scaled by $latex a$) and $latex X + Y$ (a sum of $latex X$ and $latex Y$). The detailed definition is given by Table 2.

Note: The situation when the probability distributions are different is reduced to the case when they are the same, see my book.

Property 1. Linearity of means. For any random variables $latex X,Y$ and any numbers $latex a,b$ one has

(1) $latex E(aX + bY) = aEX + bEY$.

Proof. This is one of those straightforward proofs when knowing the definitions and starting with the left-hand side is enough to arrive at the result. Using the definitions in Table 2, the mean of the linear combination is
$latex E(aX + bY)= (a{x_1} + b{y_1}){p_1} + ... + (a{x_n} + b{y_n}){p_n}$

(distributing probabilities)
$latex = a{x_1}{p_1} + b{y_1}{p_1} + ... + a{x_n}{p_n} + b{y_n}{p_n}$

(grouping by variables)
$latex = (a{x_1}{p_1} + ... + a{x_n}{p_n}) + (b{y_1}{p_1} + ... + b{y_n}{p_n})$

(pulling out constants)
$latex = a({x_1}{p_1} + ... + {x_n}{p_n}) + b({y_1}{p_1} + ... + {y_n}{p_n})=aEX+bEY.$

See applications: one, and two, and three.

Generalization to the case of a linear combination of $latex n$ variables:

$latex E({a_1}{X_1} + ... + {a_n}{X_n}) = {a_1}E{X_1} + ... + {a_n}E{X_n}$.

Special cases. a) Letting $latex a = b = 1$ in (1) we get $latex E(X + Y) = EX + EY$. This is called additivity. See an application. b) Letting in (1) $latex b = 0$ we get $latex E(aX) = aEX$. This property is called homogeneity of degree 1 (you can pull the constant out of the expected value sign). Ask your students to deduce linearity from homogeneity and additivity.

Property 2. Expected value of a constant. Everybody knows what a constant is. Ask your students what is a constant in terms of Table 1. The mean of a constant is that constant, because a constant doesn't change, rain or shine: $latex Ec = c{p_1} + ... + c{p_n} = c({p_1} + ... + {p_n}) = 1$ (we have used the completeness axiom). In particular, it follows that $latex E(EX)=EX$.

Property 3. The expectation operator preserves order: if $latex x_i\ge y_i$ for all $latex i$, then $latex EX\ge EY$. In particular, the mean of a nonnegative random variable is nonnegative: if $latex x_i\ge 0$ for all $latex i$, then $latex EX\ge 0$.

Indeed, using the fact that all probabilities are nonnegative, we get $latex EX = x_1p_1 + ... + x_np_n\ge y_1p_1 + ... + y_np_n=EY$.

Property 4. For independent variables, we have $latex EXY=(EX)(EY)$ (multiplicativity), which has important implications on its own.

The best thing about the above properties is that, although we proved them under simplified assumptions, they are always true. We keep in mind that the expectation operator $latex E$ is the device used by Mother Nature to measure the average, and most of the time she keeps hidden from us both the probabilities and the average $latex EX$.