Raising the bar

Chapter 5. Discrete Random Variables and Probability Distributions

Unit 5.4. Linearity

Linearity is one of those many properties in which both the statement of the problem and its solution are mathematical.

Problem statement. What is the relation between $latex E(aX + bY)$ and $latex EX,\,EY$?

Answer. For any random variables $latex X,\,Y$ and any numbers $latex a,\,b$ one has $latex E(aX + bY) = aEX + bEY$.

Proof. This is one of those straightforward proofs when knowing the definitions and starting with the left-hand side is enough to arrive at the result. Using the definitions in Table 5.2 (see Linear combinations of random variables) the mean of the linear combination is

$latex E(aX + bY) = (a{x_1} + b{y_1}){p_1} + ... + (a{x_n} + b{y_n}){p_n}$ (distributing probabilities)

$latex = a{x_1}{p_1} + b{y_1}{p_1} + ... + a{x_n}{p_n} + b{y_n}{p_n}$ (grouping by variables)

$latex = (a{x_1}{p_1} + ... + a{x_n}{p_n}) + (b{y_1}{p_1} + ... + b{y_n}{p_n})$ (pulling out constants)

$latex = a({x_1}{p_1} + ... + {x_n}{p_n}) + b({y_1}{p_1} + ... + {y_n}{p_n})$

$latex = aEX + bEY.$

Chewing and digesting

Application. A portfolio contains $latex {n_1}$ shares of stock 1 whose price is $latex {S_1}$ and $latex {n_2}$ shares of stock 2 whose price is $latex {S_2}$. Stock prices fluctuate and are random variables. Numbers of shares are assumed fixed and are deterministic. What is the expected value of the portfolio?

Answer. The portfolio value is its market price $latex V = {n_1}{S_1} + {n_2}{S_2}$. Since this is a linear combination, the expected value is $latex EV = {n_1}E{S_1} + {n_2}E{S_2}$.

Going sideways. (1) Rewrite the proof of linearity of means using summation signs.

(2) Do you think $latex E({X^2})$ is the same as $latex {(EX)^2}$? Hint: It should be clear that $latex {X^2} = (x_1^2,...,x_n^2)$ and $latex E{X^2} = x_1^2{p_1} + ... + x_n^2{p_n}$, see Linear combinations of random variables.

(3) While I don't give an answer for the previous question, I want to use it to discuss the difference between universal statements (which should be true for a class of objects) and existence statements (which assert existence of objects with special properties). If you say: Yes, I think that $latex E({X^2}) = {(EX)^2}$, it should be a universal statement: for ANY random variable $latex X$ one has $latex E({X^2}) = {(EX)^2}$. If, on the other hand, you say: No, I don't think so, then it should be an existence statement: I am producing $latex X$ such that $latex E({X^2}) \ne {(EX)^2}$ (whereas for other variables the equation may be true). Pay attention to the order of words when you use "for every" and "there exists". In the next two propositions the words are the same, just their order is different:

• (i) In the nursery school X, for every kid K there is a toy T such that K plays with T (T depends on K).
• (ii) In the nursery school X, there is a toy T such that every kid K plays with T (one T for all kids).

Going forward. (1) Use induction to prove the generalization to the case of a linear combination of $latex n$ variables: $latex E({a_1}{X_1} + ... + {a_n}{X_n}) = {a_1}E{X_1} + ... + {a_n}E{X_n}$.

(2) Expected value of a constant. The mean of a constant is that constant, $latex Ec = c{p_1} + ... + c{p_n} = c({p_1} + ... + {p_n}) = c$, because a constant doesn't change, rain or shine.

Homogeneity. Letting in the linearity property $latex b = 0$ we get $latex E(aX) = aEX$. This property is called homogeneity of degree 1 (you can pull the constant out of the expected value sign).

Additivity. Letting $latex a = b = 1$ in the linearity property we get $latex E(X + Y) = EX + EY$. This is called additivity.

Can you deduce linearity from homogeneity and additivity?

Previous: Linear combinations of random variables | Next: Independence and its consequences

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.