Raising the bar

Chapter 5. Discrete Random Variables and Probability Distributions

Unit 5.3. Linear combinations of random variables

Linear combination. Suppose that $latex X,Y$ are two discrete random variables with the same probability distribution $latex {p_1},...,{p_n}$. Let $latex a,b$ be real numbers. The random variable $latex aX + bY$ is called a linear combination of $latex X,Y$ with coefficients $latex a,b$. Its special cases are $latex aX$ ($latex X$ scaled by $latex a$) and $latex X + Y$ (a sum of $latex X$ and $latex Y$). The detailed definition is given by Table 5.2.

Table 5.2. Linear operations with random variables.

Comments. (1) There is only one column of probabilities because $latex X$ and $latex Y$ are assumed to have the same probability distribution. The variables in the last three columns have the same probability distribution by definition. The situation when the probability distributions are different is handled using joint probabilities (see Independence and its consequences).

(2) If you are asked what are vectors, you say the following. By a vector we call an $latex n$-tuple of real numbers $latex X = ({x_1},...,{x_n})$. Operations with vectors are defined by the last three columns of Table 5.2. When $latex n = 1$, vectors are just numbers and those operations are just usual summation and multiplication. Sometimes we need to multiply two vectors according to $latex XY = ({x_1}{y_1},...,{x_n}{y_n})$ (then what would be $latex {X^2}$?) These formal definitions plus the geometric interpretation considered next is all you need to know about vectors.

(3) Geometric interpretation. We live in the space of three-dimensional vectors. All our intuition coming from day-to-day geometrical experience carries over to the $latex n$-dimensional case. Let, for simplicity, $latex n = 2$. A vector $latex X = ({x_1},{x_2})$ is shown on the plane as an arrow starting at the origin and ending at the point $latex ({x_1},{x_2})$. The sum $latex X + Y = ({x_1} + {y_1},{x_2} + {y_2})$ is found using the parallelogram rule in Figure 5.1. The rule itself comes from physics: if two forces are applied to a point, their resultant force is found by the parallelogram rule.

Figure 5.1. Summing vectors using the parallelogram rule.

Figure 5.2. Scaling a vector.

Scaling $latex X$ by a positive number $latex a$ means lengthening it in case $latex a > 1$ and shortening in case $latex a < 1$. Scaling by a negative number means, additionally, reverting the direction of $latex X$, see Figure 5.2. To obtain a linear combination $latex aX + bY$ you first scale $latex X$ and $latex Y$ and then add the results.

Previous: General properties of means | Next: Linearity

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.