Raising the bar

All properties of variance in one place

Certainty is the mother of quiet and repose, and uncertainty the cause of variance and contentions. Edward Coke

Preliminaries: study properties of means with proofs.

Definition. Yes, uncertainty leads to variance, and we measure it by $latex Var(X)=E(X-EX)^2$. It is useful to use the name deviation from mean for $latex X-EX$ and realize that $latex E(X-EX)=0$, so that the mean of the deviation from mean cannot serve as a measure of variation of $latex X$ around $latex EX$.

Property 1. Variance of a linear combination. For any random variables $latex X,Y$ and numbers $latex a,b$ one has
(1) $latex Var(aX + bY)=a^2Var(X)+2abCov(X,Y)+b^2Var(Y).$
The term $latex 2abCov(X,Y)$ in (1) is called an interaction term. See this post for the definition and properties of covariance.
Proof.
$latex Var(aX + bY)=E[aX + bY -E(aX + bY)]^2$

(using linearity of means)
$latex =E(aX + bY-aEX -bEY)^2$

(grouping by variable)
$latex =E[a(X-EX)+b(Y-EY)]^2$

(squaring out)
$latex =E[a^2(X-EX)^2+2ab(X-EX)(Y-EY)+(Y-EY)^2]$

(using linearity of means and definitions of variance and covariance)
$latex =a^2Var(X) + 2abCov(X,Y) +b^2Var(Y).$
Property 2. Variance of a sum. Letting in (1) $latex a=b=1$ we obtain
$latex Var(X + Y) = Var(X) + 2Cov(X,Y)+Var(Y).$

Property 3. Homogeneity of degree 2. Choose $latex b=0$ in (1) to get
$latex Var(aX)=a^2Var(X).$
Exercise. What do you think is larger: $latex Var(X+Y)$ or $latex Var(X-Y)$?
Property 4. If we add a constant to a variable, its variance does not change: $latex Var(X+c)=E[X+c-E(X+c)]^2=E(X+c-EX-c)^2=E(X-EX)^2=Var(X)$
Property 5. Variance of a constant is zero: $latex Var(c)=E(c-Ec)^2=0$.

Property 6. Nonnegativity. Since the squared deviation from mean $latex (X-EX)^2$ is nonnegative, its expectation is nonnegative: $latex E(X-EX)^2\ge 0$.

Property 7. Only a constant can have variance equal to zero: If $latex Var(X)=0$, then $latex E(X-EX)^2 =(x_1-EX)^2p_1 +...+(x_n-EX)^2p_n=0$, see the definition of the expected value. Since all probabilities are positive, we conclude that $latex x_i=EX$ for all $latex i$, which means that $latex X$ is identically constant.

Property 8. Shortcut for variance. We have an identity $latex E(X-EX)^2=EX^2-(EX)^2$. Indeed, squaring out gives

$latex E(X-EX)^2 =E(X^2-2XEX+(EX)^2)$

(distributing expectation)

$latex =EX^2-2E(XEX)+E(EX)^2$

(expectation of a constant is constant)

$latex =EX^2-2(EX)^2+(EX)^2=EX^2-(EX)^2$.

All of the above properties apply to any random variables. The next one is an exception in the sense that it applies only to uncorrelated variables.

Property 9. If variables are uncorrelated, that is $latex Cov(X,Y)=0$, then from (1) we have $latex Var(aX + bY)=a^2Var(X)+b^2Var(Y).$ In particular, letting $latex a=b=1$, we get additivity: $latex Var(X+Y)=Var(X)+Var(Y).$ Recall that the expected value is always additive.

Generalizations. $latex Var(\sum a_iX_i)=\sum a_i^2Var(X_i)$ and $latex Var(\sum X_i)=\sum Var(X_i)$ if all $latex X_i$ are uncorrelated.

Among my posts, where properties of variance are used, I counted 12 so far.