25
Oct 16

## Properties of variance

### All properties of variance in one place

Certainty is the mother of quiet and repose, and uncertainty the cause of variance and contentions. Edward Coke

Preliminaries: study properties of means with proofs.

Definition. Yes, uncertainty leads to variance, and we measure it by $Var(X)=E(X-EX)^2$. It is useful to use the name deviation from mean for $X-EX$ and realize that $E(X-EX)=0$, so that the mean of the deviation from mean cannot serve as a measure of variation of $X$ around $EX$.

Property 1. Variance of a linear combination. For any random variables $X,Y$ and numbers $a,b$ one has
(1) $Var(aX + bY)=a^2Var(X)+2abCov(X,Y)+b^2Var(Y).$
The term $2abCov(X,Y)$ in (1) is called an interaction term. See this post for the definition and properties of covariance.
Proof.
$Var(aX + bY)=E[aX + bY -E(aX + bY)]^2$

(using linearity of means)
$=E(aX + bY-aEX -bEY)^2$

(grouping by variable)
$=E[a(X-EX)+b(Y-EY)]^2$

(squaring out)
$=E[a^2(X-EX)^2+2ab(X-EX)(Y-EY)+(Y-EY)^2]$

(using linearity of means and definitions of variance and covariance)
$=a^2Var(X) + 2abCov(X,Y) +b^2Var(Y).$
Property 2. Variance of a sum. Letting in (1) $a=b=1$ we obtain
$Var(X + Y) = Var(X) + 2Cov(X,Y)+Var(Y).$

Property 3. Homogeneity of degree 2. Choose $b=0$ in (1) to get
$Var(aX)=a^2Var(X).$
Exercise. What do you think is larger: $Var(X+Y)$ or $Var(X-Y)$?
Property 4. If we add a constant to a variable, its variance does not change: $Var(X+c)=E[X+c-E(X+c)]^2=E(X+c-EX-c)^2=E(X-EX)^2=Var(X)$
Property 5. Variance of a constant is zero: $Var(c)=E(c-Ec)^2=0$.

Property 6. Nonnegativity. Since the squared deviation from mean $(X-EX)^2$ is nonnegative, its expectation is nonnegative$E(X-EX)^2\ge 0$.

Property 7. Only a constant can have variance equal to zero: If $Var(X)=0$, then $E(X-EX)^2 =(x_1-EX)^2p_1 +...+(x_n-EX)^2p_n=0$, see the definition of the expected value. Since all probabilities are positive, we conclude that $x_i=EX$ for all $i$, which means that $X$ is identically constant.

Property 8. Shortcut for variance. We have an identity $E(X-EX)^2=EX^2-(EX)^2$. Indeed, squaring out gives

$E(X-EX)^2 =E(X^2-2XEX+(EX)^2)$

(distributing expectation)

$=EX^2-2E(XEX)+E(EX)^2$

(expectation of a constant is constant)

$=EX^2-2(EX)^2+(EX)^2=EX^2-(EX)^2$.

All of the above properties apply to any random variables. The next one is an exception in the sense that it applies only to uncorrelated variables.

Property 9. If variables are uncorrelated, that is $Cov(X,Y)=0$, then from (1) we have $Var(aX + bY)=a^2Var(X)+b^2Var(Y).$ In particular, letting $a=b=1$, we get additivity$Var(X+Y)=Var(X)+Var(Y).$ Recall that the expected value is always additive.

Generalizations$Var(\sum a_iX_i)=\sum a_i^2Var(X_i)$ and $Var(\sum X_i)=\sum Var(X_i)$ if all $X_i$ are uncorrelated.

Among my posts, where properties of variance are used, I counted 12 so far.

### 19 Responses for "Properties of variance"

1. […] sizes. The modern explanation is different: the t statistic arises from replacing the unknown population variance by its estimator, the sample variance, and it works regardless of the sample size. If we take a […]

2. […] of means, covariances and variances are bread and butter of professionals. Here we consider the bread - the […]

3. […] 2. Variance of the sample mean […]

4. […] To find variance, we use the shortcut: […]

5. […] 3. For independent variables, variance is additive: This easily follows from the general formula for  and equation […]

6. […] 2. The sample variance unbiasedly estimates the population variance: where . Its analog: the sample variance converges to a spike at the population […]

7. […] we derived the OLS estimators. To distinguish between sample and population means, the variance and covariance in the slope estimator will be provided with the subscript u (for "uniform", see the […]

8. […] the fact that variance is homogeneous of degree 2, we […]

9. […] are done by analogy, which students love. Fourthly, this topic paves the road to properties of the variance and covariance (recall that the slope in simple regression is covariance over […]

10. […] variances are also constant (same thing, they do not depend on ), […]

11. […] Shortcut for covariance. This is a name of the following identity […]

12. […] not everybody is good at them and 2) they clog the picture. For this reason alone it is worth using variances and covariances. For example, if are observations, let us call by variance the quantity  where  […]

13. […] the properties of means and variances we see […]

14. […] The positive one is called an arithmetic square root. The arithmetic root is applied here to (see properties of variance), so standard deviation is always nonnegative. Definition. An absolute value of a real number is […]

15. […] Variance, covariance, standard deviation and correlation: their definitions and properties are deeply rooted in the Euclidean geometry. […]

16. […] Cauchy-Schwarz inequality). Further, we can establish the sign of the number . By the properties of variance and covariance […]

17. […] usual expectations by their conditional counterparts in the definition of variance, we obtain the definition of conditional […]

18. […] change was caused by the fact that students were not familiar with properties of covariances and variances. Dougherty has a Statistics review chapter for a […]

19. […] (all other) processes. How do you define the good ones? (Hint: Properties of means, covariances and variances are bread and butter of […]