Raising the bar

Preliminaries

Review Properties of conditional expectation, especially the summary, where I introduce a new notation for conditional expectation. Everywhere I use the notation $latex E_Y\pi$ for expectation of $latex \pi$ conditional on $latex Y$, instead of $latex E(\pi|Y)$.

This post and the previous one on conditional expectation show that conditioning is a pretty advanced notion. Many introductory books use the condition $latex E_xu=0$ (the expected value of the error term $latex u=0$ conditional on the regressor $latex x$ is zero). Because of the complexity of conditioning, I think it's better to avoid this kind of assumption as much as possible.

Conditional variance properties

Replacing usual expectations by their conditional counterparts in the definition of variance, we obtain the definition of conditional variance:

(1) $latex Var_Y(X)=E_Y(X-E_YX)^2.$

Property 1. If $latex X,Y$ are independent, then $latex X-EX$ and $latex Y$ are also independent and conditioning doesn't change variance:

$latex Var_Y(X)=E_Y(X-EX)^2=E(X-EX)^2=Var(X),$

see Conditioning in case of independence.

Property 2. Generalized homogeneity of degree 2: if $latex a$ is a deterministic function, then $latex a^2(Y)$ can be pulled out:

$latex Var_Y(a(Y)X)=E_Y[a(Y)X-E_Y(a(Y)X)]^2=E_Y[a(Y)X-a(Y)E_YX]^2$

$latex =E_Y[a^2(Y)(X-E_YX)^2]=a^2(Y)E_Y(X-E_YX)^2=a^2(Y)Var_Y(X).$

Property 3. Shortcut for conditional variance:

(2) $latex Var_Y(X)=E_Y(X^2)-(E_YX)^2.$

Proof.

$latex Var_Y(X)=E_Y(X-E_YX)^2=E_Y[X^2-2XE_YX+(E_YX)^2]$

(distributing conditional expectation)

$latex =E_YX^2-2E_Y(XE_YX)+E_Y(E_YX)^2$

(applying Properties 2 and 6 from this Summary with $latex a(Y)=E_YX$)

$latex =E_YX^2-2(E_YX)^2+(E_YX)^2=E_YX^2-(E_YX)^2.$

Property 4. The law of total variance:

(3) $latex Var(X)=Var(E_YX)+E[Var_Y(X)].$

Proof. By the shortcut for usual variance and the law of iterated expectations

$latex Var(X)=EX^2-(EX)^2=E[E_Y(X^2)]-[E(E_YX)]^2$

(replacing $latex E_Y(X^2)$ from (2))

$latex =E[Var_Y(X)]+E(E_YX)^2-[E(E_YX)]^2$

(the last two terms give the shortcut for variance of $latex E_YX$)

$latex =E[Var_Y(X)]+Var(E_YX).$

Before we move further we need to define conditional covariance by

$latex Cov_Y(S,T) = E_Y(S - E_YS)(T - E_YT)$

(everywhere usual expectations are replaced by conditional ones). We say that random variables $latex S,T$ are conditionally uncorrelated if $latex Cov_Y(S,T) = 0$.

Property 5. Conditional variance of a linear combination. For any random variables $latex S,T$ and functions $latex a(Y),b(Y)$ one has

$latex Var_Y(a(Y)S + b(Y)T)=a^2(Y)Var_Y(S)+2a(Y)b(Y)Cov_Y(S,T)+b^2(Y)Var_Y(T).$

The proof is quite similar to that in case of usual variances, so we leave it to the reader. In particular, if $latex S,T$ are conditionally uncorrelated, then the interaction terms disappears:

$latex Var_Y(a(Y)S + b(Y)T)=a^2(Y)Var_Y(S)+b^2(Y)Var_Y(T).$