Raising the bar

Conditional expectation as a projector

This topic was discussed here and here. Looks like my explanations were not sufficient for some students, so here I give it a second try.

Examples of projectors

The name "projector" is almost self-explanatory. Imagine a point and a plane in the three-dimensional space (see the Wolfram embedded notebook below). Draw a perpendicular from the point to the plane. The intersection of the perpendicular with the plane is the point's projection onto that plane. Note that if the point already belongs to the plane, its projection equals the point itself. Besides, instead of projecting onto a plane we can project onto a straight line.

The above description translates into the following equations. For any point $latex (x,y,z)\in R^{3}$ define

(1) $latex P_{1}(x,y,z)=(x,0,0)$ and $latex P_{2}(x,y,z)=(x,y,0).$

In words: $latex P_{1}$ leaves the first two coordinates of the point unchanged and sets the third one to zero. (Describe yourself what $latex P_{2}$ does.)

$latex P_{2}$ projects $latex R^{3}$ onto the plane $latex L_{2}=\{(x,y,0):x,y\in R\}$ (which is two-dimensional) and $latex P_{1}$ projects $latex R^{3}$ onto the straight line $latex L_{1}=\{(x,0,0):x\in R\}$ (which is one-dimensional).

Property 1. Double application of a projector amounts to single application.

Proof. Using (1) three times we get

(1) $latex P_{2}[P_{2}(x,y,z)]=P_{2}(x,y,0)=(x,y,0)=P_{2}(x,y,z).$

Similarly, $latex P_{1}[P_{1}(x,y,z)]=P_{1}(x,y,z).$

Property 2. A successive application of two different projectors yields the projection onto a subspace of a smaller dimension.

Proof. If we apply first $latex P_{2}$ and then $latex P_{1}$, the result is

(2) $latex P_{1}[P_{2}(x,y,z)]=P_{1}(x,y,0)=(x,0,0)=P_{1}(x,y,z).$

If we change the order of projectors, we have

(3) $latex P_{2}[P_{1}(x,y,z)]=P_{2}(x,0,0)=(x,0,0)=P_{1}(x,y,z).$

Informational interpretation

If we know $latex P_{2}(x,y,z)$ then we know more about the original point $latex (x,y,z)$ than if we know $latex P_{1}(x,y,z).$ Thus, $latex P_{1}$ projects onto a smaller information set than $latex P_{2}.$ Equations (2) and (3) can be summarized as follows: the smaller info set wins.

Exercise 1. Show that both projectors are linear.

Exercise 2. Like any other linear operator in a Euclidean space, these projectors are given by some matrices. What are they?

The simple truth about conditional expectation

In the time series setup, we have a sequence of information sets

$latex ... \subset I{t}\subset I_{t+1}\subset ...$

where $latex t$ are points in time (naturally, with time the amount of available information increases). Denote

$latex E_tX=E(X|I_t)$

the expectation of $latex X$ conditional on $latex I_{t}$.

Statement. For each $latex t$, $latex E_{t}$ is a projector onto the space of random functions that depend only on the information set $latex I_{t}$.

The proof is too complex to discuss here but the implications are immediate.

Property 3. Applying Property 1, we see that double application of conditional expectation gives the same result as single application:

(4) $latex E_t(E_tX)=E_tX$

($latex E_{t}X$ is already a function of $latex I_{t}$, so conditioning it on $latex I_{t}$ doesn't change it).

Property 4. A successive conditioning on two different information sets is the same as conditioning on the smaller one:

(5) $latex E_tE_{t+1}X=E_tX,$

(6) $latex E_{t+1}E_{t}X=E_{t}X.$

(5) and (6) apply also in the case $latex E_t=E$ because for the unconditional expectation the information set is empty.

These lead to more general equations. Suppose, $latex s\leq t-1<t.$ Then using (5) many times, we get

$latex E_{s}E_{t}X=E_{s}E_{s+1}E_{t}X=...=E_{s}E_{s+1}...E_{t-1}E_{t}X=E_{s}E_{s+1}...E_{t-1}X=...=E_{s}X.$

Property 5. Conditional expectation is a linear operator: for any random variables $latex X,Y$ and random variables $latex a,b$ that depend only on $latex I_t$ we have

$latex E_t(aX+bY)=aE_tX+bE_tY.$

I prefer to use different names for (4)-(6). I call (4) a projector property. (5) is known as the Law of Iterated Expectations, see my post on the informational aspect for more intuition. (6) holds simply because at time $latex t+1$ the expectation $latex E_{t}X$ is known and behaves like a constant.

Summary. (4)-(6) can be combined into one property. The smaller information set wins: for any points in time $latex s,t$ we have $latex E_{s}E_{t}X=E_{\min \{s,t\}}X.$

In the next Wolfram embedded notebook you can manipulate both graphs to get a better angle of view.