Law of iterated expectations: geometric aspect
There will be a separate post on projectors. In the meantime, we'll have a look at simple examples that explain a lot about conditional expectations.
Examples of projectors
The name "projector" is almost self-explanatory. Imagine a point and a plane in the three-dimensional space. Draw a perpendicular from the point to the plane. The intersection of the perpendicular with the plane is the points's projection onto that plane. Note that if the point already belongs to the plane, its projection equals the point itself. Besides, instead of projecting onto a plane we can project onto a straight line.
The above description translates into the following equations. For any define
projects onto the plane (which is two-dimensional) and projects onto the straight line (which is one-dimensional).
Property 1. Double application of a projector amounts to single application.
Proof. We do this just for one of the projectors. Using (1) three times we get
Property 2. A successive application of two projectors yields the projection onto a subspace of a smaller dimension.
Proof. If we apply first and then , the result is
If we change the order of projectors, we have
Exercise 1. Show that both projectors are linear.
Exercise 2. Like any other linear operator in a Euclidean space, these projectors are given by some matrices. What are they?
The simple truth about conditional expectation
In the time series setup, we have a sequence of information sets (it's natural to assume that with time the amount of available information increases). Denote
the expectation of conditional on . For each ,
|is a projector onto the space of random functions that depend only on the information set .|
Property 1. Double application of conditional expectation gives the same result as single application:
( is already a function of , so conditioning it on doesn't change it).
Property 2. A successive conditioning on two different information sets is the same as conditioning on the smaller one:
Property 3. Conditional expectation is a linear operator: for any variables and numbers
It's easy to see that (4)-(6) are similar to (1)-(3), respectively, but I prefer to use different names for (4)-(6). I call (4) a projector property. (5) is known as the Law of Iterated Expectations, see my post on the informational aspect for more intuition. (6) holds simply because at time the expectation is known and behaves like a constant.
Summary. (4)-(6) are easy to remember as one property. The smaller information set wins: