Nov 18

Constructing a projector onto a given subspace

Constructing a projector onto a given subspace

Let L be a subspace of R^n. Let k=\dim L\ (\leq n) and fix some basis x^{(1)},...,x^{(k)} in L. Define the matrix X=(x^{(1)},...,x^{(k)}) of size n\times k (the vectors are written as column vectors).

Exercise 1. a) With the above notation, the matrix (X^TX)^{-1} exists. b) The matrix P=X(X^TX)^{-1}X^T exists. c) P is a projector.

Proof. a) The determinant of A=X^TX is not zero by linear independence of the basis vectors, so its inverse A^{-1} exists. We also know that A and its inverse are symmetric:

(1) A^T=A, (A^{-1})^T=A^{-1}.

b) To see that P exists just count the dimensions.

c) Let's prove that P is a projector. (1) allows us to make the proof compact. P is idempotent:



P is symmetric:


Exercise 2. P projects onto L\text{Img}(P)=L.

Proof. First we show that \text{Img}(P)\subseteq L. Put

(2) y=A^{-1}X^Tx,

for any x\in R^n. Then

(3) Px=XA^{-1}X^Tx=Xy=\sum x^{(j)}y_j\in L.

This shows that \text{Img}(P)\subseteq L.

Let's prove the opposite inclusion. Any element of L is of form

(4) x=Xz

with some z. Plugging (4) in (3) we have Px=XA^{-1}X^TXz=Xz which shows that \text{Img}(P)\supseteq L.

Exercise 3. Let L_1 be a subspace of R^n and let L_2=(L_1)^\perp. Then R^n=L_1\oplus L_2.

Proof. Since any element of L_1 is orthogonal to any element of L_2, we have only to show that any x\in R^n can be represented as x=l_1+l_2 with l_j\in L_j. Let P be the projector from Exercise 1, where L=L_1. Put l_1=Px, l_2=(I-P)x. l_1\in L_1=\text{Img}(P) is obvious. For any y\in R^n, (Py)\cdot l_2=y\cdot (P(I-P)x)=0, so l_2 is orthogonal to L_1. By definition of L_2, we have l_2\in L_2.

Leave a Reply

You must be logged in to post a comment.