14
Nov 18

## Constructing a projector onto a given subspace

Let $L$ be a subspace of $R^n.$ Let $k=\dim L\ (\leq n)$ and fix some basis $x^{(1)},...,x^{(k)}$ in $L.$ Define the matrix $X=(x^{(1)},...,x^{(k)})$ of size $n\times k$ (the vectors are written as column vectors).

Exercise 1. a) With the above notation, the matrix $(X^TX)^{-1}$ exists. b) The matrix $P=X(X^TX)^{-1}X^T$ exists. c) $P$ is a projector.

Proof. a) The determinant of $A=X^TX$ is not zero by linear independence of the basis vectors, so its inverse $A^{-1}$ exists. We also know that $A$ and its inverse are symmetric:

(1) $A^T=A,$ $(A^{-1})^T=A^{-1}.$

b) To see that $P$ exists just count the dimensions.

c) Let's prove that $P$ is a projector. (1) allows us to make the proof compact. $P$ is idempotent:

$P^2=(XA^{-1}X^T)(XA^{-1}X^T)=XA^{-1}(X^TX)A^{-1}X^T$ $=X(A^{-1}A)A^{-1}X^T=XA^{-1}X^T=P.$

$P$ is symmetric:

$P^T=[XA^{-1}X^T]^T=(X^T)^T(A^{-1})^TX^T=XA^{-1}X^T=P.$

Exercise 2. $P$ projects onto $L:$ $\text{Img}(P)=L.$

Proof. First we show that $\text{Img}(P)\subseteq L.$ Put

(2) $y=A^{-1}X^Tx,$

for any $x\in R^n.$ Then

$Px=XA^{-1}X^Tx=Xy=\sum x^{(j)}y_j\in L.$

This shows that $\text{Img}(P)\subseteq L.$

Let's prove the opposite inclusion. Any element of $L$ is of form $\sum x^{(j)}y_j$ with some $y.$ $N(X)=\{0\}$ because we are dealing with a basis. This fact and the general equation $N(X)\oplus \text{Img}(X^T)=R^n$ imply $\text{Img}(X^T)=R^n.$ Hence for any given $y$ there exists $x$ such that $Ay=X^Tx.$ Then (2) is true and, as above, $Px=\sum x^{(j)}y_j.$ We have proved $\text{Img}(P)\supseteq L.$