Raising the bar

Eigenvalues and eigenvectors of a projector

Exercise 1. Find eigenvalues and eigenvectors of a projector.

Solution. We know that a projector doesn't change elements from its image: $latex Px=x$ for all $latex x\in\text{Img}(P).$ This means that $latex \lambda =1$ is an eigenvalue of $latex P.$ Moreover, if $latex {x_i:i=1,...,\dim\text{Img}(P)}$ is any orthonormal system in $latex \text{Img}(P),$ each of $latex x_i$ is an eigenvector of $latex P$ corresponding to the eigenvalue $latex \lambda =1.$

Since $latex P$ maps to zero all elements from the null space $latex N(P),$ $latex \lambda =0$ is another eigenvalue. If $latex {y_i:i=1,...,\dim N(P)}$ is any orthonormal system in $latex N(P),$ each of $latex y_i$ is an eigenvector of $latex P$ corresponding to the eigenvalue $latex \lambda =0.$

A projector cannot have eigenvalues other than $latex 0$ and $latex 1.$ This is proved as follows. Suppose $latex Px=\lambda x$ with some nonzero $latex x.$ Applying $latex P$ to both sides of this equation, we get $latex Px=P^2x=\lambda Px=\lambda ^2x.$ It follows that $latex \lambda x=\lambda^2x$ and (because $latex x\neq 0$) $latex \lambda =\lambda^2.$ The last equation has only two roots: $latex 0$ and $latex 1.$

We have $latex \dim\text{Img}(P)+\dim N(P)=n$ because $latex R^n$ is an orthogonal sum of $latex N(P)$ and $latex \text{Img}(P)$.  Combining the systems $latex {x_i},$ $latex {y_i}$ we get an orthonormal basis in $latex R^{n}$ consisting of eigenvectors of $latex P$.

Trace of a projector

Recall that for a square matrix, its trace is defined as the sum of its diagonal elements.

Exercise 2. Prove that $latex tr(AB)=tr(BA)$ if both products $latex AB$ and $latex BA$ are square. It is convenient to call this property trace-commuting (we know that in general matrices do not commute).

Proof. Assume that $latex A$ is of size $latex n\times m$ and $latex B$ is of size $latex m\times n.$ For both products we need only to find the diagonal elements:

$latex AB=\left(\begin{array}{ccc}
a_{11}&...&a_{1m} \\...&...&... \\a_{n1}&...&a_{nm}\end{array}
\right)\left(\begin{array}{ccc}
b_{11}&...&b_{1n} \\...&...&... \\b_{m1}&...&b_{mn}\end{array}
\right)=\left(\begin{array}{ccc}
\sum_ia_{1i}b_{i1}&...&... \\...&...&... \\...&...&\sum_ia_{ni}b_{in}\end{array}
\right)$

$latex BA=\left(\begin{array}{ccc}
b_{11}&...&b_{1n} \\...&...&... \\b_{m1}&...&b_{mn}\end{array}
\right)\left(\begin{array}{ccc}
a_{11}&...&a_{1m} \\...&...&... \\a_{n1}&...&a_{nm}\end{array}
\right)=\left(\begin{array}{ccc}
\sum_ja_{j1}b_{1j}&...&... \\...&...&... \\...&...&\sum_ja_{jm}b_{mj}
\end{array}\right)$

All we have to do is change the order of summation:

$latex tr(AB)=\sum_j\sum_ia_{ji}b_{ij}=\sum_i\sum_ja_{ji}b_{ij}=tr(BA).$

Exercise 3. Find the trace of a projector.

Solution. In Exercise 1 we established that the projector $latex P$ has $latex p=\dim\text{Img}(P)$ eigenvalues $latex \lambda =1$ and $latex n-p$ eigenvalues $latex \lambda =0.$ $latex P$ is symmetric, so in its diagonal representation $latex P=UDU^{-1}$ there are $latex p$ unities and $latex n-p$ zeros on the diagonal of the diagonal matrix $latex D.$ By Exercise 2

$latex tr(P)=tr(UDU^{-1})=tr(DU^{-1}U)=tr(D)=p$.

← Previous: Application: Ordinary Least Squares estimator | Next: Application: estimating sigma squared →

Get the PDF book “Matrix Algebra” on Lulu