Raising the bar

Geometry of linear equations: orthogonal complement and equation solvability

From orthogonality of vectors to orthogonality of subspaces

Definition 1. Let $latex L$ be a linear subspace of $latex R^{n}.$ Its orthogonal complement $latex L^{\perp }$ is defined as the set of all vectors orthogonal to all elements of $latex L:$ $latex L^{\perp }=\{ x\in R^n:x\cdot y=0$ $latex \text{ for all}\ y\in L\}.$

Exercise 1. Let $latex L=\{ (x_1,0):x_1\in R\}$ be the $latex x$ axis on the plane. Find $latex L^{\perp}.$

Solution. The condition $latex x\cdot y=x_1y_1=0$ for all $latex x=(x_1,x_2)\in L$ implies $latex y_1=0.$ The component $latex y_2$ is free. Thus, $latex L^{\perp}=\{ (0,y_2):y_2\in R\} $ is the $latex y$ axis.

Similarly, the orthogonal complement $latex (L^{\perp})^{\perp}$ of the $latex y$ axis is the $latex x$ axis. This fact is generalized as follows:

Theorem (on second orthocomplement) For any linear subspace $latex L,$ one has $latex (L^{\perp})^{\perp}=L$.

This statement is geometrically simple but the proof is rather complex and will be omitted (see Akhiezer & Glazman, Theory of Linear Operators in Hilbert Space, Dover Publications, 1993, Chapter 1, sections 6 and 7). It's good to keep in mind what makes it possible. For any set $latex A\subset R^n,$ its orthogonal complement $latex A^{\perp}$ can be defined in the same way: $latex A^{\perp}=\{ x\in R^n:x\cdot y=0$ $latex \text{ for all }y\in A\}.$ As in Exercise 1, you can show that if $latex A=\{(1,0)\}$ contains just the unit vector of the $latex x$ axis, then $latex A^{\perp}$ is the $latex y$ axis and therefore $latex (A^{\perp})^{\perp}$ is the $latex x$ axis. Thus, in this example we have a strict inclusion $latex A\subset (A^{\perp})^{\perp}.$ The equality is achieved only for linear subspaces.

Exercise 2. In $latex R^3,$ consider the $latex x$ axis $latex L=\{(x_1,0,0):x_1\in R\}.$ Can you tell what $latex L^{\perp}$ is?

Exercise 3. $latex \{ 0\}^{\perp}=R^n$ and $latex (R^{n})^{\perp}=\{ 0\} .$

Link between image of a matrix and null space of its transpose

Exercise 4. The rule for a transpose of a product $latex (AB)^T=B^TA^T$ implies that for any $latex x\in R^k,$ $latex y\in R^n$

(1) $latex (Ax)\cdot y=(Ax)^Ty=x^TA^Ty=x\cdot (A^Ty).$

Exercise 5 (second characterization of matrix image) $latex \text{Img}(A)=N(A^T)^{\perp}$ (and, consequently, $latex \text{Img}(A^T)=N(A)^{\perp}$).

Proof. Let us prove that

(2) $latex \text{Img}(A)^{\perp}=N(A^T)$.

To this end, we first prove the inclusion $latex \text{Img}(A)^{\perp}\subset N(A^T).$ Let $latex y\in \text{Img}(A)^{\perp}$. For an arbitrary $latex x,$ we have $latex Ax\in \text{Img}(A).$ Hence by (1) $latex 0=(Ax)\cdot y=x\cdot(A^Ty).$ Since $latex x$ is arbitrary, we can put $latex x=A^Ty$ and obtain $latex \|A^Ty\|^2=0.$ This implies $latex A^Ty=0$ and $latex y\in N(A^T).$

Conversely, to prove $latex N(A^T)\subset\text{Img}(A)^{\perp},$ suppose $latex y\in N(A^T).$ Then $latex A^Ty=0$ and for an arbitrary $latex x$ we have $latex 0=x\cdot(A^Ty)=(Ax)\cdot y.$ Since $latex Ax$ runs over $latex \text{Img}(A),$ this means that $latex y\in\text{Img}(A)^{\perp}.$

Passing to orthogonal complements in (2), by the theorem on second orthocomplement we get what we need.

Definition 2. Let $latex L_{1},\ L_{2}$ be two subspaces. We say that $latex L=L_{1}\oplus L_{2}$ is their orthogonal sum if 1) every element $latex l\in L$ can be decomposed as $latex l=l_{1}+l_{2}$ with $latex l_{i}\in L_{i},$ $latex i=1,2,$ and 2) every element of $latex L_{1}$ is orthogonal to every element of $latex L_{2}.$ Orthogonality of $latex L_{1}$ to $latex L_{2}$ implies $latex L_{1}\cap L_{2}=\{0\}$ which, in turn, guarantees uniqueness of the representation $latex l=l_{1}+l_{2}.$

Exercise 6. Assume that $latex A$ is of size $latex n\times k$. Exercise 5 and Exercise 3 imply $latex R^n=\text{Img}(A)\oplus N(A^T)$ and $latex R^k=\text{Img}(A^T)\oplus N(A)$.

The importance of Exercise 5 is explained by the fact that the null space of a matrix is easier to describe analytically than the image. For the next summary, you might want to review the conclusion on the role of the null space.

Summary. 1) In order to see if $latex Ax=y$ has solutions, check if $latex y$ is orthogonal to $latex N(A^T).$ In particular, if $latex A^T$ is one-to-one, then $latex N(A^T)=\{0\}$ and $latex Ax=y$ has solutions for all $latex y$ (see Exercise 3 above).

2) If $latex A$ is one-to-one, then $latex Ax=y$ may have only unique solutions.

3) If both $latex A$ and $latex A^T$ are one-to-one, then $latex Ax=y$ has a unique solution for any $latex y.$

← Previous: Geometry of linear equations: structure of image and null space | Next: Geometry of linear equations: questions for repetition →

Get the PDF book “Matrix Algebra” on Lulu