Geometry of linear equations: orthogonal complement and equation solvability
From orthogonality of vectors to orthogonality of subspaces
Exercise 1. Let be the axis on the plane. Find
Solution. The condition for all implies The component is free. Thus, is the axis.
Similarly, the orthogonal complement of the axis is the axis. This fact is generalized as follows:
Theorem (on second orthocomplement) For any linear subspace one has .
This statement is geometrically simple but the proof is rather complex and will be omitted (see Akhiezer & Glazman, Theory of Linear Operators in Hilbert Space, Dover Publications, 1993, Chapter 1, sections 6 and 7). It's good to keep in mind what makes it possible. For any set its orthogonal complement can be defined in the same way: As in Exercise 1, you can show that if contains just the unit vector of the axis, then is the axis and therefore is the axis. Thus, in this example we have a strict inclusion The equality is achieved only for linear subspaces.
Exercise 2. In consider the axis Can you tell what is?
Exercise 3. and
Link between image of a matrix and null space of its transpose
Exercise 4. The rule for a transpose of a product implies that for any
Exercise 5 (second characterization of matrix image) (and, consequently, ).
Proof. Let us prove that
To this end, we first prove the inclusion Let . For an arbitrary we have Hence by (1) Since is arbitrary, we can put and obtain This implies and
Conversely, to prove suppose Then and for an arbitrary we have Since runs over this means that
Passing to orthogonal complements in (2), by the theorem on second orthocomplement we get what we need.
Definition 2. Let be two subspaces. We say that is their orthogonal sum if 1) every element can be decomposed as with and 2) every element of is orthogonal to every element of Orthogonality of to implies which, in turn, guarantees uniqueness of the representation
Exercise 6. Assume that is of size . Exercise 5 and Exercise 3 imply and .
The importance of Exercise 5 is explained by the fact that the null space of a matrix is easier to describe analytically than the image. For the next summary, you might want to review the conclusion on the role of the null space.
Summary. 1) In order to see if has solutions, check if is orthogonal to In particular, if is one-to-one, then and has solutions for all (see Exercise 3 above).
2) If is one-to-one, then may have only unique solutions.
3) If both and are one-to-one, then has a unique solution for any