Law and order in the set of matrices
The law is to feel by touch every little fact. The order is discussed below.
Why do complex numbers come right before this topic?
The analog of a conjugate number is the transpose How can you tell this? Using Exercise 2 we see that
This looks similar to the identity
(see Exercise 4). Therefore the mapping
is similar to
Once you know this, it's easy to come up with a couple of ideas.
Idea 1. From the characterization of real numbers (see Equation (4)) we see that matrices that satisfy (symmetric matrices) correspond to real numbers and should be better in some way than asymmetric ones.
Idea 2. From Equation (3) we see that the matrix should be symmetric and non-negative.
What is a non-negative matrix?
The set of real numbers is ordered in the sense that for any two real numbers we can say that either
or
is true. The most important property that we used in my class is this: if
and
then
(the sign of an inequality is preserved if the inequality is multiplied by a positive number). Since any two numbers can be compared like that, it is a complete order.
One way in which symmetric matrices are better than more general ones is that for symmetric matrices one can define order. The limitation caused by dimensionality is that this order is not complete (some symmetric matrices are not comparable).
Exercise 1. For the matrix and vector
find the expression
What is the value of this expression at
Solution. and
Definition 1. The function is called a quadratic form of the matrix
Here
is symmetric of size
and
Discussion. 1) The facts that is in the subscript and the argument is
mean that
is fixed and
is changing.
2) While the argument is a vector, the value of this function is a real number: acts from
to
3) does not contain constant or linear terms (of type
and
It contains only quadratic terms (write
to see that the total power is 2), that's why it is called a quadratic form and not a quadratic function.
Definition 2. We say that is positive if
for all nonzero
and non-negative if
for all
(Most sources say positive definite instead of just positive and non-negative definite instead of just non-negative. I prefer a shorter terminology. If you don't understand why in the definition of positivity we require nonzero
go back to Exercise 1). As with numbers, for two symmetric matrices
of the same size, we write
or
if
is positive or non-negative, respectively. Continuing this idea, we can say that
is negative if
is positive.
More on motivation. A legitimate definition of order would obtain if we compared the two matrices element-wise. Definition 2 is motivated by the fact that quadratic forms arise in the multivariate Taylor decomposition.
Sylvester's criterion is the only practical tool for determining positivity or non-negativity. However, in one case this is simple.
Exercise 2. Show that is symmetric and non-negative.
Solution. The symmetry is straightforward and has been shown before. Non-negativity is not difficult either:
Geometry
The graph of a quadratic form in good cases is an elliptic paraboloid and has various other names in worse cases. Geometrically, the definition of the inequality means that the graph of
is everywhere above the graph of
(at the origin they always coincide). In particular,
means that the graph of
is everywhere above the horizontal plane.
Examples. All examples are matrices of size .

Figure 1. Quadratic form of identity (elliptic paraboloid)
1) The identity matrix is positive because
, see Figure 1.

Figure 2. Parabolic cylinder
2) The matrix is non-negative. Its quadratic form
grows when
grows and stays flat when
changes and
is fixed, see Figure 2.

Figure 3. Hyperbolic paraboloid
3) The matrix is not positive or non-negative or negative or non-positive. Its quadratic form
is a parabola with branches looking upward when the second argument is fixed and a parabola with branches looking downward when the first argument is fixed, see Figure 3. When a surface behaves like that around some point, that point is called a saddle point.
Leave a Reply
You must be logged in to post a comment.