Raising the bar

General properties of symmetric matrices

Here we consider properties of symmetric matrices that will be used to prove their diagonalizability.

What are the differences between $latex R^n$ and $latex C^n?$

Vectors in both spaces have $latex n$ coordinates. In $latex R^n$ we can multiply vectors by real numbers and in $latex C^n$ - by complex numbers. This affects the notions of linear combinations, linear independence, dimension and scalar product. We indicate only the differences to watch for.

If in $latex C^n$ we multiply vectors only by real numbers, it becomes a space of dimension $latex 2n.$ Let's take $latex n=1$ to see why.

Example 1. If we take $latex e=1,$ then any complex number $latex c=a+ib$ is a multiple of $latex e:$ $latex c=c1=ce$ with the scaling coefficient $latex c.$ Thus, $latex C$ is a one-dimensional space in this sense. On the other hand, if only multiplication by real numbers is allowed, then we can take $latex e_1=1,$ $latex e_2=i$ as a basis and then $latex c=ae_1+be_2$ and $latex C$ is two-dimensional. To avoid confusion, just use scaling by the right numbers.

The scalar product in $latex R^n$ is given by $latex x\cdot y=\sum x_iy_i$ and in $latex C^n$ by $latex x\cdot y=\sum x_i\bar{y}_i.$ As a result, for the second scalar product we have $latex x\cdot (c_1y+c_2z)=\bar{c}_1x\cdot y+\bar{c}_2x\cdot z$ for complex $latex c_1,c_2$ (some people call this antilinearity, to distinguish it from linearity $latex x\cdot (ay+bz)=ax\cdot y+bx\cdot z$ for real $latex a,b$).

Definition 1. For a matrix $latex A$ with possibly complex entries we denote $latex A'=\overline{A^T}.$ The matrix $latex A'$ is called an adjoint or a conjugate of $latex A.$

Exercise 1. Prove that $latex (Ax)\cdot y=x\cdot(A'y),$ for any $latex x,y\in C^n.$

Proof. For complex numbers we have $latex \overline{\bar{c}}=c,$ $latex \overline{c_1c_2}=\bar{c}_1\bar{c}_2.$ Therefore

$latex (Ax)\cdot y=(Ax)^T\bar{y}=x^TA^T\bar{y}=x^T\overline{\overline{(A^T)}}\bar{y}=x^T\overline{(\overline{A^T}y)}=x\cdot(A'y).$

Thus, when considering matrices in $latex C^n,$ conjugation should be used instead of transposition. In particular, instead of symmetry $latex A=A^T$ the equation $latex A=A'$ should be used. Matrices satisfying the last equation are called self-adjoint. The theory of self-adjoint matrices in $latex C^n$ is very similar to that of symmetric matrices in $latex R^n.$ Keeping in mind two applications (regression analysis and optimization), we consider only square matrices with real entries. Even in this case one is forced to work with $latex C^n$ from time to time because, in general, eigenvalues can be complex numbers.

General properties of symmetric matrices

$latex A$ is assumed a square matrix with real entries. When we extend $latex A$ from $latex R^n$ to $latex C^n,$ $latex Ax$ is defined by the same expression as before but $latex x$ is allowed to be from $latex C^n$ and the scalar product in $latex R^n$ is replaced by the scalar product in $latex C^n.$ The extension is denoted $latex A_C.$

Exercise 2. If $latex A$ is symmetric, then all eigenvalues of $latex A_C$ are real.

Proof. Suppose $latex \lambda$ is an eigenvalue of $latex A_C.$ Using Exercise 1 and the symmetry of $latex A$ we have

$latex \lambda x\cdot x=(Ax)\cdot x=x\cdot(Ax)=x\cdot(\lambda x)=\bar{\lambda}x\cdot x.$

Since $latex x\cdot x=\|x\|^2>0,$ we have $latex \lambda =\bar{\lambda}.$ This shows that $latex \lambda$ is real.

Exercise 3. If $latex A$ is symmetric, then it has at least one real eigenvector.

Proof. We know that $latex A_C$ has at least one complex eigenvalue $latex \lambda$. By Exercise 2, this eigenvalue must be real. Thus, we have $latex Ax=\lambda x$ with some nonzero $latex x\in C^n.$ Separating real and imaginary parts of $latex x,$ we have $latex x=u+iv,$ $latex Au=\lambda u,$ $latex Av=\lambda v$ with some $latex u,v\in R^n.$ At least one of $latex u,v$ is not zero. Thus a real eigenvector exists.

We need to generalize Exercise 3 to the case when $latex A$ acts in a subspace. This is done in the next two exercises.

Definition 2. A subspace $latex L$ is called an invariant subspace of $latex A$ if $latex AL\subseteq L.$

Example 2. If $latex x$ is an eigenvector of $latex A,$ then the subspace $latex L$ spanned by $latex x$ is an invariant subspace of $latex A.$ This is because $latex x\in L$ implies $latex Ax=\lambda x\in L.$

Exercise 4. If $latex A$ is symmetric and $latex L$ is a non-trivial invariant subspace of $latex A$, then $latex A$ has an eigenvector in $latex L.$

Proof. By the definition of an invariant subspace, the restriction of $latex A$ to $latex L$ defined by $latex A_Lx=Ax,$ $latex x\in L,$ acts from $latex L$ to $latex L$. By Exercise 3, applied to $latex A_L,$ it has an eigenvector in $latex L$, which is also an eigenvector of $latex A.$

Exercise 5. a) If $latex \lambda $ is an (real) eigenvalue of $latex A_R,$ then it is an eigenvalue of $latex A_C.$ b) If $latex \lambda$ is a real eigenvalue of $latex A_C,$ then it is an eigenvalue of $latex A_R.$ This is summarized as $latex \sigma(A_R)=\sigma (A_C)\cap R,$ see the spectrum notation.

See if you can prove this yourself following the ideas used above.

← Previous: Eigenvalues and eigenvectors | Next: Diagonalization of symmetric matrices →

Get the PDF book “Matrix Algebra” on Lulu