Aug 18

Matrix Algebra

This is a minicourse in linear algebra. We follow a geometric approach, which allows one to produce the shortest proofs possible. The theory is presented as a series of exercises, and each exercise typically contains no more than one idea. Each section has a summary with questions for repetition.

Clarity of ideas is put above strictly logical sequencing. Omission of boring details is indicated explicitly. Complex results are given where the reader is thought to be prepared for them. One notable example is determinants, which usually are given in the beginning of a linear algebra course but here are placed closer to the end. In short, this is the course I would prefer when I was a bachelor student.

The course completely covers the theory required for MT2175 "Further Linear Algebra"(and at a higher level than Anthony and Harvey, Linear algebra: Concepts and methods).

Section 1. Matrix operations

Matrix notation and summation, columns and row vectors, axioms of summation

Vector and matrix multiplication, scalar product and its symmetry, partitioning and compatibility rule

Roadmap for studying matrix multiplication, axioms of multiplication, commutativity, associativity and distributivity

Matrix inversion: doing some housekeeping at elementary level, identity matrix, invertibility, inverse of an inverse and inverse of a product

From invertibility to determinants: argument is more important than result, determinant, multiplicativity, Invertibility condition, determinant of an inverse, right and left inverses

Matrix transposition: continuing learning by doing, symmetric matrix, determinant of a transpose, transpose of a product, transpose of an inverse

Matrix algebra: questions for repetition

Section 2. Vector operations, scalar product and related

Euclidean space geometry: vector operations, parallelogram rule, unit vector, scaling, linear combination

Euclidean space geometry: scalar product, norm and distance, homogeneity, additivity, orthogonal vectors, arithmetic square root, Pythagoras theorem

Euclidean space geometry: Cauchy-Schwarz inequality, triangle inequality, cosine of the angle between vectors, distance, ball, open set

Euclidean space geometry: questions for repetition, scalar product, norm and distance properties; parallelogram law

Section 3. Geometry of linear equations

Geometry of linear equations: matrix as a mapping, image, counter-image, first characterization of matrix image, multiplication rule for partitioned matrices

Geometry of linear equations: linear spaces and subspaces, subspace spanned by vectors

Geometry of linear equations: structure of image and null space, homogeneous and inhomogeneous equations, hyperplane

Geometry of linear equations: orthogonal complement and equation solvability, second orthocomplement, second characterization of matrix image

Geometry of linear equations: questions for repetition

Section 4. Equation solvability and linear (in)dependence

Is the inverse of a linear mapping linear? Orthonormal system and basis, matrix generated by linear mapping

Solvability of an equation with a square matrix: link between invertibility, null space, image and determinant

Linear dependence of vectors: definition and principal result, trivial cases, case of two vectors, criterion of linear independence

Basis and dimension, intersection of orthogonal subspaces, dimension additivity

Correctness of the space dimension definition

Rank of a matrix and the rank-nullity theorem, rank of a matrix and its transpose, upper bound for rank

Final touches on linear independence One more definition, correctness of the dimension definition, linear dependence of a large number of vectors

Summary and questions for repetition

Section 5. Spectral theory and diagonalization by orthogonal matrices

Complex numbers: time to turn on the beacon Imaginary unit, conjugate, absolute value, polar form

Law and order in the set of matrices Motivation, definition and geometry of order

Matrix similarity  Changing bases, motivation for matrix similarity

Orthogonal matrices Algebra, geometry and link to the transition matrix

Eigenvalues and eigenvectors Motivation, link to characteristic equation, fundamental theorem of algebra

General properties of symmetric matrices The role of C^n, self-adjoint matrices, eigenvalues of symmetric matrices, existence of one eigenvector

Diagonalization of symmetric matrices Main result: symmetry is necessary and sufficient for diagonalizability

Questions for repetition: diagonalization by orthogonal matrices

Section 6. Applications of the diagonalization theorem

Applications of the diagonal representation I Matrix positivity, matrix functions and linear differential equation

Applications of the diagonal representation II Square root of a matrix and Generalized Least Squares estimator

Applications of the diagonal representation III Absolute value and polar form for matrices

Applications of the diagonal representation IV Principal component analysis

Questions for repetition Representing a complex matrix as a linear combination of symmetric matrices and an exercise on elementary matrices.

Section 7. Projectors and their applications

Geometry and algebra of projectors Includes the characterization of the image and null space of a projector

Constructing a projector onto a given subspace The definition requires a basis of the subspace.

Application: Ordinary Least Squares estimator Includes a version of the Pythagoras theorem

Eigenvalues and eigenvectors of a projector Includes trace of a projector

Application: estimating sigma squared This is about estimating the error variance in multiple regression

Application: distribution of the estimator of the error variance

Distributions derived from normal variables Joint density of independent variables, standard normal and general normal variables

Application: distribution of sigma squared estimator More on projectors related to OLS

Questions for repetition: projectors and applications

Section 8. Determinants

Determinants: starting simple

Axioms 1-3 and Properties I-III If rows of a matrix are linearly dependent, its determinant is zero

Properties IV-VI Multilinearity and antisymmetry

Permutation matrices Definition and example

Properties of permutation matrices Pre-multiplication by permutation matrix and orthogonality

Leibniz formula for determinants Just an application of multilinearity

Determinant of a product Another illustration of multilinearity

Multilinearity in columns

Determinant of a transpose One more use of multilinearity

Cramer's rule and invertibility criterion Short and elegant proofs

Laplace expansion With special case to show intuition

Determinants: questions for repetition

Section 9. The Jordan form: geometry

Direct sums of subspaces Oblique projector, dimension additivity

Properties of root subspaces Inclusion, stabilization, related space decomposition into a direct sum

Action of a matrix in its root subspace Root vectors, their linear independence, Jordan cell

Chipping off root subspaces The whole space is a direct sum of root subspaces

Playing with bases Basis existence, relative linear independence and relative basis

Main theorem: Jordan normal form

Section 10. Sign-definite matrices

Elementary transformations Links between geometric and matrix definitions

Gaussian elimination method and its theoretical treatment

Sylvester's criterion for positive definite and negative definite matrices