Aug 18

Matrix Algebra

Section 1. Matrix operations

Matrix notation and summation, columns and row vectors, axioms of summation

Vector and matrix multiplication, scalar product and its symmetry, partitioning and compatibility rule

Roadmap for studying matrix multiplication, axioms of multiplication, commutativity, associativity and distributivity

Matrix inversion: doing some housekeeping at elementary level, identity matrix, invertibility, inverse of an inverse and inverse of a product

From invertibility to determinants: argument is more important than result, determinant, multiplicativity, Invertibility condition, determinant of an inverse, right and left inverses

Matrix transposition: continuing learning by doing, symmetric matrix, determinant of a transpose, transpose of a product, transpose of an inverse

Matrix algebra: questions for repetition

Section 2. Vector operations, scalar product and related

Euclidean space geometry: vector operations, parallelogram rule, unit vector, scaling, linear combination

Euclidean space geometry: scalar product, norm and distance, homogeneity, additivity, orthogonal vectors, arithmetic square root, Pythagoras theorem

Euclidean space geometry: Cauchy-Schwarz inequality, triangle inequality, cosine of the angle between vectors, distance, ball, open set

Euclidean space geometry: questions for repetition, scalar product, norm and distance properties; parallelogram law

Section 3. Geometry of linear equations

Geometry of linear equations: matrix as a mapping, image, counter-image, first characterization of matrix image, multiplication rule for partitioned matrices

Geometry of linear equations: linear spaces and subspaces, subspace spanned by vectors

Geometry of linear equations: structure of image and null space, homogeneous and inhomogeneous equations, hyperplane

Geometry of linear equations: orthogonal complement and equation solvability, second orthocomplement, second characterization of matrix image

Geometry of linear equations: questions for repetition

Section 4. Equation solvability and linear (in)dependence

Is the inverse of a linear mapping linear? Orthonormal system and basis, matrix generated by linear mapping

Solvability of an equation with a square matrix: link between invertibility, null space, image and determinant

Linear dependence of vectors: definition and principal result, trivial cases, case of two vectors, criterion of linear independence

Basis and dimension, intersection of orthogonal subspaces, dimension additivity

Rank of a matrix and the rank-nullity theorem, rank of a matrix and its transpose, upper bound for rank

Final touches on linear independence One more definition, correctness of the dimension definition, linear dependence of a large number of vectors

Summary and questions for repetition

Section 5. Spectral theory and diagonalization by orthogonal matrices

Complex numbers: time to turn on the beacon Imaginary unit, conjugate, absolute value, polar form

Law and order in the set of matrices Motivation, definition and geometry of order

Matrix similarity  Changing bases, motivation for matrix similarity

Orthogonal matrices Algebra, geometry and link to the transtition matrix

Eigenvalues and eigenvectors Motivation, link to characteristic equation, fundamental theorem of algebra

General properties of symmetric matrices The role of C^n, self-adjoint matrices, eigenvalues of symmetric matrices, existence of one eigenvector

Diagonalization of symmetric matrices Main result: symmetry is necessary and sufficient for diagonalizability

Questions for repetition

Section 6. Applications of the diagonalization theorem

Applications of the diagonal representation I Matrix positivity, matrix functions and linear differential equation

Applications of the diagonal representation II Square root of a matrix and Generalized Least Squares estimator

Applications of the diagonal representation III Absolute value and polar form for matrices

Applications of the diagonal representation IV Principal component analysis

Questions for repetition Representing a complex matrix as a linear combination of symmetric matrices and an exercise on elementary matrices.

Section 7. Projectors and their applications

Geometry and algebra of projectors Includes the characterization of the image and null space of a projector

Constructing a projector onto a given subspace The definition requires a basis of the subspace.

Application: Ordinary Least Squares estimator Includes a version of the Pythagoras theorem

Eigenvalues and eigenvectors of a projector Includes trace of a projector

Application: estimating sigma squared This is about estimating the error variance in multiple regression

Section 8. Determinants

Determinants: starting simple

Axioms 1-3 and Properties I-III

Properties IV-VI

Permutation matrices

Properties of permutation matrices

Leibniz formula for determinants

Determinant of a product

Multilinearity in columns

Determinant of a transpose

Cramer's rule and invertibility criterion

Laplace expansion

Determinants: questions for repetition