OLS estimator for multiple regression - as simple as possible
Here I try to explain a couple of ideas to folks not familiar with (or afraid of?) matrix algebra.
A matrix is a rectangular table of numbers. Operations with them most of the time are performed like with numbers. For example, for numbers we know that . For matrices this is also true, except that they often are denoted with capital letters:
. It is easier to describe differences than similarities.
(1) One of the differences is that for matrices we can define a new operation called transposition: the columns of the original matrix are put into rows of a new matrix, which is called a transposed of
. Visualize it like this: if
has more rows than columns, then for the transposed the opposite will be true:
(2) We know that the number
(3) The property
(4) You don't need to worry about how these operations are performed when you are given specific numerical matrices, because they can be easily done in Excel. All you have to do is watch that theoretical requirements are not violated. One of them is that, in general, matrices in a product cannot change places:
Here is an example that continues my previous post about simplified derivation of the OLS estimator. Consider multiple regression
(5)
where
Putting the hat on
Caveat. See the rigorous derivation here. My objective is not rigor but to give you something easy to do and remember.
Leave a Reply
You must be logged in to post a comment.