What is an OLS estimator?
Most sources give too much theory and a long procedure of OLS (Ordinary Least Squares) derivation. In fact, there is a simplified derivation which can be used in other situations. For example, in case of an Instrumental Variables estimator it is the ONLY derivation. However, simplicity comes at a cost. You need to know the properties of variances and covariances.
Tip. By all means avoid using summation signs because: 1) not everybody is good at them and 2) they clog the picture. For this reason alone it is worth using variances and covariances. For example, if are observations, let us call by variance the quantity where is the sample mean. If, in addition, we have observations on another variable, then we can introduce covariance by One can notice that
Try to rewrite this using summation signs, and you will see the gain from using shorter notation. The next property is less easy to establish. It is called linearity of covariance: if is a third observed variable and are any numbers, then
Now consider simple regression
where is the error term and are parameters to be estimated. The well-known formula for the OLS estimator of is in our notation
The question is: What is an easy way to obtain this?
Answer: Notice that there is in the numerator and try to expand it using (3) and (2):
The first term on the right is zero (this is shown directly), for the second term we apply (1). In case of the third term, we formally put (remember, this is a simplified derivation and we are cheating). The result is It remains to solve this for , putting a hat on it, to obtain (4). The whole derivation takes just one paragraph!