## Application: estimating sigma squared

Consider multiple regression

(1)

where

(a) the regressors are assumed deterministic, (b) the number of regressors is smaller than the number of observations (c) the regressors are linearly independent, and (d) the errors are homoscedastic and uncorrelated,

(2)

Usually students remember that should be estimated and don't pay attention to estimation of Partly this is because does not appear in the regression and partly because the result on estimation of error variance is more complex than the result on the OLS estimator of

**Definition 1**. Let be the OLS estimator of . is called the **fitted value** and is called the **residual**.

**Exercise 1**. Using the projectors and show that and

**Proof**. The first equation is obvious. From the model we have Since we have further

**Definition 2**. The **OLS estimator of** is defined by

**Exercise 2**. Prove that is unbiased:

**Proof**. Using projector properties we have

Expectations of type and would be easy to find from (2). However, we need to find where there is an obstructing See how this difficulty is overcome in the next calculation.

( is a scalar, so its trace is equal to itself)

(applying trace-commuting)

(the regressors and hence are deterministic, so we can use linearity of )

(applying (2))

because this is the dimension of the image of Therefore Thus, and