Application: estimating sigma squared
Consider multiple regression
(1)
where
(a) the regressors are assumed deterministic, (b) the number of regressors is smaller than the number of observations
(c) the regressors are linearly independent,
and (d) the errors are homoscedastic and uncorrelated,
(2)
Usually students remember that should be estimated and don't pay attention to estimation of
Partly this is because
does not appear in the regression and partly because the result on estimation of error variance is more complex than the result on the OLS estimator of
Definition 1. Let be the OLS estimator of
.
is called the fitted value and
is called the residual.
Exercise 1. Using the projectors and
show that
and
Proof. The first equation is obvious. From the model we have Since
we have further
Definition 2. The OLS estimator of is defined by
Exercise 2. Prove that is unbiased:
Proof. Using projector properties we have
Expectations of type and
would be easy to find from (2). However, we need to find
where there is an obstructing
See how this difficulty is overcome in the next calculation.
(
is a scalar, so its trace is equal to itself)
(applying trace-commuting)
(the regressors and hence
are deterministic, so we can use linearity of
)
(applying (2))
because this is the dimension of the image of
Therefore
Thus,
and
Leave a Reply
You must be logged in to post a comment.