Application: distribution of sigma squared estimator
For the formulation of multiple regression and classical conditions on its elements see Application: estimating sigma squared. There we proved unbiasedness of the OLS estimator of Here we do more: we characterize its distribution and obtain unbiasedness as a corollary.
We need a summary of what we know about the residual and the projector where
has unities and zeros on the diagonal of its diagonal representation, where is the number of regressors. With it's the opposite: it has unities and zeros on the diagonal of its diagonal representation. We can always assume that the unities come first, so in the diagonal representation
the matrix is orthogonal and can be written as
where is an identity matrix and the zeros are zero matrices of compatible dimensions.
Characterization of the distribution of
Exercise 1. Suppose the error vector is normal: Prove that the vector is standard normal.
Proof. By the properties of orthogonal matrices
This, together with the equation , proves that is standard normal.
Exercise 2. Prove that is distributed as
Proof. From (1) and (2) we have
Now (3) shows that which is the definition of
Exercise 3. Find the mean and variance of
Solution. From Exercise 2 we obtain the result proved earlier in a different way:
Further, using the variance of a standard normal