Distribution of the estimator of the error variance
If you are reading the book by Dougherty: this post is about the distribution of the estimator defined in Chapter 3.
Consider regression
(1)
where the deterministic matrix is of size
satisfies
(regressors are not collinear) and the error
satisfies
(2)
is estimated by
Denote
Using (1) we see that
and the residual
is estimated by
(3)
is a projector and has properties which are derived from those of
(4)
If is an eigenvalue of
then multiplying
by
and using the fact that
we get
Hence eigenvalues of
can be only
or
The equation
tells us that the number of eigenvalues equal to 1 is and the remaining
are zeros. Let
be the diagonal representation of
Here
is an orthogonal matrix,
(5)
and is a diagonal matrix with eigenvalues of
on the main diagonal. We can assume that the first
numbers on the diagonal of
are ones and the others are zeros.
Theorem. Let be normal. 1)
is distributed as
2) The estimators
and
are independent.
Proof. 1) We have by (4)
(6)
Denote From (2) and (5)
and is normal as a linear transformation of a normal vector. It follows that
where
is a standard normal vector with independent standard normal coordinates
Hence, (6) implies
(7)
(3) and (7) prove the first statement.
2) First we note that the vectors are independent. Since they are normal, their independence follows from
It's easy to see that This allows us to show that
is a function of
:
Independence of leads to independence of their functions
and