Distribution of the estimator of the error variance
If you are reading the book by Dougherty: this post is about the distribution of the estimator 
Consider regression
(1)
where the deterministic matrix 
(2)
(3)
(4) 
If 
tells us that the number of eigenvalues equal to 1 is 
(5)
and 
Theorem. Let 
Proof. 1) We have by (4)
(6)
Denote 
and 
(7)
(3) and (7) prove the first statement.
2) First we note that the vectors 
It's easy to see that 
Independence of
have densities 
Then they are independent if and only if their joint density 
is a product of individual densities: 
are normal, then they are independent if and only if they are uncorrelated: 
be independent standard normal variables. A standard normal variable is 
have the same density. We achieve independence, according to Theorem 1, by defining their joint density to be a product of individual densities.
is defined by
because all of 
for 
by Theorem 2 and 
and vector 
of compatible dimensions a normal vector is defined by 
and
converges to 1 in probability.
we have 
and 
(both properties can be verified in Mathematica). Hence, 
and
(c) the regressors are 
should be estimated and don't pay attention to estimation of 
Partly this is because 
does not appear in the regression and partly because the result on estimation of error variance is more complex than the result on the OLS estimator of 
be the 
is called the fitted value and 
is called the residual.
and 
show that 
and 
Since 
we have further 
and 
would be easy to find from (2). However, we need to find 
where there is an obstructing 
(
is a scalar, so its trace is equal to itself)
(applying 
(the regressors and hence 
)
(applying (2)) 
because this is the 
Therefore 
Thus, 
and 
![(Px)\cdot (Qx)=(Px)\cdot[(I-P)x]=x\cdot[(P-P^2)x]=0.](https://s0.wp.com/latex.php?latex=%28Px%29%5Ccdot+%28Qx%29%3D%28Px%29%5Ccdot%5B%28I-P%29x%5D%3Dx%5Ccdot%5B%28P-P%5E2%29x%5D%3D0.&bg=ffffff&fg=000&s=0&c=20201002)
(the dependent vector) and vectors 
(independent vectors or regressors) are given. The OLS estimator is defined as that vector 
which minimizes the total sum of squares 
we see that 
and that finding the OLS estimator means approximating 
with vectors from the image 
should be linearly independent, otherwise the solution will not be unique.
Since 
belongs to the image of 
Denoting also 
(by Exercise 1)
is a lower bound for 
This lower bound is achieved when the second term is made zero. From![P(y-X\beta)=Py-X\beta =X(X^TX)^{-1}X^Ty-X\beta=X[(X^TX)^{-1}X^Ty-\beta]](https://s0.wp.com/latex.php?latex=P%28y-X%5Cbeta%29%3DPy-X%5Cbeta+%3DX%28X%5ETX%29%5E%7B-1%7DX%5ETy-X%5Cbeta%3DX%5B%28X%5ETX%29%5E%7B-1%7DX%5ETy-%5Cbeta%5D&bg=ffffff&fg=000&s=0&c=20201002)
where 
components by
be random variables and let 
be numbers. In the 
?
Again, a matrix visualization will help:
(the sum does not depend on the index of summation, this is another point many students don't understand) and then perform the multiplication in (3).
we have 
which are diagonal elements of (1). Otherwise, for 
we get off-diagonal elements of (1). So the answer is
are assumed to run from 
to 
are uncorrelated, then the second sum in (5) disappears:
This sum can also be written as 
Hence, (5) is the same as
for all 
.
for all 
are the same but it's better to use the verbal definition.
on 
for all 
. In case of 
which may be nonlinear in 
. We still minimize RSS which takes the form 
. Nonlinear least squares estimators are the values 
. The Newton algorithm (default in Stata) is an iterative procedure that consists of steps:
.
.
will approximate the minimizing point.
You must be logged in to post a comment.