Feb 22

Estimation of parameters of a normal distribution

Estimation of parameters of a normal distribution

Here we show that the knowledge of the distribution of s^{2} for linear regression allows one to do without long calculations contained in the guide ST 2134 by J. Abdey.

Theorem. Let y_{1},...,y_{n} be independent observations from N\left( \mu,\sigma ^{2}\right) . 1) s^{2}\left( n-1\right) /\sigma ^{2} is distributed as \chi _{n-1}^{2}. 2) The estimators \bar{y} and s^{2} are independent. 3) Es^{2}=\sigma ^{2}, 4) Var\left( s^{2}\right) =\frac{2\sigma ^{4}}{n-1}, 5) \frac{s^{2}-\sigma ^{2}}{\sqrt{2\sigma ^{4}/\left(n-1\right) }} converges in distribution to N\left( 0,1\right) .

Proof. We can write y_{i}=\mu +e_{i} where e_{i} is distributed as N\left( 0,\sigma ^{2}\right) . Putting \beta =\mu ,\ y=\left(y_{1},...,y_{n}\right) ^{T}, e=\left( e_{1},...,e_{n}\right) ^{T} and X=\left( 1,...,1\right) ^{T} (a vector of ones) we satisfy (1) and (2). Since X^{T}X=n, we have \hat{\beta}=\bar{y}. Further,

r\equiv y-X\hat{  \beta}=\left( y_{1}-\bar{y},...,y_{n}-\bar{y}\right) ^{T}


s^{2}=\left\Vert r\right\Vert ^{2}/\left( n-1\right) =\sum_{i=1}^{n}\left(  y_{i}-\bar{y}\right) ^{2}/\left( n-1\right) .

Thus 1) and 2) follow from results for linear regression.

3) For a normal variable X its moment generating function is M_{X}\left( t\right) =\exp \left(\mu t+\frac{1}{2}\sigma ^{2}t^{2}\right) (see Guide ST2133, 2021, p.88). For the standard normal we get

M_{z}^{\prime }\left( t\right) =\exp \left(  \frac{1}{2}t^{2}\right) t, M_{z}^{\prime \prime }\left( t\right) =\exp \left( \frac{1}{2}t^{2}\right) (t^{2}+1),

M_{z}^{\prime \prime \prime}\left( t\right) =\exp \left( \frac{1}{2}t^{2}\right) (t^{3}+2t+t), M_{z}^{(4)}\left( t\right) =\exp \left( \frac{1}{2}t^{2}\right)  (t^{4}+6t^{2}+3).

Applying the general property EX^{r}=M_{X}^{\left(  r\right) }\left( 0\right) (same guide, p.84) we see that

Ez=0, Ez^{2}=1, Ez^{3}=0, Ez^{4}=3,

Var(z)=1, Var\left( z^{2}\right) =Ez^{4}-\left( Ez^{2}\right)  ^{2}=3-1=2.


Es^{2}=\frac{\sigma ^{2}}{n-1}E\left( z_{1}^{2}+...+z_{n-1}^{2}\right) =\frac{\sigma ^{2}}{n-1}\left( n-1\right) =\sigma ^{2}.

4) By independence of standard normals

Var\left( s^{2}\right) = \left(\frac{\sigma ^{2}}{n-1}\right) ^{2}\left[ Var\left( z_{1}^{2}\right)  +...+Var\left( z_{n-1}^{2}\right) \right] =\frac{\sigma ^{4}}{\left(  n-1\right) ^{2}}2\left( n-1\right) =\frac{2\sigma ^{4}}{n-1}.

5) By standardizing s^{2} we have \frac{s^{2}-Es^{2}}{\sigma \left(s^{2}\right) }=\frac{s^{2}-\sigma ^{2}}{\sqrt{2\sigma ^{4}/\left( n-1\right) }} and this converges in distribution to N\left( 0,1\right) by the central limit theorem.


Leave a Reply

You must be logged in to post a comment.