Simple regression: a useful comparison of what we have before and after estimation
Before anything
Initially, we have only observations
(1) .
Before estimation
Then we assume dependence between y's and x's of the form
(2)
Here are unknown parameters to be estimated and
are random errors which satisfy the basic assumption
(3)
It is convenient to call a linear part of model (2).
After estimation
The OLS estimators of are, respectively,
Using these estimators, we define the fitted value which mimics the linear part. To mimic the errors, we define residuals
. These definitions give a sample analog of (2):
(2')
The residuals also possess the property
(3')
which is a sample analog of (3).
Comparison
Before estimation | After estimation |
The estimators, fitted values and residuals are known functions of observed values (1) | |
(2) is just a product of our imagination | Its analog (2') holds by construction |
Whether (3') is true or not we don't know | Its analog (3') is always true |
Tricky question. Put . Ask your students to show that if
are deterministic, then under condition (3) along with (3') one has
(3'')
This will reveal if they know the difference between sample means and population means.
Leave a Reply
You must be logged in to post a comment.