Using these estimators, we define the fitted value which mimics the linear part. To mimic the errors, we define residuals . These definitions give a sample analog of (2):

(2')

The residuals also possess the property

(3')

which is a sample analog of (3).

Comparison

Before estimation

After estimation

are unknown, the errors are unobservable

The estimators, fitted values and residuals are known functions of observed values (1)

(2) is just a product of our imagination

Its analog (2') holds by construction

Whether (3') is true or not we don't know

Its analog (3') is always true

Tricky question. Put . Ask your students to show that if are deterministic, then under condition (3) along with (3') one has

(3'')

This will reveal if they know the difference between sample means and population means.

[…] The first answer, , tells us that if we know the individual means, we can avoid calculating by simply adding two numbers. Similarly, the second formula, , simplifies calculation of . Methodologically, this is an excellent opportunity to dive into theory. Firstly, there is good motivation. Secondly, it's easy to see the link between numbers and algebra (see tabular representations of random variables in Chapters 4 and 5 of my book (you are welcome to download the free version). Thirdly, even though this is theory, many things here are done by analogy, which students love. Fourthly, this topic paves the road to properties of the variance and covariance (recall that the slope in simple regression is covariance over variance). […]

[…] is the fitted value and is the residual, see this post. We still want to see how is far from With this purpose, from both sides of equation (2) we […]

[…] running regression, report the estimated equation. It is called a fitted line and in our case looks like this: Earnings = -13.93+2.45*S (use descriptive names and not abstract […]

[…] The first answer, , tells us that if we know the individual means, we can avoid calculating by simply adding two numbers. Similarly, the second formula, , simplifies calculation of . Methodologically, this is an excellent opportunity to dive into theory. Firstly, there is good motivation. Secondly, it's easy to see the link between numbers and algebra (see tabular representations of random variables in Chapters 4 and 5 of my book (you are welcome to download the free version). Thirdly, even though this is theory, many things here are done by analogy, which students love. Fourthly, this topic paves the road to properties of the variance and covariance (recall that the slope in simple regression is covariance over variance). […]

[…] say: the formal treatment of the true model, error term and their implications for inference is beyond the scope of this […]

[…] is the fitted value and is the residual, see this post. We still want to see how is far from With this purpose, from both sides of equation (2) we […]

[…] new in the area I thought I knew everything about. So here is the derivation. By definition, the fitted value […]

[…] running regression, report the estimated equation. It is called a fitted line and in our case looks like this: Earnings = -13.93+2.45*S (use descriptive names and not abstract […]