Raising the bar

Chapter 10. Simple regression

Unit 10.5. Goodness of fit

Definition 10.7 (Coefficient of determination). The coefficient of determination is defined by $latex R^2 = \frac{ESS}{TSS}$. It is interpreted as the percentage of total variation of $latex y$ around the sample mean $latex \bar y$ explained by the regression.

Comments. (1) The coefficient of determination is used to compare different models explaining the behavior of the same $latex y$. Economic considerations usually provide only general guidance as to what independent variables to include in the model. If you have two competing models for the same dependent variable, choose the one that has a higher $latex R^2$.

(2) For the model without the intercept $latex R^2$ may be negative and loses its meaning as percentage.

Property 1. From Exercise 10.6 it follows that

$latex R^2 = 1 - \frac{RSS}{TSS}$

and that $latex 0 \le R^2 \le 1$, which justifies its interpretation as percentage.

Property 2. The sample correlation $latex r_{y, \hat y}$ between observed $latex y$ and fitted $latex \hat y$ equals $latex \sqrt{R^2}$.

Proof. By definition,

$latex r_{y, \hat y} = \frac{\textrm{cov}_u(y, \hat y)}{\sqrt{V_u(y)} \sqrt{V_u(\hat y)}} = \frac{\overline{(y - \bar y)(\hat y - \overline{\hat y})}}{\sqrt{\overline{(y - \bar y)^2}} \, \sqrt{\overline{(\hat y - \overline{\hat y})^2}}}.$

By Corollary 10.1, $latex \overline{\hat y} = \bar y$, so

$latex r_{y, \hat y} = \frac{\overline{(y - \bar y)(\hat y - \bar y)}}{\sqrt{\overline{(y - \bar y)^2} \, \overline{(\hat y - \bar y)^2}}}.$

Here by the orthogonality relations

$latex \overline{(y - \bar y)(\hat y - \bar y)} = \overline{(\hat y + e - \bar y)(\hat y - \bar y)}$

$latex = \overline{(\hat y - \bar y)^2} + \overline{e(\hat y - \bar y)} = \overline{(\hat y - \bar y)^2}.$

Therefore

$latex r_{y, \hat y} = \frac{\overline{(\hat y - \bar y)^2}}{\sqrt{\overline{(y - \bar y)^2} \, \overline{(\hat y - \bar y)^2}}}$

$latex = \sqrt{\frac{\overline{(\hat y - \bar y)^2}}{\overline{(y - \bar y)^2}}} = \sqrt{R^2}.$

Previous: Orthogonality and its consequences | Next: Questions for repetition (Chapter 10)

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.