Raising the bar

Chapter 10. Simple regression

Unit 10.2. Linear model setup

We assume that vectors $latex (x_1, y_1), \ldots, (x_n, y_n)$ are observed and want to capture dependence of $latex y_i$ on $latex x_i$. The linear model seeks a linear dependence $latex y_i = \beta_1 + \beta_2 x_i$. Here $latex \beta_1, \beta_2$ are some numbers that do not depend on $latex i$, that is on the particular observation $latex (x_i, y_i)$. Since usually statistical data are chaotic, exact linear dependence is impossible and it is more realistic to suppose that $latex y_i$ differs from a linear function $latex \beta_1 + \beta_2 x_i$ by a random error term $latex u_i$.

Basic assumptions. (1) Thus, the main assumption is

$latex y_i = \beta_1 + \beta_2 x_i + u_i, \quad i = 1, \ldots, n. \qquad (\textrm{Eq. 10.1})$

(2) The independent variable $latex x$ is called an explanatory variable or a regressor. We assume that the regressor is deterministic (in practice this may be an unrealistic assumption but it serves well as point of departure in theory).

(3) Regarding the random variables $latex u_1, \ldots, u_n$ the minimal assumption is that their mean is zero:

$latex E u_1 = \ldots = E u_n = 0.$

An immediate consequence of these assumptions is that the dependence of population means of $latex y_i$ on $latex x_i$ is linear: $latex E y_i = \beta_1 + \beta_2 x_i$. Our first task is to estimate the intercept $latex \beta_1$ and the slope $latex \beta_2$.

Previous: Algebra of sample means | Next: OLS: definition and derivation

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.