Raising the bar

Chapter 13. Bridge to econometrics

Unit 13.3. Model with two regressors

The purpose is to derive equation (3.11) from the book. This will be done in two steps.

Step 1. A linear system with two unknowns. We start with formulas for solutions of a system of linear equations with two unknowns. Consider the system

$latex \left\{ \begin{array}{l} a_{11} x_1 + a_{12} x_2 = b_1, \\ a_{21} x_1 + a_{22} x_2 = b_2. \end{array} \right. \qquad (\textrm{Eq. 13.2})$

Here $latex x_1, x_2$ are unknown and everything else is assumed known. We use the elimination method: the given equations are rearranged in such a way that it is easy to eliminate one of the unknowns and solve the resulting equation for the other. Multiplying the first equation by $latex a_{22}$ and the second by $latex a_{12}$ we get

$latex \begin{array}{rcl} a_{11} a_{22} x_1 + a_{12} a_{22} x_2 & = & b_1 a_{22}, \\ a_{21} a_{12} x_1 + a_{22} a_{12} x_2 & = & b_2 a_{12}. \end{array}$

The coefficients of $latex x_2$ here are the same. If we subtract the second equation from the first, $latex x_2$ will disappear:

$latex (a_{11} a_{22} - a_{21} a_{12}) x_1 = b_1 a_{22} - b_2 a_{12}. \qquad (\textrm{Eq. 13.3})$

We want to rewrite this equation using a couple of definitions from matrix algebra.

Definition (Matrix and determinant). By definition, a matrix is a table of numbers:

$latex A = \left( \begin{array}{cc} a_{11} & a_{12} \\ a_{21} & a_{22} \end{array} \right).$

Its determinant is defined as

$latex \det A = a_{11} a_{22} - a_{21} a_{12}.$

One can see that the expression on the right-hand side of Eq. (13.3) is the determinant of the matrix

$latex B_1 = \left( \begin{array}{cc} b_1 & a_{12} \\ b_2 & a_{22} \end{array} \right).$

With the notation introduced Eq. (13.3) rewrites as

$latex x_1 \det A = \det B_1. \qquad (\textrm{Eq. 13.4})$

Denote

$latex B_2 = \left( \begin{array}{cc} a_{11} & b_1 \\ a_{21} & b_2 \end{array} \right).$

You can check that similar arguments lead to

$latex x_2 \det A = \det B_2. \qquad (\textrm{Eq. 13.5})$

As we see, if $latex \det A \neq 0$, then equations (Eq. 13.4) and (Eq. 13.5) have solutions

$latex x_1 = \frac{\det B_1}{\det A}, \quad x_2 = \frac{\det B_2}{\det A}. \qquad (\textrm{Eq. 13.6})$

Step 2. Application to a regression with two regressors. Consider a regression (written in vector form) with two regressors:

$latex y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + u.$

Using this model and linearity of covariance with respect to the first argument we have

$latex \begin{array}{rcl} \textrm{cov}(y, x_1) & = & \textrm{cov}(\beta_0 + \beta_1 x_1 + \beta_2 x_2 + u, x_1) \\ & = & \textrm{cov}(\beta_0, x_1) + \beta_1 \textrm{cov}(x_1, x_1) + \beta_2 \textrm{cov}(x_2, x_1) + \textrm{cov}(u, x_1). \end{array}$

Here $latex \textrm{cov}(\beta_0, x_1) = 0$ (covariance with a constant is zero), $latex \textrm{cov}(x_1, x_1) = V(x_1)$. We formally put $latex \textrm{cov}(u, x_1) = 0$ (this is not a rigorous derivation). The result is

$latex \textrm{cov}(y, x_1) = \beta_1 V(x_1) + \beta_2 \textrm{cov}(x_2, x_1).$

Similarly,

$latex \textrm{cov}(y, x_2) = \beta_1 \textrm{cov}(x_1, x_2) + \beta_2 V(x_2).$

We see that the coefficients $latex \beta_1, \beta_2$ satisfy a system of type (Eq. 13.2) with

$latex A = \left( \begin{array}{cc} V(x_1) & \textrm{cov}(x_2, x_1) \\ \textrm{cov}(x_1, x_2) & V(x_2) \end{array} \right), \quad b_1 = \textrm{cov}(y, x_1), \quad b_2 = \textrm{cov}(y, x_2).$

Therefore application of the first equation in (Eq. 13.6) gives

$latex \hat \beta_1 = \frac{\textrm{cov}(y, x_1) V(x_2) - \textrm{cov}(y, x_2) \textrm{cov}(x_2, x_1)}{V(x_1) V(x_2) - [\textrm{cov}(x_1, x_2)]^2}.$

We formally replaced $latex \beta_1$ by $latex \hat \beta_1$. This is equation (3.11) from the book.

Previous: How to read Dougherty's book | Next: Linear dependence of vectors

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.