Raising the bar

Chapter 12. Analysis of variance (ANOVA)

Unit 12.4. Two-way ANOVA: decompositions and F-tests

Leading idea. From Eq. (12.6) it follows that $latex \bar x_{g \cdot}$ estimates $latex \mu + \mu_g$ (because $latex E \bar x_{g \cdot} = \mu + \mu_g$), $latex \bar x_{\cdot b}$ estimates $latex \mu + \mu_b$ (because $latex E \bar x_{\cdot b} = \mu + \mu_b$), and $latex \bar x$ estimates $latex \mu$ (because $latex E \bar x = \mu$). Therefore in the representation

$latex x_{gb} - \bar x = (\bar x_{g \cdot} - \bar x) + (\bar x_{\cdot b} - \bar x) + (x_{gb} - \bar x_{g \cdot} - \bar x_{\cdot b} + \bar x) \qquad (\textrm{Eq. 12.9})$

the term $latex (\bar x_{g \cdot} - \bar x)$ estimates $latex \mu_g$, the term $latex (\bar x_{\cdot b} - \bar x)$ estimates $latex \mu_b$ and $latex (x_{gb} - \bar x_{g \cdot} - \bar x_{\cdot b} + \bar x)$ is a sample analog of $latex \varepsilon_{gb}$.

Definition 12.2 (Sums of squares for two-way ANOVA).

Total sum of squares: $latex SST = \sum_{g, b} (x_{gb} - \bar x)^2$.

Group sum of squares: $latex SSG = \sum_{g, b} (\bar x_{g \cdot} - \bar x)^2 = B \sum_g (\bar x_{g \cdot} - \bar x)^2$.

Block sum of squares: $latex SSB = \sum_{g, b} (\bar x_{\cdot b} - \bar x)^2 = G \sum_b (\bar x_{\cdot b} - \bar x)^2$.

Sum of squared errors: $latex SSE = \sum_{g, b} (x_{gb} - \bar x_{g \cdot} - \bar x_{\cdot b} + \bar x)^2$.

Exercise 12.7. Prove that

$latex SST = SSG + SSB + SSE. \qquad (\textrm{Eq. 12.10})$

Solution. Squaring identity (Eq. 12.9) gives

$latex SST = \sum_{g, b} (x_{gb} - \bar x)^2$

$latex = \sum_{g, b} (\bar x_{g \cdot} - \bar x)^2 + \sum_{g, b} (\bar x_{\cdot b} - \bar x)^2 + \sum_{g, b} (x_{gb} - \bar x_{g \cdot} - \bar x_{\cdot b} + \bar x)^2 + S_1 + S_2 + S_3.$

Here $latex S_1, S_2, S_3$ denote cross-products:

$latex S_1 = 2 \sum_{g, b} (\bar x_{g \cdot} - \bar x)(\bar x_{\cdot b} - \bar x) = 2 \sum_g (\bar x_{g \cdot} - \bar x) \sum_b (\bar x_{\cdot b} - \bar x)$

$latex = 2 (G \bar x - G \bar x)(B \bar x - B \bar x) = 0,$

$latex S_2 = 2 \sum_{g, b} (\bar x_{g \cdot} - \bar x)(x_{gb} - \bar x_{g \cdot} - \bar x_{\cdot b} + \bar x)$

$latex = 2 \sum_g (\bar x_{g \cdot} - \bar x) \sum_b (x_{gb} - \bar x_{g \cdot} - \bar x_{\cdot b} + \bar x)$

$latex = 2 \sum_g (\bar x_{g \cdot} - \bar x)(B \bar x_{g \cdot} - B \bar x_{g \cdot} - B \bar x + B \bar x) = 0,$

$latex S_3 = 2 \sum_{g, b} (\bar x_{\cdot b} - \bar x)(x_{gb} - \bar x_{g \cdot} - \bar x_{\cdot b} + \bar x)$

$latex = 2 \sum_b (\bar x_{\cdot b} - \bar x) \sum_g (x_{gb} - \bar x_{g \cdot} - \bar x_{\cdot b} + \bar x)$

$latex = 2 \sum_b (\bar x_{\cdot b} - \bar x)(G \bar x_{\cdot b} - G \bar x - G \bar x_{\cdot b} + G \bar x) = 0.$

It remains to apply the definitions of $latex SSG, SSB, SSE$ to finish the proof.

Three types of null hypotheses.

1. All observations come from the same population: $latex H_0: \mu_g = \mu_b = 0$ for all $latex g, b$.

2. There are no differences between groups: $latex H_0^{\textrm{groups}}: \mu_g = 0$ for all $latex g$.

3. There are no differences between blocks: $latex H_0^{\textrm{blocks}}: \mu_b = 0$ for all $latex b$.

Exercise 12.8.

$latex \quad$(1) Find the distribution of $latex SST$ under the null $latex H_0$.

$latex \quad$(2) Find the distribution of $latex SSG$ under $latex H_0^{\textrm{groups}}$.

$latex \quad$(3) Find the distribution of $latex SSB$ under $latex H_0^{\textrm{blocks}}$.

$latex \quad$(4) Find the distribution of $latex SSE$ under the null $latex H_0$.

Solution. (1) From Eq. (12.4) we see that under the null hypothesis $latex H_0$ the variables $latex x_{gb}$ are i.i.d. normal with mean $latex \mu$ and variance $latex \sigma^2$. By Theorem 12.1

$latex SST / \sigma^2 = \chi_{GB - 1}^2. \qquad (\textrm{Eq. 12.11})$

(2) If $latex H_0^{\textrm{groups}}$ holds, then from Eq. (12.6) we see that $latex \bar x_{g \cdot}$ have mean $latex \mu$, and by Eq. (12.7) their variances are $latex \sigma^2 / B$. Since they are i.i.d. normal, Theorem 12.1 allows us to conclude that $latex \sum_g (\bar x_{g \cdot} - \bar x)^2 / (\sigma^2 / B) = \chi_{G - 1}^2$. This equation can be equivalently written as

$latex SSG / \sigma^2 = \chi_{G - 1}^2. \qquad (\textrm{Eq. 12.12})$

(3) If $latex H_0^{\textrm{blocks}}$ holds, then from Eq. (12.6) we see that $latex \bar x_{\cdot b}$ have mean $latex \mu$, and by Eq. (12.8) their variances are $latex \sigma^2 / G$. As above, since $latex \bar x_{\cdot b}$ are i.i.d. normal, by Theorem 12.1 $latex \sum_b (\bar x_{\cdot b} - \bar x)^2 / (\sigma^2 / G) = \chi_{B - 1}^2$. This equation can be equivalently written as

$latex SSB / \sigma^2 = \chi_{B - 1}^2. \qquad (\textrm{Eq. 12.13})$

(4) If $latex H_0$ holds, then $latex H_0^{\textrm{groups}}$ and $latex H_0^{\textrm{blocks}}$ also hold and equations (Eq. 12.11)$latex -$(Eq. 12.13) are true. Assuming that $latex SSG, SSB$ and $latex SSE$ are independent (we lack the tools to prove this), by the subtraction property of chi square distributions we see that

$latex \frac{SSE}{\sigma^2} = \frac{SST - SSG - SSB}{\sigma^2} = \chi_{GB - 1}^2 - \chi_{G - 1}^2 - \chi_{B - 1}^2 = \chi_{(G - 1)(B - 1)}^2.$

Now it is a pure formality to rewrite the relationships obtained using the familiar format. Let $latex MSG = SSG / (G - 1)$, $latex MSB = SSB / (B - 1)$, $latex MSE = SSE / [(G - 1)(B - 1)]$. Then

$latex \frac{(G - 1) MSG}{\sigma^2} = \chi_{G - 1}^2, \quad \frac{(B - 1) MSB}{\sigma^2} = \chi_{B - 1}^2,$

$latex \frac{(G - 1)(B - 1) MSE}{\sigma^2} = \chi_{(G - 1)(B - 1)}^2.$

We summarize the results in Table 12.4.

Table 12.4. Two-way ANOVA.

Source of variation	Sum of squares	Degrees of freedom	Mean squares	F ratio
Between groups	$latex SSG$	$latex G - 1$	$latex MSG = SSG/(G - 1)$	$latex MSG/MSE = F_{G-1,(B-1)(G-1)}$
Between blocks	$latex SSB$	$latex B - 1$	$latex MSB = SSB/(B - 1)$	$latex MSB/MSE = F_{B-1,(B-1)(G-1)}$
Error	$latex SSE$	$latex (B - 1)(G - 1)$	$latex MSE = SSE/[(B - 1)(G - 1)]$
Total	$latex SST$	$latex BG - 1$

Decision rule. (a) Reject the hypothesis that there is no difference between groups if at the $latex \alpha$ significance level $latex MSG/MSE > F_{G - 1, (B - 1)(G - 1), \alpha}$ where the critical value $latex F_{G - 1, (B - 1)(G - 1), \alpha}$ is defined by $latex P(F_{G - 1, (B - 1)(G - 1)} > F_{G - 1, (B - 1)(G - 1), \alpha}) = \alpha$.

(b) Reject the hypothesis that there is no difference between blocks if at the $latex \alpha$ significance level $latex MSB/MSE > F_{B - 1, (B - 1)(G - 1), \alpha}$ where the critical value $latex F_{B - 1, (B - 1)(G - 1), \alpha}$ is defined by $latex P(F_{B - 1, (B - 1)(G - 1)} > F_{B - 1, (B - 1)(G - 1), \alpha}) = \alpha$.

---

Previous: Two-way ANOVA: setup and population model | Next: Questions for repetition (Chapter 12)

---

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.