Raising the bar

Chapter 12. Analysis of variance (ANOVA)

Unit 12.3. Two-way ANOVA: setup and population model

Difference between one-way and two-way ANOVA.

One-way ANOVA: three car makes are tested for differences in fuel consumption. The differences may be influenced by driving habits of drivers selected to test the cars. To remove this difference, out of a pool of, say, 15 drivers three teams of 5 drivers are randomly selected to test the three car types. Thus, the variability in fuel consumption is caused only by one factor $latex -$ car make.

Two-way ANOVA attempts to reveal dependence of variability on two factors: car make and driver habits. The second variable is called a blocking factor. Unlike the one-way ANOVA example above, all drivers drive all cars. If there are many drivers, they can be joined into groups (called blocks), say, by age.

Table 12.3. Visualization of data for two-way ANOVA. Observation $latex x_{gb}$ is recorded for group $latex g$ (e.g., car make) in block $latex b$ (e.g., driver group). Every group is observed in every block.

Table 12.3. Visualization of data for two-way ANOVA.

Exercise 12.5.

$latex \quad$(1) Write out the formulas for the sample means by group (summation over $latex b$) and by block (summation over $latex g$).

$latex \quad$(2) What are the expressions for the total number of observations, for the overall sample mean, and how is the overall sample mean related to the group and block means?

Solution. (1) We use a dot to indicate a variable that has been averaged out. For example, in the notation of the group sample mean the dot is put in place of $latex b$ to indicate that the block variable has been used for averaging:

$latex \bar x_{g \cdot} = \frac{1}{B} \sum_{b=1}^B x_{gb}, \quad g = 1, \ldots, G.$

Similarly, the block sample mean is defined by

$latex \bar x_{\cdot b} = \frac{1}{G} \sum_{g=1}^G x_{gb}, \quad b = 1, \ldots, B.$

(2) The total number of observations is $latex n = GB$. The overall sample mean

$latex \bar x = \frac{1}{n} \sum_{g, b} x_{gb}$

is related to the block and group means by

$latex \bar x = \frac{1}{B} \sum_{b=1}^B \bar x_{\cdot b} = \frac{1}{G} \sum_{g=1}^G \bar x_{g \cdot}.$

Population model. Assume that the observations follow a linear model

$latex x_{gb} = \mu + \mu_g + \mu_b + \varepsilon_{gb} \qquad (\textrm{Eq. 12.4})$

where the errors $latex \varepsilon_{gb}$ are normal, independent, have mean zero and the same variance $latex \sigma^2$. $latex \mu$ is an overall population mean. The parameters $latex \mu_g, \mu_b$ give group- and block-specific corrections and satisfy

$latex \sum_b \mu_b = \sum_g \mu_g = 0. \qquad (\textrm{Eq. 12.5})$

This condition tells us that $latex \mu_b$ and $latex \mu_g$ may have alternating signs. Depending on their signs, $latex \mu + \mu_g + \mu_b$ may be higher or lower than $latex \mu$.

Exercise 12.6. Derive the population models for $latex \bar x_{g \cdot}, \bar x_{\cdot b}, \bar x$.

Solution. Just average equations (Eq. 12.4) and use equations (Eq. 12.5) to get

$latex \bar x_{g \cdot} = \mu + \mu_g + \bar \varepsilon_{g \cdot}, \quad \bar x_{\cdot b} = \mu + \mu_b + \bar \varepsilon_{\cdot b}, \quad \bar x = \mu + \bar \varepsilon. \qquad (\textrm{Eq. 12.6})$

Since the errors are i.i.d., we have by homogeneity and additivity of variance

$latex V(\bar x_{g \cdot}) = V(\bar \varepsilon_{g \cdot}) = \frac{1}{B^2} V\left( \sum_{b=1}^B \varepsilon_{gb} \right) = \frac{1}{B^2} \sum_{b=1}^B V(\varepsilon_{gb}) = \frac{\sigma^2}{B}, \qquad (\textrm{Eq. 12.7})$

$latex V(\bar x_{\cdot b}) = V(\bar \varepsilon_{\cdot b}) = \frac{1}{G^2} V\left( \sum_{g=1}^G \varepsilon_{gb} \right) = \frac{1}{G^2} \sum_{g=1}^G V(\varepsilon_{gb}) = \frac{\sigma^2}{G}. \qquad (\textrm{Eq. 12.8})$

We shall use also the fact that $latex \bar \varepsilon_{g \cdot}, g = 1, \ldots, G$, are independent and $latex \bar \varepsilon_{\cdot b}, b = 1, \ldots, B$, are also independent.

Previous: One-way ANOVA: distributions and the F-test | Next: Two-way ANOVA: decompositions and F-tests

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.