Raising the bar

Chapter 9. Hypothesis Testing

Unit 9.5. Power of a test

There are not many exercises for assessing the power of a test, so this will probably be your first and last chance to learn the topic.

From the definition

$latex \beta = P(\textrm{Rejecting the alternative when it is true})$

we see that we have to find the probability of the nonrejection region (= acceptance region), where the nonrejection region is found under the null and the probability is found under the alternative hypothesis, and then put Power $latex = 1 - \beta$. More precisely, there are three steps:

Step 1. See Figure 9.5 where the "old" acceptance region (under the dashed density) is the same as in Figure 9.3. $latex \beta$ is its probability under the continuous density (blue-hatched region). Fix the sampling distribution by selecting a parameter that satisfies $latex H_0$, choose $latex \alpha$ and find the nonrejection region $latex N$. If the sample statistics falls into $latex N$, we reject the alternative.

Step 2. Suppose that the alternative is true. Fix any value $latex \mu = \mu^{*}$ of the parameter that satisfies $latex H_a$ and find the probability of rejecting $latex H_a$ under the NEW sampling distribution

$latex \beta = P(\textrm{sample statistic belongs to } N \, | \, \mu = \mu^{*}).$

$latex \beta$ estimates $latex P(\textrm{Type II error})$.

Step 3. Then Power $latex = 1 - \beta$.

Figure 9.5. Probability of the "old" acceptance region under "new" density.

Example 9.1. For the sample mean $latex \bar X$ test $latex H_0: \mu = \mu_0$ against $latex H_a: \mu \ge \mu_0$, assuming $latex \sigma$ is known and the sample size is $latex n$.

Step 1. This is a one-tail test. The margin of error is $latex ME = z_\alpha \sigma/\sqrt n$, the rejection region is $latex R = {\bar X \ge \mu_0 + ME}$ and the nonrejection region is $latex N = {\bar X < \mu_0 + ME}$. Draw an analog of Figure 9.5. $latex N$ has been found under the OLD density $latex p_{\bar X}(t | \mu = \mu_0)$.

Step 2. Now instead of $latex \mu_0$ satisfying $latex H_0$ we take $latex \mu^*$ satisfying $latex H_a$. With the new parameter we have $latex E\bar X = \mu^{*}$ and the standardized sample mean is $latex z = (\bar X - \mu^{*})/(\sigma/\sqrt n)$. The probability of the OLD region $latex N$ under the NEW density $latex p_{\bar X}(t | \mu = \mu^{*})$ is

$latex \beta = P(N | \mu = \mu^{*}) = P(\bar X \le \mu_0 + ME | \mu = \mu^{*})$

$latex = P\left( \frac{\bar X - \mu^{*}}{\sigma/\sqrt n} \le \frac{\mu_0 + ME - \mu^{*}}{\sigma/\sqrt n} \right) = P\left( z \le \frac{\mu_0 + ME - \mu^{*}}{\sigma/\sqrt n} \right).$

Step 3, concluding. Power $latex = 1 - \beta$.

Exercise 9.3. Repeat the procedure for finding the power in one of the following cases:

(a) $latex H_0: \mu = \mu_0, \, H_a: \mu > \mu_0$

(b) $latex H_0: \mu \le \mu_0, \, H_a: \mu > \mu_0$

(c) $latex H_0: \mu = \mu_0, \, H_a: \mu < \mu_0$

(d) $latex H_0: \mu \ge \mu_0, \, H_a: \mu < \mu_0$

(e) $latex H_0: \mu = \mu_0, \, H_a: \mu \ne \mu_0$

Exercise 9.4. Answer the following questions:

1. What happens to the power when $latex \mu^{*}$ moves away from $latex \mu_0$?
2. What happens to $latex 1 - \beta$ when $latex \alpha$ decreases?
3. How does the sample size affect the power?
4. How does $latex 1 - \beta$ behave if $latex \sigma$ increases?
5. Summarize your answers in an intuitive, easy-to-remember form that would not appeal to graphs.

Previous: p-values | Next: Testing for equality of two sample variances

Want the whole book (all 14 chapters, exercises, and the index) as a single PDF? Available on Lulu for $1.99.