Question

how to pick ? and ? for a test by looking at type I and type II error consequences

Answer 1

A Type I error is often represented by the Greek letter alpha (?) and a Type II error by the Greek letter beta (? ). In choosing a level of probability for a test, you are actually deciding how much you want to risk committing a Type I error—rejecting the null hypothesis when it is, in fact, true. For this reason, the area in the region of rejection is sometimes called the alpha level because it represents the likelihood of committing a Type I error.

In order to graphically depict a Type II, or ?, error, it is necessary to imagine next to the distribution for the null hypothesis a second distribution for the true alternative (see Figure 1). If the alternative hypothesis is actually true, but you fail to reject the null hypothesis for all values of the test statistic falling to the left of the critical value, then the area of the curve of the alternative (true) hypothesis lying to the left of the critical value represents the percentage of times that you will have made a Type II error.

Figure 1.Graphical depiction of the relation between Type I and Type II errors, and the power of the test.

Type I and Type II errors are inversely related: As one increases, the other decreases. The Type I, or ? (alpha), error rate is usually set in advance by the researcher. The Type II error rate for a given test is harder to know because it requires estimating the distribution of the alternative hypothesis, which is usually unknown.

A related concept is power—the probability that a test will reject the null hypothesis when it is, in fact, false. You can see from Figure 1 that power is simply 1 minus the Type II error rate (?). High power is desirable. Like ?, power can be difficult to estimate accurately, but increasing the sample size always increases power.