Question

A colleague proposes to you that a one-sample t-test should be used to evaluate the claim that 1/3 of the bacteria in a pond have eight flagella. Do you agree? Explain.

Yet another colleague explains to you that they are going to carry out 100 two-sample t-tests to see if the gene expression levels of some genes are associated with various treatments. They assure you, however, that they will choose a small significance level for each test to account for the large number of tests; they will use a significance level of 0.005 for each test. Would you say that this is sufficient to ensure a small familywise error rate? Explain.

Answer 1

1. The claim is that 1/3rd of the bacteria of the pond have eight flagella. Therefore, there are two groups here. One group of bacteria has eight flagella while the other of bacteria does not have eight flagella.

The aim is to check if the claim is true. To check for the validity of the claim, a chi-square test is appropriate since we are to see whether the observed frequencies are different from expected frequencies. The goal is not to compare the means. Therefore, a t-test would be inappropriate.

2. There are 100 two-sample t-tests that would be done. Therefore, there are multiple hypothesis being tested here. To avoid errors due to the testing of multiple hypothesis, a family-wise error correction is usually employed.

Let the case-wise significance level be $\alpha_c$

and the family-wise significance level be $\alpha_f$

Then the relation between the two is:

$\alpha_f = (1 - (1- \alpha_c)^n)$

where n is the number of hypothesis tests being done. For 100 tests, n = 100. Here the significance level being used for each test is 0.005. Substituting the values in the formula, we get;

af = (1 - (1 – 0.0005)100) 1. af = (1 - 0.9995100 :.0= (1 – 0.9512) :: af 0.0488

This level is still lower than a significance level of 0.05 which is normally employed. Therefore, a case-wise significance level of 0.0005 is sufficient to get a family-wise significance level of 0.0488.

asdfa