Question

Errors (a) In hypothesis testing, we generally want the lowest possible false positive rate, which is controlled by our a parameter. The false positive rate (type I error rate) can be stongly reduced by setting a to a very small decimal. Explain why this is not advisable. (b) Imagine I had a set of 1000 samples (not observations) of data drawn from a specific non-normal distribution p(x), and I perform a Shapiro-Wilk test on each sample, rejecting my null 640 times What can I say about the Shapiro-Wilk test in regard to p(x)? (c) Imagine I had a set of 1000 pairs of samples (observations) of data drawn from the sample normal population, and I perform an F test for equality of variance with a 95% confidence level. Approximately how many rejections of the null do you anticipate making? (d) Imagine I had a set of 1000 pairs of samples (observations) of data drawn from the sample non- normal population, and I perform an F test for equality of variance with a 95% confidence level. Approximately how many rejections of the null do you anticipate making? Can I say specifically? What about relatively?

Answer 1

This is a simple problem related to understanding the reasons for certain circumstances in hypothesis testing .

1. In a hypothesis testing we try to have a lower alpha levels (an indicator of type I error -reject null hypothesis when it is true) .This is done to reduce type I error.However think from another perspective . If we keep on decreasing alpha then on the other side, this will also increase on the chance of committing the type II erro (null hypothesis accept when it is false).

The consequence here is that if the null hypothesis is false, it may be more difficult to reject using a low value for α\alphaαalpha. So using lower values of α\alphaαalpha can increase the probability of a Type II error

If we reduce the alpha levels then it will have an impact on the sample size. It has to increase. Sample size planning aims at choosing a sufficient number of subjects to keep alpha and beta at acceptably low levels without making the study unnecessarily expensive or difficult.

That is why ewe need to have a balanced selectino of alpha.

2). Shapiro Wilk test is a test of checking normality of the distribution in a sample.

In our example ,we have taken 1000 samples and out of it 640 tests have rejected the null hypothesis according to the shapiro wilk test.

The null-hypothesis of this test is that the population is normally distributed. Thus, on the one hand, if the p-value is less than the chosen alpha level, then the null hypothesis is rejected and there is evidence that the data tested are not normally distributed. 64 % of the data sample tested indicated that the samples taken were non normal . But we have taken 1000 non normal samples .

This means that about 36% data samples(360 out of 1000) are certified as normal while they are not. This leads to type II error .

3). Now in this case we have been given with 1000 pairs of samples which are normal and we do the F test for equality of variance of each of them .

The hypothesis that the two variances are equal is rejected if

F>Fα,N1−1,N2−1		for an upper one-tailed test
F<F1−α,N1−1,N2−1		for a lower one-tailed test
F<F1−α/2,N1−1,N2−1 or F>Fα/2,N1−1,N2−1		for a two-tailed test

(F denote concerned F statistic )

Clearly all samples will be rejected which will fall below F(0.95,999,999) or above (F0.05,999,999) .

The degree of freedom N1-1 and N2-1 is equal to (1000-1)=999 for each case.

The calculated F statistic is 1.11 (from df1=999,df2=999 and alpha =0.95)

We may expect at most 5 % rejectino of null hypothesis.( for a normal distribution )

4). Now if the same thing is given for a non normal distribution .

In statistics, an F-test of equality of variances is a test for the null hypothesis that two normal populations have the same variance. Notionally, any F-test can be regarded as a comparison of two variances .

So we will not be in a position to tell for sure about the proportion of rejects of null hypothesis of this non normal distribution with the given set of information.