Question

Explain the influence a level of significance and sample size has on hypothesis testing. Provide an example of the influence and explain how it impacts business decisions.

Answer 1

The significance level can be seen as the probability of rejecting the null hypothesis when it is in fact true. For instance, a significance level of 0.07 will depict that there is a 7% risk of coming to the conclusion that there is a difference but in reality, there is no difference. If we have lower significance, it will mean that one will need stronger evidence before the rejection of the null hypothesis.

We have to compare the p-value with the significance level. In case the significance level is more than the p-value, the null hypothesis can be rejected and we can say that there is a statistical significance. Alternatively, it will indicate that there is enough proof that the sample is quite strong to reject the null hypothesis at the given population level.

The hypothesis test becomes more sensitive when the size of the sample is increased. This indicates that the chances of rejecting the null hypothesis are there when it is in fact false. Therefore the power of the test would increase. The sample size does not have any influence on the effect size and thus the chances of committing Type II error get reduced with the increasing sample size.