Question

Which of the following is true when thinking about statistical significance and effect sizes?

A. A statistically significant effect will always have a meaningful effect size.

B. A statistically significant effect will usually have a large effect size.

C. A large effect size is will always be statistically significant.

D. A statistically non-significant effect can have a large effect size.

Answer 1

The correct answer is C. A large effect size is will always be statistically significant. To answer this question, we will use the Cohen’s d as example. The Cohen’s d (or just d) is measure of effect size which is used when two means are compared in a statistical test such as a t-test. D is the difference in the two groups means divided by the average of their standard deviations. So, in this case if we observe a d=1, the two groups 'mean will differ by a half a standard deviation. Cohen made a scale of effect size, where d=0.2 is a small effect size, d=0.5 is a medium effect size and d=0.8 is a large effect size. When two means have not a difference by 0.2 standard deviations or more, the difference are not statistically significant. So, we can expect the a large effect size will be statistically significant.