Question

Answer needs to be clear for thumbs up

Why do we care about doing multiple comparisons? Is there a requirement to adjust confidence intervals or only to adjust hypothesis tests?

Answer 1

So in hypothesis testing, specially in ANOVA, one the hypothesis is rejected

$H_0: \mu_1 = \mu_2 = \mu_3$ vs $H_1: H_0 is false$

We need to know why it got rejected? Are all means different? Or just that two are same and 3rd is different. Which ones are same. To know this we do multiple testing of pairwise mean. Then the hypothesis becomes:

$H_0: \left ( \mu_1 = \mu_2, \mu_2 = \mu_3, \mu_3=\mu_1\right )$ vs H1: H0 is false

Now we need to adjust the confidence interval since the size is still  $\alpha = 0.05$. If we keep same CI then we might make mistake in 3 hypothesis above and end up making Type I error upto 3*$\alpha = 0.15$ which we do not want. Think of cases where there is 24 pairs of means?

That's why we need to adjust the Hypothesis as well as Confidence interval