Question

Create two different data sets such that a T test under the assumption of equal variances will reject the null hypothesis, but the T test under the assumption of unequal variances (Hint: Welch's T statistic) will fail to reject the null hypothesis. Explain why the two data sets fail to reject under unequal variance assumption and why it rejects until equal variance assumption.

Thanks

Answer 1

Two Sample t Test: unequal variances

Theorem 1: Let x̄ and ȳ be the sample means and s_x and s_y be the sample standard deviations of two sets of data of size n_x and n_y respectively. If x and y are normal, or n_x and n_y are sufficiently large for the Central Limit Theorem to hold, then the random variable

has distribution T(m) where

n* ly nx-1 ny-1

Observation: The nearest integer to m can be used.

An alternative calculation (Satterthwaite’s correction) of m (which has the same value) is as follows

(n - 1) (ny-1)

where

2 S

Observation: This theorem can be used to test the difference between sample means even when the population variances are unknown and unequal. The resulting test, called, Welch’s t-test, will have a lower number of degrees of freedom than (n_x – 1) + ( n_y – 1), which was sufficient for the case where the variances were equal. When n_xand n_y are approximately equal, then the degrees of freedom and the value of t in Theorem 1 are approximately the same as those in Theorem 1 of Two Sample t Test with Equal Variances.

Real Statistics Function: The Real Statistics Resource Pack provides the following supplemental function.

DF_POOLED(R1, R2) = degrees of freedom for the two sample t test for samples in ranges R1 and R2, especially when the two samples have unequal variances (i.e. m in Theorem 1).

Excel Function: Excel provides the function TTEST to handle the various two sample t-tests.

TTEST(R1, R2, tails, type) = p-value of the t-test for the difference between the means of two samples R1 and R2, where tails = 1 (one-tailed) or 2 (two-tailed) and type takes the values:

1. the samples have paired values from the same population
2. the samples are from populations with the same variance
3. the samples are from populations with different variances

These three types correspond to the Excel data analysis tools

• t-Test: Paired Two Sample for Mean
• t-Test: Two-Sample Assuming Equal Variance
• t-Test: Two-Sample Assuming Unequal Variance

Note that the type 3 TTEST uses the value of the degrees of freedom as indicated in Theorem 1 unrounded, while the associated data analysis tool rounds the degrees of freedom as indicated in the theorem to the nearest integer. We will explain the type 1 TTEST in Paired Sample t Test.

This function ignores all empty and non-numeric cells. The value of alpha is assumed to be .05.

Example 1: In Example 1 of Two Sample t Test with Equal Variances, we assumed that the population variances were equal since the sample variances were almost the same. We now repeat the analysis assuming that the variances are not necessarily equal.

We use the Excel formula TTEST(A4:A13,B4:B13,2,3). The first two parameters represent the data for each sample (without labels). The 3^rd parameter indicates that we desire a two-tailed test and the 4^th parameter indicates a type 3 test. Since

TTEST(A4:A13,B4:B13,2,3) = 0.043456 < .05 = α

we reject the null hypothesis. Note that if we use the type 2 test, TTEST(R1, R2, 2, 2) = 0.043053, the result won’t be very different, thus confirming our assumption that the population variances are almost equal.

Example 2: We repeat the analysis from Example 1 but with different data for the new flavoring.

3 New Old 12 Box Plot 32 45 16 25 35 30 25 20 15 10 18 14 10 18 10 21 12 28 13 40 14 15 10 16 19.8 17 150.6222 18.76667 variance New Old 10 size 11.1 mean

Figure 1 – Sample data and box plots for Example 2

Clearly, the sample variances are quite unequal. Using the T.TEST function with = 3 we get

T.TEST(A4:A13 ,B4:B13, 2, 3) = 0.05773 > .05 = α

and so this time we cannot reject the null hypothesis (for the two-tailed test). Note that if we had used the test with equal variances, namely T.TEST(A4:A13, B4:B13, 2, 2) = 0.048747 < .05 = α, then we would have rejected the null hypothesis.

We can also use Excel’s t-Test: Two-Sample Assuming Unequal Variances data analysis tool to get the same result (see Figure 2).

35 t-Test: Two-Sample Assuming Unequal Variances 37 38 Mean 39 Variance 40 Observations 41 Hypothesized Mean Difference 42 df 43 t Stat 44 P T<-t) one-tail 45 t Critical one-tail 46 P(T< t) two-tail 47 t Critical two-tail New Old 11.1 150.6222 18.76667 10 19.8 10 0 2.113863 0.029093 1.795885 0.058186 2.200985

Figure 2 – Data analysis for the data from Figure 1

Observation: Generally, even if one variance is up to 4 times the other, the equal variance assumption will give good results. This rule of thumb is clearly violated in Example 2, and so we need to use the t test with unequal population variances.

Real Statistics Data Analysis Tool: The Real Statistics Resource Pack provides a data analysis tool called T Tests and Non-parametric Equivalents, which combines the analyses for equal and unequal variances, as well as providing confidence intervals and the Cohen effect size. A second measure of effect size is also provided, which we will study in Dichotomous Variables and the t-test.

Example 3: Repeat Example 2 using the Real Statistics data analysis tool.

Enter Ctrl-m and select T Tests and Non-parametric Equivalents from the menu. Fill in the dialog box that appears as shown in Figure 3.

T Tests and Non-parametric Equivalents Input Range 1 Input Range 2 Column headings included with data Alpha0.05 Hyp Mean/Median0 A3:B13 Fill OK Fill Cancel Help Test type Options One sample C Two paired samples T test Non-parametric Two independent samples Non-parametric test options rectionUse continuity correction Use ties corr Indude exact test e simulation # of Iterations 10000 Output Range E19

Figure 3 – Dialog box for T Test and Non-parametric Equivalents

Choose the Two independent samples and T test options and press OK. The output appears in Figure 4.

Figure 4 – Real Statistics data analysis for data from Figure 1

19 T Test: Two Independent Samples 20 21 SUMMARY 22 Groups Count Mean Variance Cohen d 23 New 24 Old 25 Pooled 26 27 T TEST: Equal Variances 28 29 One Tai 4.115688 2.113863 30 TwoTa 4.115688 2.113863 31 32 T TEST: Unequal Variances Hyp Mean 10 10 19.8 150.6222 11.1 18.76667 84.69444 0.945348 Alpha 0.05 std errt-stat value t-crit ower u effect r 18 0.024373 1.734064 18 0.048747 2.100922 0.05326 17.34674 yes 0.445955 yes 0.445955 Alpha 0.05 std err t-stat value t-crit loweru effect r SI 34 One Ta 4.115688 2.113863 11.20841 0.028865 1.795885 35 Two T 4.115688 2.113863 11.20841 0.05773 2.200985 -0.35857 17.75857no 0.533885 yes 0.533885

We can see from Figure 4 that the degrees of freedom have been reduced from 18 to 11.208 under the assumption of unequal variances. We can get this same value by using the formula =DF_POOLED(A4:A13, B4:B13).

Observation: The input data for the two independent sample t test can have missing data, indicated by empty cells or cells with non-numeric data. Such cells will be ignored in the analysis.