Question

In a study of the effectiveness of an arts appreciation class to improve student creativity, researchers randomly assigned 200 third-grade students to participate in an experimental arts program. Another 200 third-grade students were randomly assigned to participate in a traditional arts education program. At the end of the study, students were measured on a creativity test. The mean score for the treatment group was 78.5 and the mean score for the control group was 77.4. The researchers did significance test to see whether the treatment group scored higher, on average. The p-value was 0.23. What is a reasonable interpretation of these results?

• A. There is a 77% chance that the null hypothesis is correct.

• B. There is only a 23% chance that the observed differences are real, and so only a 23% chance that the program was effective.

• C. Using a 5% significance level, we can conclude that the program was not effective, but there is a 5% chance this conclusion is wrong.

• D. The observed differences in means could be due to chance.

Answer 1

Reasonable interpretation of these results

B. There is only a 23% chance that the observed differences are real, and so only a 23% chance that the program was effective.