Click on the following links. Please note these will open in a new window.

Article 1: Algina, J., Oshima, T. C., & Lin, W. (1994). Type I error rates for welch’s test and james’s second-order test under nonnormality and inequality of variance when there are two groups. Journal of Educational and Behavioral Statistics, 19(3), 275. doi:10.2307/1165297.

Summary/Abstract: Type I error rates were estimated for three tests that compare means by using data from two independent samples: the independent samples t-test, Welch’s approximate degrees of freedom test, and James-s second-order test. Type I error rates were estimated for skewed distributions, equal and unequal variances, equal and unequal sample sizes, and a range of total sample sizes. Welch’s test and James’s test have very similar Type I error rates and tend to control the Type I error rate as well or better than the independent samples t-test does. The results provide guidance about the total sample sizes required for controlling Type I error rates.

Questions to Consider

1. Under normality but with variance inequality, how does sample size (small versus large) impact the actual Type I error rate of the independent samples t-test?

2. The authors state that when the data are sampled from distributions that are skewed and the variances are different for the sampled populations that:

1. Welch’s test and James’s second-order test equally control Type I error.
2. Welch’s test and independent samples t-test equally control Type I error.
3. independent-samples t-tests control Type I error.
4. James’s second-order test and independent samples t-test equally control Type I error.

3. The power of ____ test tends to decline slightly as skew increases.

1. Independent samples t
2. Welch’s
3. James’s second-order
4.  ANOVA
    

Article 2: Keselman, H., Othman, A. R., Wilcox, R. R., & Fradette, K. (2004). The new and improved two-sample t test. Psychological Science, 15(1), 47–51. doi:10.1111/j.0963-7214.2004.01501008.x.

Summary/Abstract: This article considers the problem of comparing two independent groups in terms of some measure of location. It is well known that with Student’s two-independent-sample t-test, the actual level of significance can be well above or below the nominal level, confidence intervals can have inaccurate probability coverage, and power can be low relative to other methods. A solution to deal with heterogeneity is Welch’s (1938) test. Welch’s test deals with heteroscedasticity but can have poor power under arbitrarily small departures from normality. Yuen (1974) generalized Welch’s test to trimmed means; her method provides improved control over the probability of a Type I error, but problems remain. Transformations for skewness improve matters, but the probability of a Type I error remains unsatisfactory in some situations. We find that a transformation for skewness combined with a bootstrap method improves Type I error control and probability coverage even if sample sizes are small.

Questions to Consider

1. According to the authors, researchers can achieve better Type I error control by doing what?

2. Student’s two-independent-sample t-test can have poor power under arbitrarily _____ from normality.

1. large departures
2. intermediate departures
3. small departures
4. extreme departures

3. To decrease the probability of Type I error rates you should do all of the following except for:

1. applying a transformation to a heteroscedastic statistic.
2. assessing significance through a bootstrap method.
3. applying a moderate amount of trimming.
4. standardizing the data set.