Click on the following links. Please note these will open in a new window.

Article 1: Ho, A. D., & Yu, C. C. (2015). Descriptive statistics for modern test score distributions. Educational and Psychological Measurement, 75(3), 365–388. doi:10.1177/0013164414548576

Summary/Abstract: Many statistical analyses benefit from the assumption that unconditional or conditional distributions are continuous and normal. More than 50 years ago in this journal, Lord and Cook chronicled departures from normality in educational tests, and Micerri similarly showed that the normality assumption is met rarely in educational and psychological practice. In this article, the authors extend these previous analyses to state-level educational test score distributions that are an increasingly common target of high-stakes analysis and interpretation. Among 504 scale-score and raw-score distributions from state testing programs from recent years, nonnormal distributions are common and are often associated with particular state programs. The authors explain how scaling procedures from item response theory lead to nonnormal distributions as well as unusual patterns of discreteness. The authors recommend that distributional descriptive statistics be calculated routinely to inform model selection for large-scale test score data, and they illustrate consequences of nonnormality using sensitivity studies that compare baseline results to those from normalized score scales.

Questions to Consider

1. How are skewness and kurtosis used as rough indicators of the degree of normality of distributions?

2. The authors define ceiling and floor effects as insufficient measurement precision to support desired distinctions between examinees at the ______ and ____ regions of the score scale.

1. upper; middle
2. upper; lower
3. lower; upper
4. third; fourth

3. For a normal distribution skewness is __ and kurtosis is __.

1. 1; 2
2. 3; 4
3. 0; 3
4. 3; 1
    

Article 2: Zijlstra, W. P., Ark, L. A., & Sijtsma, K. (2010). Outliers in questionnaire data: Can they be detected and should they be removed? Journal of Educational and Behavioral Statistics, 36(2), 186–212. doi:10.3102/1076998610366263.

Summary/Abstract: Outliers in questionnaire data are unusual observations, which may bias statistical results, and outlier statistics may be used to detect such outliers. The authors investigated the effect outliers have on the specificity and the sensitivity of each of six different outlier statistics. The Mahalanobis distance and the item-pair-based outlier statistics were found to have the best combination of specificity and sensitivity. Next, it was investigated how outliers influenced the bias in the percentile rank score, Cronbach’s alpha, and the validity coefficient. Outliers due to random responding and faking produced considerable bias, and outliers due to extreme responding produced little bias. Finally, the influence of removing discordant observations on bias was studied. Removing observations due to random responding identified by means of the Mahalanobis distance, the local outlier factor, and the item-pair-based outlier statistic reduced bias.

Questions to Consider

1. What are the differences between regular, contaminant, and suspect observations?

2. Which of the following is a not a statistical approach to detect outliers?

1. item-based outlier statistic
2. item-based outlier statistic
3. LOF
4. divergent style score

3. The authors found that random responding and faking resulted in:

1. more bias than extreme responding
2. less bias than extreme responding
3. extreme outliers
4. less bias