Journal Articles

Dunning, Thad. 2008. Improving Causal Inference: Strengths and Limitations of Natural Experiments. Political Research Quarterly 61 (2): 282-293.

Abstract: Social scientists increasingly exploit natural experiments in their research. This article surveys recent applications in political science, with the goal of illustrating the inferential advantages provided by this research design. When treatment assignment is less than “as if” random, studies may be something less than natural experiments, and familiar threats to valid causal inference in observational settings can arise. The author proposes a continuum of plausibility for natural experiments, defined by the extent to which treatment assignment is plausibly “as if” random, and locates several leading studies along this continuum.

Discussion Questions:

  1. Please discuss how natural experiments differ from true randomized experiments, and why they have grown as a popular mode of analysis in social science research.
  2. Please what benefits natural experiments have provided social scientists in terms of being more able to make substantive inferences in research.
  3. Please discuss what limitations the author identifies when using natural experiments.

 

Green, Jane. 2010. Points of Intersection Between Randomized Experiments and Quasi-Experiments. The Annals of the American Academy of Political and Social Science 628 (1): 97-111. 

Abstract: There are several contexts in which randomization in research design is not possible. Researchers may be interested in the effects of large-scale events or policy initiatives, or of complex causal processes that cannot be replicated. Threats to external validity in randomized experiments will lead to greater dividends for some research questions in a quasi-experimental design. Quasi-experiments can offer opportunities for testing causal theories, if they are carefully constructed to counter threats to internal validity. Furthermore, randomization is not a simple guarantee against threats to internal validity where few units of observation are used. An appreciation of the intersection of these approaches—between stronger quasi-experiments and weaker randomized experiments—can help guide our choice of experimental method. One can make the same argument about the benefits of pursuing good opportunities with quasi-experiments over scant opportunities with randomized experiments as is made more often in reverse.

Discussion Questions:

  1. How are quasi-experiments distinguished from pure experiments, and where may there be intersections in both approaches?
  2. Please discuss situations in research where pure randomized research designs are not feasible strategies.
  3. Please discuss the potential for generating internal and external validity in quasi-experiments.

 

Stellmack, Mark A. 2013. Attenders Versus Slackers: A Classroom Demonstration of Quasi-Experimentation and Self-Selecting Samples. Teaching of Psychology 40 (3): 238-241. 

Abstract: Studies of the effects of class attendance on class performance typically are quasi-experimental because students choose whether or not to attend class; that is, the samples are self-selecting. The lack of random assignment prevents one from establishing a causal relationship between attendance and performance. Relating attendance to performance using the students in a research methods class, regardless of whether the data show a significant relationship, can serve as the basis of a discussion of quasi experiments and the alternative explanations that are inherent in their design. This article gives suggestions for presenting and discussing the data.

Discussion Questions:

  1. Please discuss why a quasi-experiment was used as the approach for studying the effects of class attendance on performance, and what recommendations the researcher offers for addressing limitations of the study.
  2. What issues of internal and external validity are presented in this research?
  3. What applications does this study provide to real-world situations?

 

Taylor, Bruce, and Daniel J. Woods. 2010. “Injuries to Officers and Suspects in Police Use-of-Force Cases: A Quasi-Experimental Evaluation.” Police Quarterly 13(3): 260 -289.

Abstract: The Conducted Energy Device (CED) weapon holds the potential to reduce injuries for officers/suspects. However, the dearth of research on CEDs makes it difficult to make informed decisions about its deployment. We conducted a quasi experiment to compare 4 years of data from seven law enforcement agencies (LEAs) with CED deployment with six matched LEAs without CED deployment. Compared with non-CED sites, CED sites had lower rates of officer injuries, suspect severe injuries, and officers and suspects receiving injuries requiring medical attention. Our results suggest that CEDs can be effective in helping minimize physical struggles and resulting injuries in use-of-force cases.

Discussion Questions:

  1. Please discuss why a quasi-experiment was an appropriate approach to study the CED weapon versus other types of experiments.
  2. Please discuss why a randomized clinical trial is not a possible strategy for this study, and how the researchers consider this in their design.
  3. What limitations / barriers do the researchers identify as inherent to their research?