Larzeler, Robert E., Emilio Ferrer, Brett R. Kuhn, and Ketevan Danelia. 2010. Differences in causal estimates from longitudinal analyses of residualized versus simple gain scores: Contrasting controls for selection and regression artifacts. International Journal of Behavioral Development 34 (2): 180-189.
Abstract: This study estimates the causal effects of six corrective actions for children's problem behaviors, comparing four types of longitudinal analyses that correct for pre-existing differences in a cohort of 1,464 4- and 5-year-olds from Canadian National Longitudinal Survey of Children and Youth (NLSCY) data. Analyses of residualized gain scores found apparently detrimental effects of all corrective actions by parents and professionals on subsequent antisocial behavior and hyperactivity. In contrast, analyses of simple gain scores found only apparently beneficial effects. Temporally reversed analyses yielded the same pattern of results, consistent with selection biases and regression artifacts, not with unidirectional causal effects. The findings were similar for corrective actions by professionals (e.g., Ritalin, psychotherapy) and by parents (physical and nonphysical punishment, scolding/yelling, “hostile-ineffective” parenting). Longitudinal analyses should check for similar artifacts by implementing temporally-reversed analyses and by determining whether causally relevant coefficients would replicate without artifacts biased in their favor.
- What role does causal inference play in human developmental research?
- Please discuss how control variables were used in this study to control for selection bias.
- In utilizing longitudinal data, what steps may researchers take to better be able to draw out causal effects?
Li, Mingxiang. 2013. Using the Propensity Score Method to Estimate Causal Effects: A Review and Practical Guide. Organizational Research Methods 16 (2): 188-226.
Abstract: Evidence-based management requires management scholars to draw causal inferences. Researchers generally rely on observational data sets and regression models where the independent variables have not been exogenously manipulated to estimate causal effects; however, using such models on observational data sets can produce a biased effect size of treatment intervention. This article introduces the propensity score method (PSM)—which has previously been widely employed in social science disciplines such as public health and economics—to the management field. This research reviews the PSM literature, develops a procedure for applying the PSM to estimate the causal effects of intervention, elaborates on the procedure using an empirical example, and discusses the potential application of the PSM in different management fields. The implementation of the PSM in the management field will increase researchers’ ability to draw causal inferences using observational data sets.
- Provide an overview of how multiple regression is used to draw causal inferences in organizational management (i.e. to assess managerial practices, performance, etc.).
- Please discuss the term “causal effect.” What are examples that researchers may explore in organizational analyses?
- Please discuss what advantages the propensity score method generates in increasing a researcher’s ability to make causal inferences from observational datasets.
Morgan, Stephen L., and David J. Harding. 2006. Matching Estimators of Causal Effects: Prospects and Pitfalls in Theory and Practice. Sociological Methods & Research 35 (1): 3-60.
Abstract: As the counterfactual model of causality has increased in popularity, sociologists have returned to matching as a research methodology. In this article, advances over the past two decades in matching estimators are explained, and the practical limitations of matching techniques are emphasized. The authors introduce matching methods by focusing first on ideal scenarios in which stratification and weighting procedures warrant causal inference. Then, they discuss how matching is often undertaken in practice, offering an overview of the most prominent data analysis routines. With four hypothetical examples, they demonstrate how the assumptions behind matching estimators often break down in practice. Even so, the authors argue that matching techniques can be used effectively to strengthen the prosecution of causal questions in sociology.
- Please discuss the uses of matching as a research methodology in the social sciences and as a mode for studying causality.
- Please discuss the relationship between matching and regression, as outlined in the article.
- What are specific benefits and limitations inherent to matching, as discussed in the scenarios outlined by the authors?
Stuart, Elizabeth. 2007. Estimating Causal Effects Using School-Level Data Sets. Educational Researcher 36 (4): 187-198.
Abstract: Education researchers, practitioners, and policymakers alike are committed to identifying interventions that teach students more effectively. Increased emphasis on evaluation and accountability has increased desire for sound evaluations of these interventions; and at the same time, school-level data have become increasingly available. This article shows researchers how to bridge these two trends through careful use of school-level data to estimate the effectiveness of particular interventions. The author provides an overview of common methods for estimating causal effects with school-level data, including randomized experiments, regression analysis, pre–post studies, and nonexperimental comparison group designs. She stresses the importance of careful design of nonexperimental studies, particularly the need to compare units that were similar before treatment assignment. She gives examples of analyses that use school-level data and concludes with advice for researchers.
- Please discuss what opportunities multiple regression analysis provides to educators in order to better understand how interventions affect student learning.
- Please discuss how the three building blocks of treatment and control conditions, units, and potential outcomes are used by the researchers to help estimate the effects of specific interventions on the learning process.
- What are the ramifications (i.e. policy, organizational, administrative) of utilizing datasets to make comparisons across schools to better understand the influence of interventions?