Recommended Readings and SAGE Journal Articles
Referenced Peer-reviewed Journal Articles
Below are the Hyperlinks to the actual journal articles or the home page to journals where the full-text articles can be accessed. Most articles can be accessed through a university’s library database or through the library’s associated inter-library loan system.
Referenced Texts
When available, we provided the home pages of the companion Websites of the texts we referenced in our book. For books that do not have companion Websites we provided the link to homepage of the publisher of the book, which provides more information about how to access or purchase the book.
Abrams, L. S., Shannon, S. K. S., & Sangalang, C. (2008). Transition services for incarcerated youth: A mixed methods evaluation study. Children and Youth Services Review, 30, 522–535.
Acee, T. W., & Weinstein, C. E. (2010). Effects of value-reappraisal intervention on statistics students’ motivation and performance. Journal of Experimental Education, 78(4), 487–512.
We propose a methodological framework for conducting collaborative and analytic autoethnography (CAAE) in action research inquiry. Similar to other action research methodologies, CAAE is: (a) systematic, framed by a research question and employing explicit, transparent methods; (b) problem-based, drawn from health educators’ practices; and (c) cyclical, predicated on the assumption that a plan for implementing changes is derived from the findings and will be tested in subsequent action research cycles. The autoethnographic action research framework comprises four phases: inductive stage/formative assessment—developing a hypothesis (phase 1), pre-deductive stage—organizing the study (phase 2), deductive stage/summative evaluation—collecting data and testing the hypothesis (phase 3), and synthesis stage—constructing meaning (phase 4). Collaborative analytic autoethnography supports reflexivity, awareness, and agency by providing practitioners with an explicit framework for continually evaluating the complexities of self, the Other (i.e. experiencing self from another’s perspective), and context.
Anderson, R. B., Doherty, M. E., & Friedrich, J. C. (2008). Sample size and correlational inference. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(4), 929-944.
Baker, M. (2015). Over half of psychology studies fail reproducibility test. Nature, doi:10.1038/nature.2015.18248.
Basadur, M., Graen, G. B., & Scandura, T. A. (1986). Training effects on attitudes toward divergent thinking among manufacturing engineers. Journal of Applied Psychology, 71(4), 612–617.
Baylor, A. L., & Kim, S. (2009). Designing nonverbal communication for pedagogical agents: When less is more. Computers in Human Behavior, 25, 450–457.
Bazeley, P., & Jackson, K. (2013). Qualitative data analysis with NVivo (2nd ed.) Thousand Oaks, CA: Sage.
Beeke, S., Wilkinson, R., & Maxin, J. (2007). Individual variation in agrammatism: A single case study of the influence of interaction. Journal of Language and Communication Disorders, 42(6), 629–647.
Berman, H., Mulcachy, G. A., Forchuk, C., Edmunds, K. A., Haldenby, A., & Lopez, R. (2009). Uprooted and displaced: A critical narrative study of homeless, aboriginal, and newcomer girls in Canada. Issues in Mental Health Nursing, 30(7), 418–430.
Bernard, R. S., Cohen, L. L., & Moffett, K. (2008). A token economy for exercise adherence in pediatric cystic fibrosis: A single-subject analysis. Journal of Pediatric Psychology, 34(4), 354–365.
Birks, M., Chapman, Y., & Francis, K. (2008). Memoing in qualitative research: Probing data and processes. Journal of Research in Nursing, 13(1), 68–75.
Bohannon, J. (2015). Many psychology papers fail replication test. Science, 349(6251), 910-911.
Bortree, D. S., & Seltzer, T. (2009). Dialogic strategies and outcomes: An analysis of environmental advocacy groups’ Facebook profiles. Public Relations Review, 35(3), 317-310.
Box, G. E. P. (2009). Statistical control by monitoring and adjustment, statistics for experimenters: Design, innovation, and discovery (Wiley Series in Probability and Statistics) (2nd ed). Hoboken, NJ: Wiley.
This study sought to examine the effects of Tier 2 intervention in a multitiered model on the performance of first- and second-grade students who were identified as having mathematics difficulties. A regression discontinuity design was utilized. Participants included 126 (Tier 2, n = 26) first graders and 140 (Tier 2, n = 25) second graders. Tier 2 students received 15-min intervention booster lessons for 18 weeks in early mathematics skills and concepts. Results showed a significant intervention effect for second-grade Tier 2 students on the Texas Early Mathematics Inventories—Progress Monitoring (TEMI-PM) total standard score. The effect was not significant for first-grade Tier 2 students.
Burgess, G., Grogan, S., & Burwitz, L. (2006). Effects of a 6-week aerobic dance intervention on body image and physical self-perceptions in adolescent girls. Body Image, 3, 57–66.
Butler-Kisber, L. (2010). Qualitative inquiry: Thematic, narrative and art-informed perspectives. Thousand Oaks, CA: Sage.
Campbell, D. T. (1957). Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54, 297–312.
Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Skokie, IL: Rand McNally.
Cernin, P. A., & Lichtenberg, P. A. (2009). Behavioral treatment for depressed mood: A pleasant events intervention for seniors residing in assisted living. Clinical Gerontologist, 32, 324–331.
Chakraborti, C., Boonyasai, R. T., Wright, S. M., & Kern, D. E. (2008). A systematic review of teamwork training interventions in medical student and resident education. Journal of General Internal Medicine, 23(6), 846-53.
Chao, P., Bryan, T., Burstein, K., & Ergul, C. (2006). Family-centered intervention for young children at-risk for language and behavior problems. Early Childhood Education Journal, 34(2), 147–153.
This study assessed whether persons with spinal cord injuries who were successfully rehabilitated differed from those who were not with regard to positive and negative affect, life satisfaction, and depression. An ex post facto research design compared persons with spinal cord injuries who were previously employed with persons with spinal cord injuries who were not employed after receiving services from a southeastern state Division of Vocational Rehabilitation Services program. There were no significant differences between the two groups based on age, gender, race, current level of education, and time employed prior to their injury.The unsuccessfully rehabilitated had lower positive affect and higher negative affect, were less satisfied with life and more depressed, and had a higher percentage of cervical and lumbar injuries than the successfully rehabilitated. Positive psychotherapy techniques are recommended to see if they can decrease depression and improve satisfaction with life and positive affect, while decreasing negative affect.
Charmaz, K. (2014). Constructing grounded theory: A practical guide through qualitative analysis (2nd ed). London, England: Sage.
Chenail, Ronald (1992). Presenting qualitative data. The Qualitative Report, 2(3).
Chenail, R. (2010). Getting specific about qualitative research generalizability. Journal of Ethnographic & Qualitative Research, 5, 1–11.
Chevalier, J. M., & Buckles, D. J. (2013). Participatory action research: Theory and methods for engaged
inquiry. London, England: Routledge.
Clandinin, D. J., & Connelly, F. M. (2004). Narrative inquiry: Experience and story in qualitative research. San Francisco, CA: Wiley.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erbaum.
Collins, K. M. T., & O’Cathain, A. (2009). Ten points about mixed methods research to be considered by the novice researcher. International Journal of Multiple Research Approaches, 3(1), 2–7.
Comaskey, E. M., Savage, R. S., & Abrami, P. (2009). A randomized efficacy study of web-based synthetic and analytic programmes among disadvantaged urban kindergarten children. Journal of Research in Reading, 32(1), 92–108.
Cook, T. D., & Campbell, D. T. (1979). Quasi-Experimentation: Design and analysis issues for field settings. Chicago, IL: Rand McNally.
Cook, T. D., & Steiner, P. M. (2010). Case matching and the reduction of selection bias in quasi-experiments: The relative importance of pretest measures of outcome, of unreliable measurement, and of mode of data analysis. Psychological Methods, 15(1), 56–68.
Coppola, L., Mastrolorenzo, L., Coppola, A., De Biase, M., Adamo, G., Forte, R., . . . Riccardi, A. (2013). QT dispersion in mild cognitive impairment: A possible tool for predicting the risk of progression to dementia? International Journal of Geriatric Psychiatry, 28(6), 632-639.
Corbin, J., & Strauss, A. (2015). Basics of qualitative research: Techniques and procedures for developing grounded theory (4th ed.). Thousand Oaks, CA: Sage.
Creswell, J. W. (2012). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.). Thousand Oaks, CA: Sage.
Creswell, J. W. (2013). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Thousand Oaks, CA: Sage.
Creswell, J. W. (2014). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (5th ed.). Upper Saddle River, NJ: Pearson.
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.
Cribbie, R. A., Arpin-Cribbie, C. A., & Gruman, J. A. (2010). Tests of equivalence for one-way independent groups designs. The Journal of Experimental Education, 78, 1–13.
Dasgupta, T., Pillai, N. S., & Rubin, D. B. (2014). Causal inference from 2k factorial designs using potential outcomes. Journal of the Statistical Society: Series B (Statistical Methodology), 77(4), 727-753.
Dennis, J. K. (2003). Problem-based learning in online vs. face-to-face environments. Education for Health, 16(2), 198–209.
DeVellis, R. F. (2011). Scale development: Theory and applications (3rd ed.). Thousand Oaks, CA: Sage.
Dixon, M. R., Jackson, J. W., Small, S. L., Horner-King, M. J., Lik, N. M. K, Garcia, Y., & Rosales, R. (2009). Creating single-subject design graphs in Microsoft Excel™ 2007. Journal of Applied Behavior Analysis, 42(2), 277–293.
Multiple imputation is widely accepted as the method of choice to address item-nonresponse in surveys. However, research on imputation strategies for the hierarchical structures that are typically found in the data in educational contexts is still limited. While a multilevel imputation model should be preferred from a theoretical point of view if the analysis model of interest is also a multilevel model, many practitioners prefer a fixed effects imputation model with dummies for the clusters since these models are easy to set up with standard imputation software. In this article, we theoretically and empirically evaluate the impacts of this simplified approach. We illustrate that the cluster effects that are often of central interest in educational research can be biased if a fixed effects imputation model is used. We show that the potential bias depends on three quantities: the amount of missingness, the intraclass correlation, and the cluster size. We argue that the bias for the random effects can be substantial while the bias for the fixed effects will be negligible in most real-data situations. We further illustrate this with an application using data from the German National Educational Panel Survey.
Dugard, P., File, P., & Todman, J. B., (2001). Single-case and small-n experimental designs: A practical guide to randomization tests (2nd ed). Mahwah, NJ: Lawrence Erlbaum Associates.
Elliott, M. S., & Williams, D. I. (2002). A qualitative evaluation of an employee counselling service from the perspective of client, counselor, and organization. Counselling Psychology Quarterly, 15(2), 201–208.
Erdogan, Y., Aydin, E., & Kabaca, T. (2008). Exploring the psychological predictors of programming achievement. Journal of Instructional Psychology, 35(3), 264–270.
Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavioral Research Methods, 39(2), 175-191.
Fetterman, D. M. (2009). Ethnography: Step-by-step (3rd ed.). Thousand Oaks, CA: Sage.
Finchman, J. E. (2008). Response rates and responsiveness for surveys, standards, and the journal. American Journal of Pharmaceutical Education, 72(2), 1-3.
Fink, A. G. (2012). How to conduct surveys: A step-by-step guide (5th ed.). Thousand Oaks, CA: Sage.
Forchuk, C., Norman, R., Malla, A., Martin, M., McLean, T., & Cheng, S., . . . Gibney, C. (2002). Schizophrenia and the motivation for smoking. Perspectives in Psychiatric Care, 38(2), 41–49.
Fowler, F. J. (2013). Survey research methods (Applied Social Research Methods (5th ed.). Thousand Oaks, CA: Sage.
Gall, M. D., Gall, J. P., & Borg, W. R. (2007). Educational research: An introduction (8th ed.). Boston, MA: Pearson.
Many children diagnosed with autism spectrum disorder (ASD) exhibit difficulties with complex language and social communication. Direct Instruction (DI) is an empirically supported curriculum designed to teach complex language skills to children with and at risk of learning disabilities. Only recently, the effectiveness of DI has been evaluated among children with autism. The present study evaluated the effectiveness of the DILanguage for Learning curriculum among 18 children diagnosed with ASD. Immediate post-intervention language scores on curriculum post-tests were significantly higher than pre-intervention scores and remained significantly higher than pre-intervention scores up to 6 to 8 months following the intervention. Comparing language skills across groups, children already exposed to the intervention exhibited significantly higher language skills than their non-exposed waitlist counterparts.
Gast, D. L., & Ledford, J. R. (2014). Single case research methodology: Applications in special education and behavioral sciences (2nd ed.). London, England: Routledge.
Geertz, C. (1998). The world in pieces: Cultural politics at the end of the century. FOCAAL: European Journal of Anthropology, 32, 91–117.
Gier, V. S., Kreiner, D. S., & Natz-Gonzalez, A. (2009). Harmful effects of preexisting inappropriate highlighting on reading comprehension and metacognitive accuracy. The Journal of General Psychology, 136(3), 287–300.
Giuffre, M. (1997). Designing research: Ex post facto designs. Journal of PeriAnesthesia Nursing, 12(3), 191–195.
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Hawthorne, NY: Aldine Transaction.
Glass, V. G., Wilson, V. L., & Gottman, J. M. (2008). Design and analysis of time-series experiments. Charlotte, NC: Information Age Publishing.
Glombiewski, J. A., Sawyer, A. T., Gutermann, J., Koenig, K., Rief, W., & Hofmann, S. G. (2010). Psychological treatments for fibromyalgia: A meta-analysis. Pain, 151(2), 280-295.
Golafshani, N. (2003). Understanding reliability and validity in qualitative research. The Qualitative Report, 8(4), 597–607.
Green, M., & Cifuentes, L. (2008). An exploration of online environments supporting follow-up to face-to-face professional development. Journal of Technology and Teacher Education, 16(3), 283–306.
Green, S., & Salkind, N. J. (2013). Using SPSS for Windows and Macintosh (7th ed.). Upper Saddle River, NJ: Prentice Hall.
Guadagno, R. E., & Sagarin, B. J. (2010). Sex differences in jealousy: An evolutionary perspective on online infidelity. Journal of Applied Social Psychology, 40(10), 2636–2655.
Guba, E. G. (1981). Criteria for assessing the trustworthiness of naturalistic inquiries.Educational Communication and Technology Journal 29, 75–91.
Guyz, D., Dickson-Swift, V., Kenny, A., & Threlkeld, G. (2015). Gadamerian philosophical hermeneutics as a useful methodological framework for the Delphi technique. International Journal of Qualitative Studies on Health and Well-Being, 10, 1-13.
Hall-Kenyon, K. M., Bingham, G. E., & Korth, B. B. (2009). How do linguistically diverse students fare in full- and half-day kindergarten? Examining academic achievement, instructional quality, and attendance. Early Education and Development, 20, 25–52.
Han, G. S., & Davies, C. (2006). Ethnicity, health and medical care: Towards a critical realist analysis of general practice in the Korean community in Sydney. Ethnicity and Health, 11(4), 409–430.
Hannan, M., Happ, M. B., & Charron-Prochownik, D. (2009). Mothers’ perspectives about reproductive health discussion with adolescent daughters with diabetes. The Diabetes Educator, 35(2), 265–273.
The Hare Psychopathy Checklist (PCL-R) is an important predictor of violent behavior and is widely used to make important decisions about forensic clients. Some research casts doubt on whether the scoring of the PCL-R in clinical practice matches that attained in research and, therefore, whether the use of the PCL-R is warranted in high-stakes decisions. We examined scoring correspondence of the PCL-R in 58 offenders where scoring by trained clinicians was compared with that by a very experienced researcher whose scoring was of known predictive validity (or with a student supervised by this experienced researcher). Research and clinical scorers showed good agreement (Spearman’s rank order correlation = .85; intraclass correlation coefficient = .79, absolute agreement for single measures), especially on those parts of the PCL-R that are most consistently and robustly associated with violence. We conclude that trained clinicians can achieve acceptable reliability and validity when scoring the PCL-R, especially for risk assessment.
Haydon, T., Mancil, G. R., & Van Loan, C. (2009). Using opportunities to respond in a general education classroom: A case study. Education and Treatment of Children, 32(2), 267–278.
Hedayat, A. S., Stufken, J., & Yang, M. (2006). Optimal and efficient crossover designs when subject effects are random. Journal of the American Statistical Association, 101, 1031–1038.
Helmer, O. (1967). Analysis of the future: The Delphi method (No. RAND-P-3558). Santa Monica, CA: The RAND Corporation.
Hine, C. (2015). Ethnography for the internet: Embedded, embodied and everyday. New York, NY: Bloomsbury Academic.
Hinkelman, K. (2012). Design and analysis of experiments (Volume 3): Special designs and applications. Hoboken, NJ: Wiley.
Hinnant, J. B., O’Brien, M., & Ghazarian, S. R. (2009). The longitudinal relations of teacher expectations to achievement in early school years. Journal of Educational Psychology, 101(3), 662–670.
Hirakata, P. (2009). Narratives of dissociation: Insights into the treatment of dissociation in individuals who were sexually abused as children. Journal of Trauma & Dissociation, 10(1), 297–314.
Hlavinkova, L., Mentel, A., Kollarova, J., & Kristufkova, Z. (2014). Effectiveness of a prevention campaign on HIVAIDS knowledge among adolescents in Eastern Slovakia. International Journal of Public Health, 59(6), 905-911.
Hofer, M., Burkhard, L., & Allemand, M. (2015). Age differences in emotion regulation during a distressing film scene. Journal of Media Psychology: Theories, Methods, and Applications, 27(2), 47-52.
Imbens, G. W., & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142, 615-–635.
Imbens, G. W., & Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge, UK: Cambridge University Press.
Jaccard, J., & Jacoby, J. (2010). Theory construction and model-building skills: A practical guide for social scientists. New York, NY: Guilford Press.
Jones, M. A., Stratten, G., Reilly, T., & Unnithan, V. B. (2004). A school-based survey of recurrent non-specific low-back pain prevalence and consequences in children. Health Education Research, 19(3), 284–289.
Jones, S. R., & Hill, K. E. (2003). Understanding patterns of commitment: Student motivation for community service involvement. Journal of Higher Education, 74(5), 516–539.
Kaldjian, L. C., Jones, E. W., Rosenthal, G. E., Tripp-Reimer, T., & Hillis, S. L. (2006). An empirically derived taxonomy of factors affecting physicians’ willingness to disclose medical errors. Journal of General Internal Medicine, 21, 942–948.
Kameoka, S., Yagi, J., Arai, Y., Nosaka, S., Saito, A., Miyake, W.,… Asukai, N. (2015). Feasibility of trauma-focused cognitive behavioral therapy for traumatized children in japan: A pilot study. International Journal of Mental Health Systems, 9.
Kartalova-O’Doherty, Y., & Doherty, D. T. (2008). Coping strategies and styles of family carers of persons with enduring mental illness: A mixed methods analysis. Scandinavian Journal of Caring Sciences, 22(1), 19–28.
Kazdin, A. E., Esveldt-Dawson, K., French, N. H., & Unis, A. S. (1987). Problem-solving skills training and relationship therapy in the treatment of antisocial child behavior. Journal of Counseling and Clinical Psychology, 55(1), 76–85.
Keeney, S., McKenna, H., & Hasson, F. (2011). The Delphi technique in nursing and health research. Hoboken: Wiley.
Keith, T. Z. (2014). Multiple regression and beyond (2nd ed.). Boston, MA: Pearson.
Kemp, A., Quintana, D. S., Quinn, C., Hopkinson, P., & Harris, A. (2014). Major depressive disorder with melancholia displays robust alterations in resting state heart rate and variability: Implications for future morbidity and mortality. Frontiers in Psychology, 5(1387).
Kennedy, T. D., Burnett, K. F., & Edmonds, W. A. (2011). Intellectual, behavioral, and personality correlates of violent vs. non-violent juvenile offenders. Aggressive Behavior, 37(4), 315-325.
Kline, R. B. (2010). Principles and practice of structural equation modeling (3rd ed.). New York, NY. Guilford Press.
Knapp, T. R., & Schafer, W. D. (2009). From gain score t to ANOVA F (and vice versa). Practical Assessment, Research & Evaluation, 14(6), 1-7.
Kraemer, H. C., & Blasey, C. (2015). How many subjects?: Statistical power analysis in research (2nd ed.). Thousand Oaks, CA: Sage.
The purpose of this study was to investigate the effects of four instructional methods on students’ mathematical reasoning and metacognitive knowledge. The participants were 384 eighth-grade students. The instructional methods were cooperative learning combined with metacognitive training (COOP+META), individualized learning combined with metacognitive training (IND+META), cooperative learning without metacognitive training (COOP), and individualized learning without metacognitive training (IND). Results showed that the COOP+META group significantly outperformed the IND +META group, which in turn significantly outperformed the COOP and IND groups on graph interpretation and various aspects of mathematical explanations. Furthermore, the metacognitive groups (COOP+META and IND +META) outperformed their counterparts (COOP and IND) on graph construction (transfer tasks) and metacognitive knowledge. This article presents theoretical and practical implications of the findings.
Lachin, J. M., Matts, J. P., & Wei, L. J. (1988). Randomization in clinical trials: Conclusions and recommendations. Controlled Clinical Trials, 9, 365–374.
Lapadat, J. C. (2004). Autobiographical memories of early language and literacy development. Narrative Inquiry, 14(1), 113–140.
Lavrakas, P. J. (Ed.). (2009). Encyclopedia of survey research methods. Thousand Oaks, CA: Sage.
Leake, M., & Lesik, S. A. (2007). Do remedial English programs impact first-year success in college? An illustration of the regression-discontinuity design. International Journal of Research & Method in Education, 30(1), 89–99.
Minimal research has been conducted contrasting the effectiveness of various testing accommodations for college students diagnosed with ADHD. The current assumption is that these students are best served by extending the time they have to take a test. It is the supposition of these investigators that paced item presentation may be a more beneficial accommodation than extended time. To test the effects of paced item presentation, the investigators designed a mixed methods sequential explanatory study to explore the relationship between computer-paced and student-paced item presentation on the academic test performance in college students diagnosed with ADHD. The participants were randomly assigned to 1 of 2 testing conditions. Half of the participants were provided a computer-paced testing condition, and half were provided a student-paced testing condition within a computer-based environment. Interviews were conducted after the test administration to discern the students' perceptions of the value of the various components of the testing environment. No significant differences were found in performance scores between the students tested under the two conditions; however, the interview data illuminated the quantitative findings in that the students reported that the computer-based testing environment itself, as well as other environmental variables, provided a beneficial structure and format conducive to their overall successful performance under both accommodations. The practicability of university disability offices offering a computerized format for students diagnosed with ADHD is also discussed.
Lee, S., Paek, J., Han, S., & Ryu, H. (in press). The utility of a Korean version of the rem sleep behavior disorder screening questionnaire in patients with obstructive sleep apnea. Journal of the Neurological Sciences.
Lee, Y., Park, S., Kim, M., Son, C., & Lee, M. (2005). The effects of visual illustrations on learners’ achievement and interest in PDA- (personal digital assistant) based learning. Journal of Educational Computing Research, 33(2), 173–187.
Lee, J. J., & Rubin, D. B. (2015). Valid randomization-based p-values for partially post hoc subgroup analyses. Statistics in Medicine. DOI: 10.1002/sim.6531
Leech, N. L., Barrett, K. C., & Morgan, G. A. (2014). SPSS for introductory and intermediate statistics: Use and interpretation (7th ed.). London, England: Routledge.
Lesik, S. A. (2006). Applying the regression-discontinuity design to infer causality with non-random assignment. The Review of Higher Education, 30(1), 1–19.
Lesik, S. A. (2008). Studying the effectiveness of programs and initiatives in higher education using the regression-discontinuity design. Handbook of Theory and Research, 23, 277–297.
Levy, P. S., & Lemeshow, S. (2009). Sampling of populations: Methods and applications (4th ed.). New York, NY: Wiley.
Lieblich, A., Tuval-Mashiach, R., & Zilber, T. (1998). Narrative research: Reading, analysis, and interpretation. Thousand Oaks, CA: Sage..
Lincoln, Y. S., & Guba, E. G. (2013). The constructivist credo. Walnut Creek, CA: Left Coast Press.
Lincoln, Y. S., Linham, S. A., & Guba, E. G. (2011). Paradigmatic controversies, contradictions, and emerging influences., revisted. In N. K. Denzin & Y. S. Lincoln (Eds.), The sage handbook of qualitative research (4th ed.), pp. 191–215. Thousand Oaks, CA: Sage.
Linden, A., Trochim, W. M. K., & Adams, J. L. (2006). Evaluating program effectiveness using the regression point displacement design. Evaluation & the Health Professions, 29(4), 407–423.
Loh, J. (2013). Inquiry into issues of trustworthiness and quality in narrative studies: A perspective. The Qualitative Report, 18(65), 1 - 15.
Lopez, E. N., Drobes, D. J., Thompson, K. J., & Brandon, T. H. (2008). Effects of body image challenge on smoking motivation among college females. Health Psychology, 27(3), 243–251.
Luytena, H., Tymms, P., & Jones, P. (2009). Assessing school effects without controlling for prior achievement? School Effectiveness and School Improvement, 20(2), 145–165.
Lyon, A. R., Gershenson, R. A., Farahmand, F. K., Thaxter, P. J., Behling, S., & Budd, K. S. (2009). Effectiveness of teacher-child interaction training (TCIT) in a preschool setting. Behavior Modification, 33(6), 855–884.
Madison, D. S. (2011). Critical ethnography: Methods, ethics, and performance (2nd ed.). Thousand Oaks, CA: Sage.
Maguire, P. (1987). Doing participatory research: A feminist approach. Amherst, MA: Center for International Education.
Makagon, D., & Neumann, M. (2008). Recording culture. Thousand Oaks, CA: Sage.
Malcolm, T. (2010). The curious case of case study: A viewpoint. International Journal of Social Research Methodology: Theory and Practice, 13(4), 329–339.
Mancil, G. R., Haydon, T., & Whitby, P. (2009). Differential effects of paper and computer-assisted Social Stories™ on inappropriate behavior in children with autism. Focus on Autism and Other Developmental Disabilities, 24(4), 205–215.
Manley, R. A., & Zinser, R. (2012). A Delphi study to update CTE teacher competencies. Education & Training, 54(6), 488-503.
Manolov, R., Solanas, A., Bulté, I., & Onghena, P. (2010). Data-division-specific robustness and power of randomization tests for ABAB designs. Journal of Experimental Education, 78, 191–214.
This article presents the results of an 11-year longitudinal study of the impact of America’s Choice comprehensive school reform (CSR) design on student learning gains in Rochester, New York. A quasi-experimental interrupted time-series approach using Bayesian hierarchical growth curve analysis with crossed random effects is used to compare the annual gains in test performance of students attending America’s Choice schools to those of students attending other Rochester schools and to those of students attending America’s Choice schools before they adopted this CSR model. Findings reveal significant annual effects, which accumulate over time, in elementary and middle grades reading and mathematics.
McAdams, D. P., Josselson, R., & Lieblich, A. (2006). Identity and story: Creating self in narrative. Washington, DC: American Psychological Association.
Increasingly, faculty members are incorporating service-learning projects into their courses, resulting in enhanced student learning and development. However, does service learning encourage subsequent student involvement in the community? This study investigates the effect faculty professionals can have through service learning on their students’intentions to participate in community service. A modified version of the Solomon four-group design analysis as well as a hierarchical regression were used to assess the change in student intentions after exposure to three service-learning treatments. Findings indicate that service learning has a significant impact on students’intentions to participate in community service. In particular, by adding a lecture to the standard service-learning format, faculty members can increase student intentions to participate in community service.
McNemar, Q. (2007). On the use of Latin squares in psychology. Psychological Bulletin, 48(5), 398–401.
Meijer, J., & Oostdam, R. (2007). Test anxiety and intelligence testing: A closer examination of the stage-fright hypothesis and the influence of stressful instruction. Anxiety, Stress, and Coping, 20(1), 77–91.
Several approaches exist for organizing and analyzing data in a phenomenological qualitative study. Transcendental phenomenology, based on principles identified byHusserl (1931) and translated into a qualitative method by Moustakas (1994), holds promise as a viable procedure for phenomenological research. However, to best understand the approach to transcendental phenomenology, the procedures need to be illustrated by a qualitative study that employs this approach. This article first discusses the procedures for organizing and analyzing data according to Moustakas (1994). Then it illustrates each step in the data analysis procedure of transcendental phenomenology using a study of reinvestment or the “ripple effect” for nine individuals who have participated in a youth leadership mentoring program from the 1970s to the present. Transcendental phenomenology works well for this study as this methodology provides logical, systematic, and coherent design elements that lead to an essential description of the experience.
Morgan, B. J. (2001). Evaluation of an educational intervention for military tobacco users. Military Medicine, 166(12), 1094–1098.
Morse, J. M. (1991). Approaches to qualitative-quantitative methodological triangulation. Nursing Research, 40, 120–123.
Morse, J. M., & Niehaus, L. (2009). Mixed methods design: Principles and procedures. Walnut Creek, CA: Left Coast Press.
Meyers, L. S., Gamst, G., & Guarino, A. J. (2012). Applied multivariate research: Design and interpretation (2nd ed.). Thousand Oaks, CA: Sage.
Mundia, L. (2011). Social desirability, non-response bias and reliability in long self-report measure: Illustrations from the MMPI-2 administered to Brunei student teachers. Educational Psychology, 31(2), 207-224.
Nagel, T., Robinson, G., Condon, J., & Trauer, T. (2009). Approach to treatment of mental illness and substance dependence in remote indigenous communities: Results of a mixed methods study. Australian Journal of Rural Health, 17(4), 174–182.
Nastasi, B. K., Hitchcock, J., Sarkar, S., Burkholder, G., Varjas, K., & Jayasena, A. (2007). Mixed methods in intervention research: Theory to adaptation. Journal of Mixed Methods Research, 1(2), 164–183.
O’Brien, R. G., & Kaiser, M. K. (1985). MANOVA method for analyzing repeated measures designs: An extensive primer. Psychological Bulletin, 97(2), 316-333.
Pace, L. A. (2011). Statistical analysis using Excel 2007. Upper Saddle River, NJ: Prentice Hall.
Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.
Palinkas, L. A., Horwitz, S. A., Green, C. A., Wisdom, J. P., Duan, N., & Hoagwood, K. (2014). Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administrative Policy and Mental Health. DOI 10.1007/s10488-013-0528-y
Park, Y. S., Riddle, J., & Tekian, A. (2014). Validity evidence of resident competency ratings and the identification of problem residents. Medical Education, 48(6), 614-622.
Perilloux, C., Cloud, J. M., & Buss, D. M. (2013). Women’s physical attractiveness and short-term mating strategies. Personality and Individual Differences, 54, 490-495.
Plano Clark, V. L. (2005). Cross-disciplinary analysis of the use of mixed methods in physics education research, counseling psychology, and primary care. Dissertation Abstracts International, 66, 02A.
Polkinghorne, D. E. (1988). Narrative knowing and the human sciences. Albany: State University of New York Press.
Pollio, H. R., Henley, T., & Thompson, C. B. (1997). The phenomenology of everyday life. Cambridge, UK: Cambridge University Press.
Prinzie, P., & Onghena, P. (2005). Cohort sequential design. In B. S. Everrit & D. Howell (Eds.), Encyclopedia of statistics in behavioral science (pp. 319–322). West Sussex, UK: Wiley-Blackwell.
Previous research on the effects of organizational restructuring has primarily been conducted using self-report cross-sectional data. These forms of data are vulnerable to several threats to both internal and external validity. Using a Solomon four-group design, this study examined the effects of an organizational restructuring while simultaneously testing for the effects of pretesting, history, and interactions between pretesting and the organizational restructuring. Results suggest the organizational restructuring had consistent negative effects on employee levels of job security, organizational commitment, perceptions of time pressure, psychological well-being, and turnover intentions. Effects on job satisfaction, physical health, and perceptions of role ambiguity were less consistently noted. Theoretical and practical implications of the results are discussed and directions for future research are proposed.
Drawing on interviews and comparative ethnographic fieldwork in two day labor hiring sites (a street corner labor market and a “regulated” day labor worker center), this article examines the discourses through which Latino immigrant day laborers make sense of, and find dignity within, their ongoing quest for work. My findings reveal a clear pattern of “boundary work” along the center/street divide, wherein each group of day laborers asserts its dignity and masculinity by repudiating what they construe to be the feminine submission exemplified by the other group. I argue that gender both shapes and is shaped through the articulation of these moral boundaries and show how workers' struggle to attain dignity—in this case, via strategies of social differentiation and distinction—can act against the formation of a collective identity.
Ragin, C. C. (1997). Turning the tables: How case-oriented research challenges variable-oriented research. Comparative Social Research, 16, 27–42.
Randall, R. R., & Barcikowski, R. S. (1987). Sphericity tests and repeated measures data. Paper presented at the annual meeting of the American Educational Research Association, Washington, D. C., April.
Reese, H. W. (1997). Counterbalancing and other uses of repeated-measures Latin-square designs: Analysis and interpretations. Journal of Experimental Child Psychology, 64, 137–158.
Reynolds, L. M., McCambridge, S. A., Bissett, I. P., & Consedine, N. S. (2014). Trait and state disgust: An experimental investigation of disgust and avoidance in colorectal cancer decision scenarios. Health Psychology, 33(12), 1495-1506.
This article examines the role of the African American uncle as a vital yet overlooked form of social support and social capital in the lives of adolescent African American male sons living in single-female-headed households. Research rarely examines the affective roles and functions of men in Black families; moreover, poor urban Black male youth are typically portrayed as a monolithic and homogeneous group who lack positive relationships with their biological fathers. The absence of these relationships has been correlated to numerous social problems for Black male youth—specifically, delinquency and violent behavior. Although much of the work on African American fatherhood has focused on the role of the biological father (and, to some extent, the stepfather), minimal attention has been given to men within extended familial networks and their impact on successful adolescent development among young African American males.
Riessman, C. K. (2007). Narrative methods for the human sciences. Thousand Oaks, CA: Sage.
Rihoux, B., & Lobe, B. (2009). The case for qualitative comparative analysis (QCA): Adding leverage for thick cross-case comparison. In D. Byrne & C. C. Ragin (Eds.), The Sage handbook of case-based methods (pp. 222–242). Thousand Oaks, CA: Sage.
Rimondini, M., Del Piccolo, L., Goss, C., Mazzi, M., Paccaloni, M., & Zimmerman, C. (2010). The evaluation of training in patient-centered interviewing skills for psychiatric residents. Psychological Medicine, 40, 467–476.
Rubin, D. B. (2006). Matched sampling for causal effects. Cambridge, UK: Cambridge University Press.
Rubin, D. B. (2007). The design versus the analysis of observational studies for causal effects: Parallels with the design of randomized trials. Statistics in Medicine, 26(1), 20–36.
Rubin, D. B. (2008). For objective causal inference, design trumps analysis. The Annals of Applied Statistics, 2(3), 808–840.
Ryan, T. P. (2007). Modern experimental design. Hoboken, NJ: Wiley.
Ryndak, D. L., Storch, J. F., & Hoppey, D. (2008). One family’s perspective of their experiences with school and district personnel over time related to inclusive educational services for a family member with significant disabilities. International Journal of Whole Schooling, 4(2), 29–51.
Sawilowsky, S., Kelley, D. L., Blair, R. C., & Markman, B. S. (1994). Meta-analysis and the Solomon four-group design. Journal of Experimental Education, 62, 361–376.
Schoenfeld, N. A., & Mathur, S. R. (2009). Effects of cognitive-behavioral intervention on the school performance of students with emotional or behavioral disorders and anxiety. Behavioral Disorders, 34(4), 184–195.
Schultz, K. F., Altman, D. G., & Moher, D. (2010). CONSORT 2010 statement: updated guidelines for
reporting parallel group randomised trials. Research Methods & Reporting, 340, 698-702.
This study details the collection, analysis, and interpretation of data from a national multi-institutional longitudinal mixed methods study of college impact and student development of liberal arts outcomes. The authors found three sets of practices in the quantitative data that corroborated with the themes that emerged from the qualitative data: academic challenge, diversity experiences, and supportive relationships aided the transition in the first year. The authors discuss these relationships and their mechanisms for fostering student learning in the first year of college.
Shadish, W. R. (2010). Campbell and Rubin: A primer and comparison of their approaches to causal inference in field settings. Psychological Methods, 15, 3–17.
Shadish, W. R. (2014a). Analysis and meta-analysis of single-case designs: An introduction. Journal of School Psychology, 52, 109-122
Shadish, W.R. (2014b). Statistical analyses of single-case designs: The shape of things to come. Current Directions in Psychological Science, 23, 139-146.
Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random to nonrandom assignment. Journal of the American Statistical Association, 103, 1334–1344.
Shadish, W. R., & Cook, T. D. (2009). The renaissance of field experimentation in evaluating interventions. The Annual Review of Psychology, 60, 607–629.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
In this article, the author demonstrates how a phenomenological approach may be used to describe the changes of women's bodies at menopause. In Korea, the middle age of Korean women has been referred as the golden age and autumn season, and which means Korean thing highly of internal mature rather than external facts. When Korean women experience menopausal, they feel they are in the middle of storm, and just have passed by in the time. They can expert andfeel tomorrow's weather by the changes of their body. While the experience menopausal, they canextremely perceive changes of their body, and at the same time, incidence rate of breast cancer is no the increase. Mastectomy has serious effects on both women's body and spirit.Phenomenological analysis on the changes of body shows that the middle age women of Korea feel "Evidently visible scratches of years", "My body is a live weather forecaster", "Khi is getting weaker", "Being fearful suddenly but too late", "The more aged, the more precious life becomes","Being comfortable is better than being beautiful", "Wish to go back to their past", "Intense desire for re- challenging life", "Better friends than husband". In conclusion, there of no way to fully understand someone without knowing one's lived experience within the cultural context. To understand artificial changes by surgery as well as natural ones, and even to do holistic nursing, qualitative research is essentially required.
Sidani, Y. M. (2007). Perceptions of leader transformational ability: The role of leader speech and follower self-esteem. Journal of Management Development, 26(8), 710–722.
Singer, J. A. (1997). Message in a bottle: Stories of men and addiction. New York, NY: Free Press.
Singer, J. A., & Bonalume, L. (2010a). Toward the scientific study of autobiographical memory narratives in psychotherapy. Pragmatic Case Studies in Psychotherapy, 6, 215–222.
Singer, J. A., & Bonalume, L. (2010b). Autobiographical memory narratives in psychotherapy: A coding system applied to the case of Cynthia. Pragmatic Case Studies in Psychotherapy, 6, 134–188.
Smith, J. A., Flowers, P., & Larkin, M. (2009). Interpretive phenomenological analysis: Theory, method, and research. Thousand Oaks, CA: Sage.
Smith, M. E. (2007). Self-deception among men who are mandated to attend a batterer intervention program. Perspectives in Psychiatric Care, 43(4), 193–203.
Solomon, R. L. (1949). An extension of the control group design. Psychological Bulletin, 46, 137–150.
Spector, P. E. (1981). Research designs. Beverly Hills, CA: Sage.
Spector, P. E. (1997). Job satisfaction: Application, assessment, causes, and consequences. Thousand Oaks, CA: Sage.
Spradley, J. P. (1979). The ethnographic interview. New York, NY: Holt, Rinehart & Winston.
Spradley, J. P. (1980). Participant observation. New York, NY: Holt, Rinehart & Winston.
Steinfield, C., Ellison, N. B., & Lampe, C. (2008). Social capital, self-esteem, and use of online social network sites: A longitudinal analysis. Journal of Applied Developmental Psychology, 29, 434–445.
Stern, S. E., Mullennix, J. W., & Wilson, S. J. (2002). Effects of perceived disability on persuasiveness of computer-synthesized speech. Journal of Applied Psychology, 87(2), 411–417.
Stewart, A. L., King, A. C., & Haskell, W. L. (1993). Endurance exercise and health-related quality of life in 50-65 year-old adults. The Gerontologist, 33(6), 782–789.
Steyn, R. (2009). Re-designing the Solomon four-group: Can we improve on this exemplary model? Design Principles and Practices: An International Journal, 3(1), 1833–1874.
Stringer, E. T. (2013). Action research (4th ed). Thousand Oaks, CA: Sage.
Stufflebeam, D. L., & Coryn, C. L. S. (2014). Evaluation theory, models, & applications (2nd ed). San Francisco, CA: Jossey-Bass.
Tashakkori, A., & Teddlie, C. B. (2002). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage.
Tashakkori, A., & Teddlie, C. B. (2010a). SAGE handbook of mixed methods in social & behavioral research (2nd ed.). Thousand Oaks, CA: Sage.
Tashakkori, A., & Teddlie, C. B. (2010b). Putting the human back into “human research methodology”: The researcher in mixed methods research. Journal of Mixed Methods Research, 4(4), 271–277.
Tate, R., Perdices, M., Rosenkoetter, U., McDonald, S., Togher, L., Shadish, W. R., Horner, R., Kratochwill, T., Barlow, D., Kazdin, A., Sampson, M., Shamseer, L., & Vohra, S. (in preparation). The Single-Case Reporting Guideline in Behavioural Interventions (SCRIBE): Explanation and Elaboration.
Thomas, J. (1993). Doing critical ethnography. Newbury Park, CA: Sage.
Thompson, K. J., & Oliver, B. E. (2012). Response rates in business surveys: Going beyond the usual performance measure. Journal of Official Statistics, 28(2), 221-237,
Trochim, W. (2001). Regression-discontinuity design. In N. J. Smelser, J. D. Wright, & P. B. Baltes (Eds.), International encyclopedia of the social and behavioral sciences (Vol. 19, pp. 12940-–12945). North-Holland, Amsterdam: Pergamon.
Trochim, W., & Campbell, D. T. (1996). The regression point displacement design for evaluating community-based pilot programs and demonstration projects. Unpublished manuscript. Retrieved from http://www.socialresearchmethods.net/research/RPD/RPD.pdf
Trochim, W., & Cappelleri, J. C. (1992). Cutoff assignment strategies for enhancing randomized clinical trials. Controlled Clinical Trials, 13, 190–212.
Vagle, M. D. (2014). Crafting phenomenological research. Walnut Creek, CA: Left Coast Press.
van Maanen, J. (1988). Tales of the field: On writing ethnography. Chicago, IL: University of Chicago Press.
van Manen, M. (1990). Researching lived experience: Human science for an action sensitive pedagogy. Albany: The State University of New York.
van Manen, M. (2014). Phenomenology of practice: Meaning-giving methods in phenomenological research and writing. Walnut Creek, CA: Left Coast Press.
Varcoe, C., Browne, A. J., Wong, S., & Smye, V. L. (2009). Harms and benefits: Collecting ethnicity data in a clinical context. Social Science & Medicine, 68, 1659–1666.
Villalta-Gil, V., Roca, M., Gonzales, N., Domènec, E., Cuca, Escanilla, A., . . . Haro, J. M. (2009). Dog-assisted therapy in the treatment of chronic schizophrenia inpatients. Antrozoös, 22(2), 149–159.
Viswanathan, M. (2005). Measurement error and research design. Thousand Oaks, CA: Sage.
Vogt, W. P. (2005). Dictionary of statistics and methodology: A non-technical guide for the social sciences (3rd ed.). Thousand Oaks, CA: Sage.
von Eckartsberg, R. (1997). Introducing existential-phenomenological psychology. In R. Valle (Ed.), Phenomenological inquiry in psychology: Existential and transpersonal dimensions (pp. 3–20). New York, NY: Plenum.
Walker, C. O., & Greene, B. A. (2009). The relations between student motivational beliefs and cognitive engagement in high school. Journal of Educational Research, 102(6), 463–471.
Walton Braver, M. C., & Braver, S. L. (1988). Statistical treatment of the Solomon four-group design: A meta-analytic approach. Psychological Bulletin, 104(1), 150–154.
Wertz, F. J., Charmaz, K., McMullen, L. M., Josselson, R., Anderson, R., & Emalinda, M. (2011). Five ways of doing qualitative analysis: Phenomenological psychology, grounded theory, discourse analysis, narrative research, and intuitive inquiry. New York, NY: Guilford Press.
White, P. (2009). Developing research questions: A guide for social scientists. New York, NY: Palgrave Macmillan.
Whitman, D. S., Van Rooy, D. L., Viswesvaran, C., & Alonso, A. (2008). The susceptibility of a mixed model measure of emotional intelligence to faking: A Solomon four-group design. Psychology Science, 50(1), 44–63.
Williams, E. N., & Morrow, S. L. (2009). Achieving trustworthiness in qualitative research: A pan-paradigmatic perspective. Psychotherapy Research, 19(4 & 5), 576–582.
Wolcott, H. F. (1994). Transforming qualitative data: Description, analysis, and interpretation. Thousand Oaks, CA: Sage.
Asthma is the leading cause of missed school days despite advancements in asthma treatment. This may be, in part, due to a lack of understanding about asthma. Okay With Asthma™, an online story with psychosocial management strategies for school-age children, was pilot tested to measure its effect on asthma knowledge and attitude. The online program delivers content about asthma through a digital story and story-writing program. Using a one-group pretest-post-test quasi-experimental design, 35 children with moderate to severe asthma completed a pretest measure of asthma knowledge and attitudes and then completed Okay With Asthma™. At 1 week and 2 weeks after the intervention, the children completed the measures again. There were significant improvements in asthma knowledge scores at the 1- and 2-week evaluations and significant improvements in attitude scores 2 weeks after the program. Okay With Asthma™ specifically targets school-age children and teaches them how to use school resources and peers while managing their asthma.
Yin, R. K. (Ed.). (2004). The case study anthology. Thousand Oaks, CA: Sage.
Yin, R. K. (2012). Applications of case study research (3rd ed.). Thousand Oaks, CA: Sage.
Yin, R. K. (2013). Case study research: Design and methods (5th ed.). Thousand Oaks, CA: Sage.
Young, P., & Miller-Smith, K. (2006). Effects of state mandated policy (site-based councils) and of potential role incumbents on teacher screening decisions in high and low performing schools. Education Policy and Analysis Archives, 14(7), 1–21.
Zimmerman, D. W., & Zumbo, B. D. (2009). Hazards in choosing between pooled and separate-variances t tests. Psicológica, 30, 371-390.
Few researchers have explored how relationships between patients and providers might change problem solving in clinical practice. The authors used grounded theory to study dyads of 11 people with diabetes and poor glycemic control, and 8 nurses interacting in diabetes teams. Relational Potential for Change was identified as a core category that involved three types of relationships. Professionals mostly shifted between less effective relationships characterized by I-you-distant provider dominance and I-you-blurred sympathy. Although rarely seen, a third relationship, I-you-sorted mutuality proved more effective than the others in exploiting the Relational Potential for Change. The three types of relationship differed in (a) scope of problem solving, (b) the roles assigned to the patient and the professionals, (c) use of difficult feelings and different points of view, and (d) quality of knowledge achieved as the basis for problem solving and decision making. The authors discuss implications for practice and further research.
Zolotor, A. J., Runyan, D. K., Dunne, M. P., Jain, D., Petrus, H. R., Ramirez, C., . . . Isaeva, O. (2009). ISPCAN Child Abuse Screening Tool Children’s Version (ICAST-C): Instrument development and multi-national pilot testing. Child Abuse & Neglect, 33(11), 833–841.
This study assessed the effect of cognitive tools on scaffolding students defining an ill-structured problem. Seventy-nine 10th-grade students used the Pollution Solutionsoftware to study an environmental problem. A quasi-experimental, mixed-method design was used. One class received the organization tool, a second class received the higher-order thinking tool, a third class received both tools, and a control group received neither tool. The organization tool group obtained a moderate understanding of the problem, which was significantly higher than the control group. Students who received the organization tool also asked significantly more problem-related questions than the other groups. The higher-order thinking tool prompted students to ask significantly more questions than the other groups. The combination tool group did not perform as well as expected. This study raised some questions about whether combining tools will add the effects of these individual tools.