A.    Checklist for Exploring Text

   1.    Planning: Thinking through ‘who’, ‘where’, ‘when’, ‘how’ and ‘what’. Have you considered:

• Creation of a list of ‘texts’ you wish to explore. If the breadth of texts you wish to explore is overly wide, you will need to develop an appropriate sampling strategy
• Access - how you will locate and access texts
• How you plan to control your biases
• How you might develop the skills/resources needed to carry out your textual exploration
• Strategies for ensuring credibility
• What it is you are looking for or trying to find in your texts
• Ethics/ethics approval
• Contingencies

   2.    Gathering. Have you:

• Organized – for collected text, you will want to develop and employ an organization and management scheme.
• Copied – make copies of original text for the purpose of annotation.

   3.    Reviewing. Have you:

• Assessed the authenticity and credibility of the ‘text’
• Explored the text’s agenda, i.e. reviewed the text and considered any inherent biases

   4.    Interrogating. Have you explored:

• Background Information
• Questions ‘about’ the text - Who produced it? What did they produce it for? What were the circumstances of production? When, where and why was it produced?
• Content within the text

   5.    Reflecting and refining. Have you:

• Improved your processes as you go
• Reflected on any difficulties associated with gathering the texts, reviewing the sources, and exploring the content
• Reviewed your notes
• Made modifications
• Kept reviewing and refining until you are comfortable with the process and data collected

6.    Analysing data. Have you considered:

• Mode of analysis - data collected in textual analysis is best analysed quantitatively (tallying and statistical analysis) or qualitatively (through deeper reflective processes).

B.    Steps in Secondary Data Analysis: Stepping Your Way through Effective Secondary Data Analysis

1.    Determine your research question – As indicated above, knowing exactly what you are looking for

2.    Locating data – Knowing what is out there and whether you can gain access to it. A quick Internet search, possibly with the help of a librarian, will reveal a wealth of options.

3.    Evaluating relevance of the data – Considering things like the data’s original purpose, when it was collected, population, sampling strategy/sample, data collection protocols, operationalization of concepts, questions asked, and form/shape of the data.

4.    Assessing credibility of the data – Establishing the credentials of the original researchers, searching for full explication of methods including any problems encountered, determining how consistent the data is with data from other sources, discovering whether the data has been used in any credible published research.

5.    Analysis – This will generally involve a range of statistical processes as discussed in Chapter 13.

C.    Steps in Systematic Data Analysis: Stepping Your Way through Effective Systematic Data Analysis

1.    Formulate the research question – like any research process, a clear, unambiguous research question will help set the direction for your study, i.e.) what type of health promotions campaigns have been most effective in reducing smoking rates of Australian teenagers or Does school leadership makes a difference to educational standards?

2.    Develop and use an explicit, reproducible methodology – key to systematic reviews are that bias is minimized and that methods are transparent and reproducible. 

3.    Develop and use clear inclusion/ exclusion criteria – the array of literature out there is vast. Determining clear selection criteria for inclusion is essential. 

4.    Develop and use an explicit search strategy – it is important to identify all studies that meet the eligibility criteria set in #3. The search for studies needs to be extensive should be extensive and draw on multiple databases.

5.    Critically assess the validity of the findings in included studies – this is likely to involve critical appraisal guides and quality checklists that cover participant recruitment, data collection methods and modes of analysis. Assessment is often conducted by two or more reviewers who know both the topic area and commonly used methods.

6.    Analyse of findings across the studies – this can involve analysis, comparison and synthesis of results using methodological criteria. This is often the case for qualitative studies. Quantitative studies generally attempt to use statistical methods to explore differences between studies and combine their effects (see meta-analysis below). If divergences are found, the source of the divergence is analysed.

7.    Synthesis and interpretation of results – synthesized results need to be interpreted in light of both the limitations of the review and the studies it contains.  An example here might be the inclusion of only studies reported in English. This level of transparency allows readers to assess the review credibility and applicability of findings

D.    Steps in Meta-Analysis: Stepping Your Way through Effective Meta-Data Analysis

Since meta-analysis is a subset (but not a requirement) of systematic reviews, basic steps are similar. Therefore, only point 6 is expanded upon. For more on the other points see checklist for systematic reviews:

1.    Formulate the research question

2.    Develop and use an explicit, reproducible methodology (this step was done when meta-analysis was originally decided upon)

3.    Develop and use clear inclusion/ exclusion criteria

4.    Develop and use an explicit search strategy

5.    Critically assess the validity of the findings in included studies 

6.    Analyse of findings across the studies - statistical analysis would involve decisions related to:

-  the dependent and independent variables under review

-  how studies will be weighted according to sample size

-  how to conduct sensitivity analysis (the extent to which study results stay the same given different approaches to aggregating data)

7.    Synthesis and interpretation of results