Critical appraisal framework
The purpose of the study
- Is the knowledge sought already available?
- Is there an important reason for the research to be undertaken?
- Are the potential outcomes of the study realistic?
- Was/were the researcher(s) appropriately qualified/supported to undertake the research?
- Are there any concerns about any funders of the research in relation to the process of the research described?
Research problem and research questions
- Is the problem significant and researchable and have all potential ways of solving the problem been considered?
- Are all research questions and hypotheses developed directly from the problem?
- Did the research place unethical or unrealistic demands on participants?
Literature search and review
- Was there a search of a wide range of literature pertinent to the topic?
- Was there a search strategy with named databases and key search terms?
- Was the review balanced and not biased?
- Was the literature critically appraised?
- Was any conflicting evidence clearly presented?
- Did the literature review provide rationale and direction for the research?
- Is the literature review up to date?
- Were any limitations of the literature identified?
- Ethical issues should be considered at all stages of the study.
- Is there evidence of approval from the appropriate Research Ethics Committee?
- Were any governance issues dealt with appropriately?
- There should be clear evidence that privacy, dignity, anonymity and confidentiality were maintained throughout the study.
- The researcher should have identified ethical issues related to the study.
- Were any participants fully informed about the nature of the research?
- Was an appropriate sampling strategy used?
- If a random sample was selected, was it genuinely random?
- Were any biases in the sample group identified?
- Was the target population identified in a quantitative study?
- Was there a clear account of how participants were recruited and selected to take part in the study?
- Was there any coercion in recruiting participants?
- Was there clear evidence that participants gave informed consent?
- If the participants were vulnerable, has this been clearly considered in the study?
- Were all the participants accounted for throughout the study, i.e., was any attrition noted and discussed?
Research design and data collection
- Was the design of the study appropriate to the research questions?
- Was an appropriate method of data collection used?
- Are the advantages and disadvantages of the method(s) discussed?
- Was the reliability and validity of any data collection tool tested?
- Was a pilot study conducted and the results used to inform the main study?
- What types of questions were asked?
- Is there an explanation of the observer role in the study if needed?
- Were the participants protected from physical and psychological harm?
- Was the issue of ‘deception’ dealt with appropriately in observational studies?
- Were the data gathered by appropriate people?
- Was the researcher’s role and relationship with the participants fully considered?
Results and analysis of findings
- Were the results and analysis linked back to the original research question?
- Were the results and analysis manipulated in order to favour particular findings?
- Are steps that have been taken to demonstrate the trustworthiness of the data in a qualitative study clear?
- Was there any evidence of lost data?
- Was there evidence of a statistician’s input into complex quantitative analysis?
- Is the use of specific statistical tests justified?
- Is there a link between the research approach, the level of measurement gained from the data collection tool and the statistical tests used?
- Is the level of significance produced by the statistical testing given?
- In qualitative studies, were the data authenticated?
Conclusions, recommendations and limitations
- Were the conclusions and recommendations based on the results of the study?
- Was it clear that there was no intention to mislead or give false conclusions?
- Was the hypothesis accepted or rejected?
- Was the sample selected considered in relation to the recommendations?
- Did the researcher acknowledge any limitations?
- Were limitations of the findings of the study identified, as well as limitations of the study design and techniques?
- Are any implications for practice and policy considered?
- The researcher should acknowledge sources of support and funding.
- When critically appraising research, readers should acknowledge their own limitations and gain assistance when necessary.
- The role of service users and carers in the research should be explained.