Critical appraisal framework

The purpose of the study

  • Is the knowledge sought already available?
  • Is there an important reason for the research to be undertaken?
  • Are the potential outcomes of the study realistic?
  • Was/were the researcher(s) appropriately qualified/supported to undertake the research?
  • Are there any concerns about any funders of the research in relation to the process of the research described?

Research problem and research questions

  • Is the problem significant and researchable and have all potential ways of solving the problem been considered?
  • Are all research questions and hypotheses developed directly from the problem?
  • Did the research place unethical or unrealistic demands on participants?

Literature search and review

  • Was there a search of a wide range of literature pertinent to the topic?
  • Was there a search strategy with named databases and key search terms?
  • Was the review balanced and not biased?
  • Was the literature critically appraised?
  • Was any conflicting evidence clearly presented?
  • Did the literature review provide rationale and direction for the research?
  • Is the literature review up to date?
  • Were any limitations of the literature identified?

Ethical issues

  • Ethical issues should be considered at all stages of the study.
  • Is there evidence of approval from the appropriate Research Ethics Committee?
  • Were any governance issues dealt with appropriately?
  • There should be clear evidence that privacy, dignity, anonymity and confidentiality were maintained throughout the study.
  • The researcher should have identified ethical issues related to the study.
  • Were any participants fully informed about the nature of the research?

Sample selection

  • Was an appropriate sampling strategy used?
  • If a random sample was selected, was it genuinely random?
  • Were any biases in the sample group identified?
  • Was the target population identified in a quantitative study?
  • Was there a clear account of how participants were recruited and selected to take part in the study?
  • Was there any coercion in recruiting participants?
  • Was there clear evidence that participants gave informed consent?
  • If the participants were vulnerable, has this been clearly considered in the study?
  • Were all the participants accounted for throughout the study, i.e., was any attrition noted and discussed?

Research design and data collection

  • Was the design of the study appropriate to the research questions?
  • Was an appropriate method of data collection used?
  • Are the advantages and disadvantages of the method(s) discussed?
  • Was the reliability and validity of any data collection tool tested?
  • Was a pilot study conducted and the results used to inform the main study?
  • What types of questions were asked?
  • Is there an explanation of the observer role in the study if needed?
  • Were the participants protected from physical and psychological harm?
  • Was the issue of ‘deception’ dealt with appropriately in observational studies?
  • Were the data gathered by appropriate people?
  • Was the researcher’s role and relationship with the participants fully considered?

Results and analysis of findings

  • Were the results and analysis linked back to the original research question?
  • Were the results and analysis manipulated in order to favour particular findings?
  • Are steps that have been taken to demonstrate the trustworthiness of the data in a qualitative study clear?
  • Was there any evidence of lost data?
  • Was there evidence of a statistician’s input into complex quantitative analysis?
  • Is the use of specific statistical tests justified?
  • Is there a link between the research approach, the level of measurement gained from the data collection tool and the statistical tests used?
  • Is the level of significance produced by the statistical testing given?
  • In qualitative studies, were the data authenticated?

Conclusions, recommendations and limitations

  • Were the conclusions and recommendations based on the results of the study?
  • Was it clear that there was no intention to mislead or give false conclusions?
  • Was the hypothesis accepted or rejected?
  • Was the sample selected considered in relation to the recommendations?
  • Did the researcher acknowledge any limitations?
  • Were limitations of the findings of the study identified, as well as limitations of the study design and techniques?
  • Are any implications for practice and policy considered?

General points

  • The researcher should acknowledge sources of support and funding.
  • When critically appraising research, readers should acknowledge their own limitations and gain assistance when necessary.
  • The role of service users and carers in the research should be explained.