Given the prolific nature of Academic Publishing, it can be very difficult to come to a conclusion based on single research papers, no matter how high-impact or widely read. This is especially important in the field of Medicine where high quality evidence of the benefits and risk of treatments is literally a matter of life and death.
Well-funded studies with large sample sizes but with possible evidence of bias may show one conclusion (for example the prescribing of a certain drug) while smaller independent studies may show possible evidence of harm of the same drug. These studies need to be looked at together, or we risk:
Some recommended additional reading
Editorial: What Makes Systematic Reviews Systematic and Why are They the Highest Level of Evidence
YouTube: Mixed Results in Sets of Studies (Importance of systematic review)
YouTube: Introduction to Meta-analysis (Importance of meta-analysis)
Cochrane: Evidence Synthesis - What is it and why do we need it?
Doing A Literature Review In Health And Social Care - The Systematic Review (Lancaster Subscription)
A good starting point for understanding the need for Systematic Reviews, and start your journey towards choosing a topic is to understand about "uncertainty". Topics chosen for a Systematic Review are not chosen out of general interest or curiosity, but due to an inherent uncertainty in the literature, meaning no clear conclusion or evidence exists for one course of action over another. Uncertainty can be found in:
Boolean
A type of algebra used in database searches. Usually you would use AND / OR searching and avoid the NOT function in a Systematic Review
Heterogeneity
Are all the studies in a systematic review similar? Then it would be stated that the paper has 'Low Heterogeneity. If the studies included in a systematic review differ due to research methodologies or reporting methods, then it has possibly 'High Hetereogenity' and it would be difficult to meaningfully compare them.
Highly Sensitive Search Strategy (HSSS)
A search strategy that is highly sensitive, missing as few relevant papers as possible with the drawback of an increased number of irrelevant papers.
Inclusion/Exclusion/Eligibility Criteria
What papers you are Including in your Systematic Review (via your Systematic search) and What you are excluding (Via Screening).
Meta-Analysis and Synthesis
A statistical method of combining evidence, or results of studies. A systematic review is refered to a type of 'Evidence Synthesis' and the Meta-Analysis can also be called 'Evidence Synthesis. Not all systematic reviews have a Meta-Analysis, and it may be unsuitable in mixed-methods and non-health Systematic Reviews.
Protocol
The writeup of the methodology of the Systematic Review and detaining the planning, search strategy and methods. It should be prepared before commencement of a Systematic Review and referred to for guidance.
PRISMA 2020
PRISMA is a minimum set of items for reporting in systematic reviews and meta-analyses. Typically used for interventions in medicine, it is one of the most popular protocols to adapt to other non-clinical subject areas. The PRISMA flow diagram is in extensive use as a visual representation of a literature search.
Proximity Operators
An operator used in a database search. The most commonly used operator is 'near' which retrieves papers if words are within a defined number of each other. In EBSCO databases it is N3 while in OVID it is ADJ3.
.RIS
A standard document format for exporting references from database searches, and then importing into reference management software or screening applications/websites.
Reproducibility
The ability to easily reproduce a literature search strategy in the future
Search Strategy
The 'strategy' for your systematic search. This should be based if possible on a methodology that retrieves the most relevant papers possible with a risk of some non-relevant papers included. The aim is to reduce the number of papers to a point that can be pragmatically screened, not produce a search that limits to a small number of highly relevant papers!
It may be that a traditional Systematic Review is not suitable for all questions.
However: If you find either a poor evidence base, zero evidence, or significant Heterogenity, the reasons for this can be studied in further research.
Congratulations!
This guide is split into two parts, specific guidance pertaining to Health, and also guidance focusing on Science and Technology SRs.
You can then visit the systematic review 'Pathway' with a recommended general process and useful links called 'Undertaking your Systematic Review'.
If you have decided that a systematic review may not be the most suitable course of action, you may wish to consider an alternative review type. The paper Meeting the review family: exploring review types and associated information retrieval requirements identified 48 unique review types, however below are the more commonly seen in research.
Rapid/Pragmatic Review
A review type usually performed in a shorter time period, and often used for new and emergent topics. Has some elements of a Systematic Review, but with limits on time some pragmatic shortcuts (Such as less sensitive searches, or AI Screening) may be introduced with a possible risk of bias. Review the Cochrane Rapid Reviews Methods Group paper for further information.
Umbrella Review
With the popularity of Systematic Reviews some subjects may already have multiple published reviews. An umbrella review is a review of previously published Systematic Reviews with a uniform treatment of meta-analysis across all reviews. These form one of the most rigorous and highest levels of evidence synthesis. Read Ten simple rules for conducting umbrella reviews for a summary.
Systemized Review:
Many systematic reviews which are produced on a sole basis in small teams could accurately be referred to as Systemized Reviews. They may lack some of the elements of a Systematic Review such as a meta-analysis, however still be produced with elements of a Systematic Review.
Scoping Review
A type of review that aims to identify the scope of literature on a topic. Useful to identify the feasibility of a systematic review. Read Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach for a summary.
Integrative Review
A review type that combines diverse methodologies such as experimental and non-experimental research. Read ‘The integrative review: updated methodology’
Executive Evidence Summary
A piece of work not intended for journal publishing, but for the attention of organisational or national bodies in setting policy or guidance. These can be produced in a manner similar to a systematic review to create a highly compelling and evidence-based work.
Systematic Literature Review
A summary of the literature on a topic. It may be that either due to time constraints, qualitative nature of the topic or possible high levels of heterogeneity in studies, a systematic review with meta-analysis isn't possible to complete. A literature review completed to the high standards of a systematic review, perhaps without a meta-analysis may be suitable.
Rapid Evidence Appraisal (REA)
More rigorous than a literature review. REAs summarise what is and is not known in a subject, with elements of a systematic review included that minimise bias. Designed to be quicker than a systematic review.
Realist Review
While Systematic reviews focus on a specific topic, realist reviews represent a theory-based approach to synthesis. This type of review can require a greater commitment of time working with an Information Professional or Librarian, so ensuring you have access to such staff is key. Read Scoping and Searching to Support Realist Approaches in Doing Realist Research (Sage Research Methods Core Subscription)
Note that many of these review types are very similar! "Almost" systematic reviews in different disciplines such as sociology or computing may be called different names while sharing tell-tale methodologies such as a published protocol, search strategy reporting or Prisma flow diagrams.
The paper A typology of reviews: an analysis of 14 review types and associated methodologies has an extensive list of the strengths and weaknesses of different review types.
The Knowledge Synthesis Decision Tool from Unity Health Toronto can help you decide what type of review to do.
This guide is available (CC BY-NC 3.0)
If you do reuse, please let us know so we can share any updates.