Statistics from Altmetric.com
Meta-analyses and systematic reviews are often taught as top tier research since they offer comprehensive evidence synthesis. Systematic reviews describe many related, thoughtfully selected studies put together to provide a bigger picture and may include a meta-analysis which statistically evaluates these studies to generate a quantified and summarised outcome. The quality of a systematic review and/or meta-analysis is highly dependent on the quality of the studies included, as studies with high risk of bias(such as those lacking randomization) or of lower quality evidence (such as retrospective case-control studies) can consequentially deteriorate the validity.1 Quality meta-analyses can reduce the risk of claiming there is a difference when really there is not (termed a type I error), when multiple studies support a similar result. They can also reduce the opposite risk, claiming there is no difference when really there is one (termed a type II error). This error happens more often when the true difference is of small magnitude. Meta analyses offer larger sample sizes by pooling many individual studies which increases the power to detect smaller differences.1 Therefore, when hearing, ‘A recent meta-analysis shows…' eager trainees hoping to practice evidence based medicine(EBM) often embrace that research. I was one of these eager trainees, but after assisting with a meta-analysis, I became acutely aware of some of the risks inherent to meta-analyses and caution a critical eye even among this top tiered evidence category. Below, I describe three take-home points I learnt and that I hope may benefit other medical trainees in EBM.
Know the Inclusion and Exclusion Criteria, aka the PRISMA flow chart
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement describes how articles are selected to be included in a review which is only part of its 27-item checklist. Clear, specific and well-defined PRISMA statements give readers confidence in the thoughtful intent and diligence of the authors. …
Contributors No financial support was needed, and I have no conflicts of interest. Figures were created by myself (Figure 1) or modified from sources under Creative Commons Attribution 4.0 license and attribution given as specified (Figure 2) and then both adapted using other texts as cited.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; internally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.