Objectives Systematic reviews and meta-analyses are the foundation of modern evidence-based medicine and their use is becoming more prolific than randomised trials. However, systematic reviews are receiving increasing criticism for contributing to research waste, varying massively in their reporting or methodological quality, being misleading and serving conflicted interests.¹ Systematic reviews are upheld as being objective, dispassionate scientific processes but they can bias the evidence base. Influential decisions are made by people at all stages of a review including question setting, searching, study selection, interpretation, and making recommendations for research or practice. We know that biased research is prevalent in primary research and it is becoming increasingly apparent that questionable research practices also affect systematic reviews.

• Improve the systematic use of existing evidence by exploring and highlighting biases and poor conduct in systematic reviews

• Emphasise under–recognised biases or influences that can affect systematic reviews and disseminate them in a clear and accessible way to relevant audiences including health researchers, policy makers and journal editors

Method Initiatives for assessing methodological rigour and reporting such as Cochrane or PRISMA do not discuss errors, misconduct or bias which are attributable to the review team, and such influences can affect the reliability and validity of evidence syntheses.

Standard conflicts of interest (CoI) statements focus on narrow commercial interests and are inadequate to address potentially hidden agendas in systematic review teams. For example, CoI are frequently not declared as such by review authors who are psychological therapists when reviewing their own psychotherapy (researcher allegiance). Reviews with a high number of authors affiliated to the intervention are more likely to have positive conclusions and more likely to be lower quality reviews (confirmation bias). At least a quarter of investigators in biomedical research have industry affiliations, and a significant relationship exists between industry sponsorship and pro-industry conclusions (sponsorship bias).

Results Stakeholder participation in reviews, whilst being potentially beneficial, can also be problematic to manage as involvement of content area experts can make it more difficult to perform an unbiased review (one-sided reference bias). The opportunity to assess why the review authors were motivated to assess the evidence base is too limited if relying on information such as funding source and the expression of pecuniary conflicts.

Currently consumers of systematic reviews cannot rely on journal publication declarations to know whether those conducting a review have vested interests or are appropriately skilled. Policy decisions which ultimately affect patient care are influenced by systematic reviews therefore the integrity of their conduct requires more scrutiny.

Potential solutions:

• Empirical research to scrutinise less obvious CoI and examine the impact on published systematic reviews conclusions

• A dedicated website to clearly explain biases that systematic reviews are prone to

• Guidelines to help peer–reviewers, journal editors and the public understand appropriate systematic review team composition and assess potential for bias or poor conduct.

Reference

1. Ioannidis J. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. The Milbank Quarterly. 2016;94(3):485–514.