Increased transparency in study design and analysis is one proposed solution to the perceived reproducibility crisis facing science. Systematic review and meta-analysis—through which individual studies on a specific association are ascertained, assessed for quality and quantitatively combined—is a critical process for building consensus in medical research. However, the conventional publication model creates static evidence summaries that force the quality assessment criteria and analytical choices of a small number of authors onto all stakeholders, some of whom will have different views on the quality assessment and key features of the analysis. This leads to discordant inferences from meta-analysis results and delayed arrival at consensus. We propose a shift to interactive meta-analysis, through which stakeholders can take control of the evidence synthesis using their own quality criteria and preferred analytic approach—including the option to incorporate prior information on the association in question—to reveal how their summary estimate differs from that reported by the original analysts. We demonstrate this concept using a web-based meta-analysis of the association between genetic variation in a key tamoxifen-metabolising enzyme and breast cancer recurrence in tamoxifen-treated women. We argue that interactive meta-analyses would speed consensus-building to the degree that they reveal invariance of inferences to different study selection and analysis criteria. On the other hand, when inferences are found to differ substantially as a function of these choices, the disparities highlight where future research resources should be invested to resolve lingering sources of disagreement.
- breast tumours
- statistics & research methods
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Contributors All authors contributed equally to the conceptualisation and data curation for this paper. TPA wrote the first draft of the manuscript and created the web application that accompanies this paper. All authors critically reviewed and revised the manuscript.
Funding This work was supported in part by the US National Library of Medicine (R01LM013049 awarded to TLL), the US National Cancer Institute (R01CA166825 awarded to TLL and F31CA239566 awarded to LJC) and the US National Institute of General Medical Sciences (P20GM103644 awarded to TPA).
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.