You are here

  • Home
  • Online First
  • Inverse publication reporting bias favouring null, negative results

Article Text

Article menu

other Versions

Download PDFPDF
Analysis
Inverse publication reporting bias favouring null, negative results
  1. John P A Ioannidis1,2,3,4,5
  1. 1Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford, California, USA
  2. 2Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, California, USA
  3. 3Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, California, USA
  4. 4Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, California, USA
  5. 5Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, USA
  1. Correspondence to Dr John P A Ioannidis, Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, CA 94305, USA; jioannid{at}stanford.edu

https://doi.org/10.1136/bmjebm-2023-112292

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • Publication reporting bias is known to be a major threat for evidence-based medicine, but it is less appreciated that an inverse bias may operate sometimes, favouring the publication and dissemination of non-significant, null results.

    WHAT THIS STUDY ADDS

    • Inverse publication reporting bias may operate in diverse settings including non-inferiority studies, adverse events literature, other reassurance-oriented narratives, the Proteus phenomenon and reproducibility checks.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • More transparency in protocols, statistical analysis plans and registration, reduction of conflicts and heightened awareness may help diminish the impact of inverse publication reporting bias.

    Introduction

    Classic publication (‘file drawer’) bias and related reporting biases threaten the validity of the scientific literature. Theoretical considerations and empirical data have demonstrated that in many fields and settings, there is preference for publishing studies with statistically significant results and/or larger effect sizes; moreover, among published studies, there is a preference for reporting outcomes and analyses with similarly biased profile.1 2 However, sometimes a preference may exist for reporting studies, outcomes and/or analyses with non-statistically significant, ‘null’ results and/or smaller effects sizes. This could manifest as suppression of the publication of entire studies with significant results (the inverse of the classic file drawer problem); suppression of the reporting of specific unfavourable outcomes and analyses; or manipulation/alteration of the undesirable statistically significant studies or of specific outcomes and analyses. Besides numerical data being suppressed and/or distorted, biased interpretation may add another layer: highlighting preferentially non-significant and/or smaller effects and making and disseminating conclusions disproportionately favouring the null.

    For example, the reporting of the Merck-sponsored VIGOR trial attenuated the significantly increased cardiovascular harm of rofecoxib (Vioxx).3 The VIGOR publication reported an interim analysis with different termination dates for cardiovascular and gastrointestinal events, thus several rofecoxib cardiovascular events were not counted. The harm was further minimised by a post hoc subgroup analysis. The trial also added spin in …

    View Full Text

    Footnotes

    • Contributors JPAI had the original idea, wrote the paper and approved the final version and he is the guarantor of the work.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.