Restoring biomedical literature with RIAT
BMJ 2018; 361 doi: https://doi.org/10.1136/bmj.k1742 (Published 26 April 2018) Cite this as: BMJ 2018;361:k1742
- Peter Doshi, assistant professor1,
- Larissa Shamseer, postdoctoral fellow1,
- Mark Jones, statistician2,
- Tom Jefferson, senior associate tutor3
- 1University of Maryland School of Pharmacy, Baltimore, MD, USA
- 2University of Queensland School of Public Health, Brisbane, Australia
- 3Oxford University Centre for Evidence Based Medicine, Oxford, UK
- Correspondence to: P Doshi pdoshi{at}rx.umaryland.edu
Clinical trials are arguably medicine’s most powerful methodological tool for providing evidence on the effects of healthcare interventions. But for all the promises of evidence based medicine, the problem of publication bias from unpublished trials has threatened to undermine the enterprise. And over the past decade, a growing body of research has shown that trial publications—even those in the world’s most prestigious, peer reviewed journals—cannot be taken at face value.123456 Journal articles may inaccurately reflect both the study’s design and the data collected. Incompleteness and inaccuracy of the public record is a delicate ethical issue since clinical trials are experiments on humans. Whether intentional or not, these problems can cause harm to people and waste public resources.7
Noting these problems, in 2005 The BMJ’s former editor, Richard Smith, suggested that journals were no longer the ideal vehicle for reporting trials; regulated websites would be better.8 Three years later, the ClinicalTrials.gov registry opened its results database, and US law requires that all applicable trials report their basic results, including all prespecified primary and secondary outcomes and certain adverse events. However, although registers can hold more comprehensive information than journal publications,9 compliance with reporting requirements remains inadequate.10 Various mechanisms now exist for third parties to access large unabridged clinical study reports of industry sponsored trials of drugs and biological treatments, allowing researchers to assess trials as regulators do.1112
Restoring trust
But the reality is that journal articles—not independent analyses of underlying trial data—remain the backbone of the evidence relied on for clinical decision making. And so long as this remains true, restoring trust in what is published in biomedical journals remains vital for the progress of science and healthcare.
We launched the Restoring Invisible and Abandoned Trials (RIAT) initiative in 2013 to tackle the fundamental problems of trial invisibility and distortion that damage the biomedical literature.13 The RIAT concept posits that when original investigators and sponsors abandon their trials, either by not pursuing publication or by refusing to correct demonstrable errors in a trial publication, third parties that have obtained the underlying trial data are free to publish and correct the record.13
The basic ingredients of the initiative are simple: restorative authors (those wishing to correct the record) access underlying trial documents and data, and analyse it following the original trial protocol. Authors then draft a manuscript reporting the trial and submit it to a journal for publication.
During peer review and, importantly, after publication, restorative authors are expected to make all underlying data, sufficiently de-identified, available electronically. The BMJ and PLOS Medicine were the first journals to endorse RIAT.14 Four more journals have since followed suit, and others have expressed a willingness to consider RIAT manuscripts for publication.15
But four years on, only a few RIAT teams have assembled and carried out trial restorations.161718 Despite some notable successes—for example, republication of the infamous paroxetine Study 32917 and first publication of a four decade old study of a morning sickness drug18—the reality is that few people know how to access underlying trial data, and even fewer have the time to carry out a reanalysis.
New support
To energise this movement, the Laura and John Arnold Foundation has agreed to provide $1.4m (£1m; €1.1m) to fund the RIAT Support Center under our stewardship. The virtual support centre (www.restoringtrials.org) aims to assist researchers who would like to restore a trial by providing free support and is open to people anywhere in the world.
We will help verify the need for restoration by ensuring that the abandonment of the trial is properly documented. Because third party access to complex trial data and documents is still in its infancy, we will also provide advice on the scope and granularity of data needed for proper restoration and help restorative authors obtain data from sponsors and governments. We also offer a glossary, a directory of clinical trial data sources, and statistical advice.
The RIAT Support Center aims to raise awareness of misreporting or invisibility of clinical trials and to improve knowledge of publication and other reporting biases. We will develop systems to quickly identify misreported and unpublished trials that have the potential, if restored, to have a large effect on medical or public health practice.
The centre will also administer grant competitions. In the first, up to $150 000 will be awarded to researchers wishing to correct a misreported trial or publish a completed trial that remains unpublished. Applications will be adjudicated by an external panel of judges.
We welcome feedback and contributions from clinicians, researchers, and the wider public. Important questions remain, including how we rapidly identify affected trials and how we develop a legal and ethical framework for publishing underlying clinical trial data alongside restored trials. If you have technical skills or wisdom to share, please get in touch at support{at}restoringtrials.org.
Footnotes
Competing interests: We have read and understood BMJ policy on declaration of interests and declare the following: The Laura and John Arnold Foundation is a philanthropic organisation that supports the salaries of all the authors. For a complete list of disclosures, please see http://restoringtrials.org/about-us/. PD is also an associate editor at The BMJ.
Provenance and peer review: Commissioned; not externally peer reviewed.