Elsevier

European Journal of Cancer

Volume 61, July 2016, Pages 29-35
European Journal of Cancer

Original Research
Bias in reporting of randomised clinical trials in oncology

https://doi.org/10.1016/j.ejca.2016.03.066Get rights and content

Highlights

  • Bias in reporting of efficacy outcomes is common for studies with a negative primary end-point.

  • Bias in reporting can lead to off-label misuse of experimental therapies, if they are approved for other indications.

  • Spin is used frequently to distract the reader in reports of randomised controlled trials.

  • Toxicity is under-reported, especially for studies with a positive primary end-point, leading to a biased view of the safety of new treatments.

  • Funding did not appear to have a significant influence to increase bias in reporting.

Abstract

Background

Bias in reporting efficacy and toxicity in clinical trials may impact treatment decisions. Here, we report quality of reporting of efficacy and of toxicity in articles describing randomised controlled trials (RCTs) of cancer therapy and the association between biased reporting and study results, funding and financial relationships of the authors with the sponsor.

Materials and methods

We reviewed articles published from July 2010 to December 2012 in six high-impact journals reporting RCTs of systemic treatment for cancer. Bias in reporting of the primary end-point and toxicity were assessed. Associations between biased reporting and study results, funding source and financial ties of the author with the funding source were evaluated using logistic regression.

Results

Two hundred articles were identified. Among 107 RCTs where there was no statistically significant difference in the primary end-point between the two arms, 50 (47%) reports used biased reporting in the abstract of the paper to imply benefit of the experimental treatment. Toxicity was not reported in the abstract in 18.5% of the studies and this was associated with a positive primary end-point. Source of funding and financial ties were not associated with biased reporting.

Conclusions

Bias in reporting of efficacy outcomes is common for studies with a negative primary end-point and can lead to off-label misuse of experimental therapies, if they are approved for other indications. Toxicity is under-reported, especially for studies with a positive primary end-point, leading to a biased view of the safety of new treatments.

Introduction

Clinical trials are undertaken to evaluate efficacy and toxicity of new interventions. Typically, phase II trials in oncology evaluate whether a drug has clinical activity for a given tumour, and if results are encouraging, these studies may be followed by large phase III trials that determine if this new intervention is more effective and/or less toxic than the established standard of care. Study reports should be accurate, objective and balanced with the main focus on the predefined primary outcome measure. Bias in reporting occurs when there is selective reporting of results that can influence the impression of the reader. Biased reporting can lead to a phase III randomised clinical trial where hundreds of patients are exposed to a drug that may not have shown appropriate activity or tolerability in a phase II trial [1] or to inappropriate clinical decisions based on a phase III randomised controlled trial (RCT).

Several factors affect the quality and trustworthiness of studies reported in the medical literature. Among these factors are whether reporting is consistent with the statistical results [2], [3], if end-points are changed during the course of a clinical trial (usually to allow reporting of a more positive result) [4], if toxicity is clearly reported and how funding (especially from for-profit sources) affects reporting of results [5].

Scientific articles are not simply reports of facts. Authors have many opportunities to consciously or subconsciously shape the impression of their results for readers; that is, to use biased language in their scientific report [6], [7], [8]. Spin is defined as use of reporting strategies to highlight beneficial aspects of an intervention, despite a statistically non-significant difference in the primary outcome or to distract the reader from statistically non-significant results [5]. It is important to recognise the presence of spin in reports of clinical trials and to evaluate how misinterpretation of results affects reader judgement about credibility [9].

Here we review manuscripts reporting RCTs evaluating systemic therapies for cancer to quantify the extent of biased reporting and the impact of financial relationships on biased reporting. We also recorded conflicts of interest of the first and corresponding authors. We hypothesised that despite the availability of guidelines to minimise bias, this remains prevalent and can be influenced by authors' financial ties.

Section snippets

Literature search and study selection

A comprehensive search of all articles published between July 2010 and December 2012 in the New England Journal of Medicine, The Lancet, the Journal of the American Medical Association, Lancet Oncology, the Journal of Clinical Oncology and the Journal of the National Cancer Institute was performed manually to extract papers reporting results of RCTs for cancer patients. The rationale for selection of the highest impact journals was the assumption that they would contain reports of clinical

Results

A total of 403 articles were identified initially and 200 RCTs (48 phase II studies and 152 phase III studies) were eligible for analysis (Fig. 1). The characteristics of the trials are described in Table 1. Ninety-four protocols were available for comparison. Inter-reviewer agreement was very good: Cohen's κ for was 0.88 (95% CI = 0.80–0.96) for assessment of spin in reporting of efficacy and 0.88 (95% CI = 0.81–0.96) for evaluation of under-reporting of toxicity.

One hundred ninety-three

Discussion

We reported previously that bias in reporting outcomes was present in almost 60% of articles reporting studies with a negative primary end-point in RCTs for breast cancer [4]. Here we confirm biased reporting of efficacy for 47% of RCTs evaluating treatments for a variety of tumour sites, when there was no statistically significant difference in the primary end-point, even though our study was limited to journals with high-impact factors that publish reports of clinical trials that may lead to

Funding

Dr. Vera-Badillo acknowledges research funding from Consejo Nacional de Ciencia y Tecnologia, Mexico (Ref: 214638).

Conflict of interest statement

None declared.

References (18)

There are more references available in the full text version of this article.

Cited by (35)

  • Outcome reporting bias in nephrology randomized clinical trials: Examining outcomes represented by graphical illustrations

    2022, Contemporary Clinical Trials Communications
    Citation Excerpt :

    In a large majority of cases, the graphical illustrations leading to ORB represented outcomes that were statistically significant or positive results relating to the trial intervention. Previous research has shown a similar high prevalence of ORB in RCTs, but no previous study has focused on the contribution of graphical illustrations to ORB [6–8,18–20]. This is very surprising, given the key role that data visualization has assumed in the medical literature since the first principles were historically presented by authors such as Tufte and Cleveland, and particularly with the advent and growth of social media and electronic publishing [21,22].

  • Extent and prevalence of spin in randomized controlled trials in dentistry

    2020, Journal of Dentistry
    Citation Excerpt :

    A similar frequency of spin in the abstract was found in a study of low back pain systematic reviews (80 %) [22]. In other studies, various frequencies of spin have been reported, ranging from 35 to 55% [10,23,24]. The variability in the prevalence of spin among the various scientific fields may be the result of different methodology and tested interventions.

View all citing articles on Scopus
View full text