Original ResearchBias in reporting of randomised clinical trials in oncology
Introduction
Clinical trials are undertaken to evaluate efficacy and toxicity of new interventions. Typically, phase II trials in oncology evaluate whether a drug has clinical activity for a given tumour, and if results are encouraging, these studies may be followed by large phase III trials that determine if this new intervention is more effective and/or less toxic than the established standard of care. Study reports should be accurate, objective and balanced with the main focus on the predefined primary outcome measure. Bias in reporting occurs when there is selective reporting of results that can influence the impression of the reader. Biased reporting can lead to a phase III randomised clinical trial where hundreds of patients are exposed to a drug that may not have shown appropriate activity or tolerability in a phase II trial [1] or to inappropriate clinical decisions based on a phase III randomised controlled trial (RCT).
Several factors affect the quality and trustworthiness of studies reported in the medical literature. Among these factors are whether reporting is consistent with the statistical results [2], [3], if end-points are changed during the course of a clinical trial (usually to allow reporting of a more positive result) [4], if toxicity is clearly reported and how funding (especially from for-profit sources) affects reporting of results [5].
Scientific articles are not simply reports of facts. Authors have many opportunities to consciously or subconsciously shape the impression of their results for readers; that is, to use biased language in their scientific report [6], [7], [8]. Spin is defined as use of reporting strategies to highlight beneficial aspects of an intervention, despite a statistically non-significant difference in the primary outcome or to distract the reader from statistically non-significant results [5]. It is important to recognise the presence of spin in reports of clinical trials and to evaluate how misinterpretation of results affects reader judgement about credibility [9].
Here we review manuscripts reporting RCTs evaluating systemic therapies for cancer to quantify the extent of biased reporting and the impact of financial relationships on biased reporting. We also recorded conflicts of interest of the first and corresponding authors. We hypothesised that despite the availability of guidelines to minimise bias, this remains prevalent and can be influenced by authors' financial ties.
Section snippets
Literature search and study selection
A comprehensive search of all articles published between July 2010 and December 2012 in the New England Journal of Medicine, The Lancet, the Journal of the American Medical Association, Lancet Oncology, the Journal of Clinical Oncology and the Journal of the National Cancer Institute was performed manually to extract papers reporting results of RCTs for cancer patients. The rationale for selection of the highest impact journals was the assumption that they would contain reports of clinical
Results
A total of 403 articles were identified initially and 200 RCTs (48 phase II studies and 152 phase III studies) were eligible for analysis (Fig. 1). The characteristics of the trials are described in Table 1. Ninety-four protocols were available for comparison. Inter-reviewer agreement was very good: Cohen's κ for was 0.88 (95% CI = 0.80–0.96) for assessment of spin in reporting of efficacy and 0.88 (95% CI = 0.81–0.96) for evaluation of under-reporting of toxicity.
One hundred ninety-three
Discussion
We reported previously that bias in reporting outcomes was present in almost 60% of articles reporting studies with a negative primary end-point in RCTs for breast cancer [4]. Here we confirm biased reporting of efficacy for 47% of RCTs evaluating treatments for a variety of tumour sites, when there was no statistically significant difference in the primary end-point, even though our study was limited to journals with high-impact factors that publish reports of clinical trials that may lead to
Funding
Dr. Vera-Badillo acknowledges research funding from Consejo Nacional de Ciencia y Tecnologia, Mexico (Ref: 214638).
Conflict of interest statement
None declared.
References (18)
- et al.
Bias in reporting of end points of efficacy and toxicity in randomized, clinical trials for women with breast cancer
Ann Oncol
(2013) Limitations are not properly acknowledged in the scientific literature
J Clin Epidemiol
(2007)- et al.
The quality of reporting of trial abstracts is suboptimal: survey of major general medical journals
J Clin Epidemiol
(2009) - et al.
Immunogenicity and adverse events of avian influenza A H5N1 vaccine in healthy adults: multiple-treatments meta-analysis
Lancet Infect Dis
(2009) - et al.
Phase III trials with docetaxel-based combinations for metastatic castration-resistant prostate cancer: time to learn from past experiences
J Clin Oncol
(2013) - et al.
Powerful spin in the conclusion of Wampold et al.'s re-analysis of placebo versus no-treatment trials despite similar results as in original review
J Clin Psychol
(2007) - et al.
Research ethics: ethical issues of data reporting and the quest for authenticity
Acad Emerg Med
(2000) - et al.
Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of cancer: the SPIIN randomized controlled trial
J Clin Oncol
(2014) The rhetoric of research. Embrace scientific rhetoric for its power
BMJ
(1995)
Cited by (35)
Outcome reporting bias in nephrology randomized clinical trials: Examining outcomes represented by graphical illustrations
2022, Contemporary Clinical Trials CommunicationsCitation Excerpt :In a large majority of cases, the graphical illustrations leading to ORB represented outcomes that were statistically significant or positive results relating to the trial intervention. Previous research has shown a similar high prevalence of ORB in RCTs, but no previous study has focused on the contribution of graphical illustrations to ORB [6–8,18–20]. This is very surprising, given the key role that data visualization has assumed in the medical literature since the first principles were historically presented by authors such as Tufte and Cleveland, and particularly with the advent and growth of social media and electronic publishing [21,22].
Extent and prevalence of spin in randomized controlled trials in dentistry
2020, Journal of DentistryCitation Excerpt :A similar frequency of spin in the abstract was found in a study of low back pain systematic reviews (80 %) [22]. In other studies, various frequencies of spin have been reported, ranging from 35 to 55% [10,23,24]. The variability in the prevalence of spin among the various scientific fields may be the result of different methodology and tested interventions.
A critique of the fragility index – Authors' reply
2019, The Lancet Oncology