Original ArticleA new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity
Section snippets
Background
Clinical and health-policy decision making should be based on the best evidence. Systematic reviews and meta-analyses are the cornerstones of therapeutic evaluation because they summarize all available evidence. Clinicians, decision makers, and researchers often use systematic reviews and meta-analyses (henceforth referred to as systematic reviews) to keep up to date with the medical literature, develop clinical practice guidelines, and sometimes plan future research [1], [2]. Patients can also
Methods
First, in a four-phase consensus process, we identified, by literature review and interviews, potential types of spin in systematic reviews of the effects of therapeutic interventions and developed a classification scheme of spin. Second, we ranked types of spin in abstracts according to their severity by using a Q-sort survey with invited members of the Cochrane Collaboration.
Identification of types of spin in systematic reviews
From our literature review and consensus process, we identified 39 different types of spin: 28 relevant to the main text of a report and 21 in the report's abstract and title. From the 39 types of spin, 13 (33%) were specific to systematic reviews (Table 1), and 26 (67%) could occur in other study designs. For example, the extrapolation of study findings from a surrogate marker to the global improvement of the disease could occur in systematic reviews as well as in RCTs, but the inadequate
Discussion
To our knowledge, our study is the first to provide a classification scheme of spin in systematic reviews. We identified 28 types of spin in the main text and 21 in the abstract that were classified in three categories: (1) misleading reporting, (2) misleading interpretation, and (3) inappropriate extrapolation. The ranking of spin in abstract according to their severity highlighted that the most severe were (1) recommendations for clinical practice not supported by findings in the conclusion,
Acknowledgments
The authors thank Laura Smales (BioMedEditing, Toronto, Canada) for English language proofreading.
The authors thank all participants of the survey who agreed to be named: Roland Bachter, Ruth Brassington, Mohammed T. Ansari, Evan Mayo-Wilson, Yoon K. Loke, Wilco Jacobs, Mariska M.G. Leeflang, Lucy Turner, Jennifer Petkovic, Andrea Tricco, Veronica Pitt, Marialena Trivella, Donna Coffin, Sean Grant, Barbara Mintzes, Jarn Wetterslev, Jan Odgaard-Jensen, Julian Higgins, Emma Sydenham, Roberta
References (55)
- et al.
Observational studies often make clinical practice recommendations: an empirical evaluation of authors' attitudes
J Clin Epidemiol
(2013) - et al.
Belief beyond the evidence: using the proposed effect of breakfast on obesity to show 2 practices that distort scientific evidence
Am J Clin Nutr
(2013) - et al.
Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting
Lancet
(2010) - et al.
How to increase value and reduce waste when research priorities are set
Lancet
(2014) Meta-analysis and causal inference: a case study of benzene and non-Hodgkin lymphoma
Ann Epidemiol
(2010)- et al.
Attention should be given to multiplicity issues in systematic reviews
J Clin Epidemiol
(2008) - et al.
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
J Clin Epidemiol
(2010) - et al.
Systematic reviews: synthesis of best evidence for clinical decisions
Ann Intern Med
(1997) - et al.
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
PLoS Med
(2010) - et al.
“Spin” in scientific writing: scientific mischief and legal jeopardy
Med Law
(2007)
Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes
JAMA
The rhetoric of research
BMJ
Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study
PLoS Med
“Spin” in wound care research: the reporting and interpretation of randomized controlled trials with statistically non-significant primary outcome results or unspecified primary outcomes
Trials
Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of Cancer: the SPIIN randomized controlled trial
J Clin Oncol
Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “spin”
Radiology
Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention
BMC Med Res Methodol
Use of causal language in observational studies of obesity and nutrition
Obes Facts
Assessment of claims of improved prediction beyond the Framingham risk score
JAMA
Spin and boasting in research articles
Arch Pediatr Adolesc Med
Powerful spin in the conclusion of Wampold et al.'s re-analysis of placebo versus no-treatment trials despite similar results as in original review
J Clin Psychol
The association between exaggeration in health related science news and academic press releases: retrospective observational study
BMJ
Reviewing the reviews. How strong is the evidence? How clear are the conclusions?
Intl J Technol Assess Health Care
The case of the misleading funnel plot
BMJ
Colloid use for fluid resuscitation: evidence and spin
Ann Intern Med
Interpreting epidemiological evidence: how meta-analysis and causal inference methods are related
Int J Epidemiol
Selective publication of antidepressant trials and its influence on apparent efficacy
N Engl J Med
Cited by (129)
Reporting Bias is Highly Prevalent in Systematic Reviews and Meta-Analyses of Platelet Rich Plasma Injections for Hip Osteoarthritis
2024, Arthroscopy, Sports Medicine, and RehabilitationReporting Bias in the Form of Positive Spin Is Highly Prevalent in Abstracts of Systematic Reviews on Primary Repair of the Anterior Cruciate Ligament
2024, Arthroscopy - Journal of Arthroscopic and Related SurgeryEvaluation of spin in reviews of biodegradable balloon spacers for massive irreparable rotator cuff tears
2024, Journal of Shoulder and Elbow SurgeryEvaluation of Spin in the Abstracts of Systematic Reviews and Meta-Analyses of Ulnar Collateral Ligament Reconstruction
2023, Arthroscopy, Sports Medicine, and RehabilitationEvaluation of Quality of Systematic Reviews and Meta-Analyses Abstracts for Pediatric Strabismus Therapies
2023, American Journal of OphthalmologySystematic review finds “spin” practices and poor reporting standards in studies on machine learning-based prediction models
2023, Journal of Clinical Epidemiology
Conflict of interest: None.
Funding: A.Y. was funded by a grant from the Fondation pour la Recherche Médicale (FDT20140930875). D.M. is funded through a University Research Chair.
Transparency declaration: A.Y. affirms that this article is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained [55].