Original Article
A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity

https://doi.org/10.1016/j.jclinepi.2016.01.020Get rights and content

Abstract

Objectives

We aimed to (1) identify and classify spin (i.e., a description that overstates efficacy and/or understates harm) in systematic reviews and (2) rank spin in abstracts of systematic reviews according to their severity (i.e., the likelihood of distorting readers' interpretation of the results).

Study Design

First, we used a four-phase consensus process to develop a classification of different types of spin. Second, we ranked the types of spin in abstracts according to their severity using a Q-sort survey with members of the Cochrane Collaboration.

Results

We identified 39 types of spin, 28 from the main text and 21 from the abstract; 13 were specific to the systematic review design. Spin was classified into three categories: (1) misleading reporting, (2) misleading interpretation, and (3) inappropriate extrapolation. Spin ranked as the most severe by the 122 people who participated in the survey were (1) recommendations for clinical practice not supported by findings in the conclusion, (2) misleading title, and (3) selective reporting.

Conclusion

This study allowed for identifying spin that is likely to distort interpretation. Our classification could help authors, editors, and reviewers avoid spin in reports of systematic reviews.

Section snippets

Background

Clinical and health-policy decision making should be based on the best evidence. Systematic reviews and meta-analyses are the cornerstones of therapeutic evaluation because they summarize all available evidence. Clinicians, decision makers, and researchers often use systematic reviews and meta-analyses (henceforth referred to as systematic reviews) to keep up to date with the medical literature, develop clinical practice guidelines, and sometimes plan future research [1], [2]. Patients can also

Methods

First, in a four-phase consensus process, we identified, by literature review and interviews, potential types of spin in systematic reviews of the effects of therapeutic interventions and developed a classification scheme of spin. Second, we ranked types of spin in abstracts according to their severity by using a Q-sort survey with invited members of the Cochrane Collaboration.

Identification of types of spin in systematic reviews

From our literature review and consensus process, we identified 39 different types of spin: 28 relevant to the main text of a report and 21 in the report's abstract and title. From the 39 types of spin, 13 (33%) were specific to systematic reviews (Table 1), and 26 (67%) could occur in other study designs. For example, the extrapolation of study findings from a surrogate marker to the global improvement of the disease could occur in systematic reviews as well as in RCTs, but the inadequate

Discussion

To our knowledge, our study is the first to provide a classification scheme of spin in systematic reviews. We identified 28 types of spin in the main text and 21 in the abstract that were classified in three categories: (1) misleading reporting, (2) misleading interpretation, and (3) inappropriate extrapolation. The ranking of spin in abstract according to their severity highlighted that the most severe were (1) recommendations for clinical practice not supported by findings in the conclusion,

Acknowledgments

The authors thank Laura Smales (BioMedEditing, Toronto, Canada) for English language proofreading.

The authors thank all participants of the survey who agreed to be named: Roland Bachter, Ruth Brassington, Mohammed T. Ansari, Evan Mayo-Wilson, Yoon K. Loke, Wilco Jacobs, Mariska M.G. Leeflang, Lucy Turner, Jennifer Petkovic, Andrea Tricco, Veronica Pitt, Marialena Trivella, Donna Coffin, Sean Grant, Barbara Mintzes, Jarn Wetterslev, Jan Odgaard-Jensen, Julian Higgins, Emma Sydenham, Roberta

References (55)

  • I. Boutron et al.

    Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes

    JAMA

    (2010)
  • R. Horton

    The rhetoric of research

    BMJ

    (1995)
  • A. Yavchitz et al.

    Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study

    PLoS Med

    (2012)
  • S. Lockyer et al.

    “Spin” in wound care research: the reporting and interpretation of randomized controlled trials with statistically non-significant primary outcome results or unspecified primary outcomes

    Trials

    (2013)
  • I. Boutron et al.

    Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of Cancer: the SPIIN randomized controlled trial

    J Clin Oncol

    (2014)
  • E.A. Ochodo et al.

    Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “spin”

    Radiology

    (2013)
  • C. Lazarus et al.

    Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention

    BMC Med Res Methodol

    (2015)
  • S.S. Cofield et al.

    Use of causal language in observational studies of obesity and nutrition

    Obes Facts

    (2010)
  • I. Tzoulaki et al.

    Assessment of claims of improved prediction beyond the Framingham risk score

    JAMA

    (2009)
  • P. Cummings et al.

    Spin and boasting in research articles

    Arch Pediatr Adolesc Med

    (2012)
  • A. Hróbjartsson et al.

    Powerful spin in the conclusion of Wampold et al.'s re-analysis of placebo versus no-treatment trials despite similar results as in original review

    J Clin Psychol

    (2007)
  • P. Sumner et al.

    The association between exaggeration in health related science news and academic press releases: retrospective observational study

    BMJ

    (2014)
  • J. Ezzo et al.

    Reviewing the reviews. How strong is the evidence? How clear are the conclusions?

    Intl J Technol Assess Health Care

    (2001)
  • J. Lau et al.

    The case of the misleading funnel plot

    BMJ

    (2006)
  • D. Cook et al.

    Colloid use for fluid resuscitation: evidence and spin

    Ann Intern Med

    (2001)
  • D.L. Weed

    Interpreting epidemiological evidence: how meta-analysis and causal inference methods are related

    Int J Epidemiol

    (2000)
  • E.H. Turner et al.

    Selective publication of antidepressant trials and its influence on apparent efficacy

    N Engl J Med

    (2008)
  • Cited by (0)

    Conflict of interest: None.

    Funding: A.Y. was funded by a grant from the Fondation pour la Recherche Médicale (FDT20140930875). D.M. is funded through a University Research Chair.

    Transparency declaration: A.Y. affirms that this article is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained [55].

    View full text