Article Text

Download PDFPDF
General medicine
Trial design for ineffectiveness research: a mixed-methods survey
  1. Kevin Riggs1,2,
  2. Joshua Richman2,3,
  3. Stefan Kertesz1,2
  1. 1 Medicine, University of Alabama School of Medicine, Birmingham, Alabama, USA
  2. 2 Birmingham VA Medical Center, Birmingham, Alabama, United States
  3. 3 Surgery, University of Alabama School of Medicine, Birmingham, Alabama, USA
  1. Correspondence to Dr Kevin Riggs, Medicine, University of Alabama School of Medicine, Birmingham, Alabama, USA; kriggs{at}


High-quality research demonstrating a lack of effectiveness may facilitate the ‘de-adoption’ of ineffective health services. However, there has been little debate on the optimal design for ineffectiveness research—studies exploring the research hypothesis that an intervention is ineffective. The aim of this study was to explore investigators’ preferences for trial design for ineffectiveness research. We conducted a mixed-methods online survey with principle investigators identified from A vignette described researchers planning a trial to test a widely used intervention they hypothesised was ineffective. One multiple-choice question asked whether a superiority trial or equivalence trial design was favoured, and one free-response question asked about the reasons for that choice. Free-response answers were analysed using content analysis to identify related reasons. 139 participants completed the survey (completion rate 37.5%). Overall, 56.8% favoured superiority trials, 27.3% favoured equivalence trials and 15.8% were unsure. Reasons identified for favouring superiority trials were: (1) evidence of superiority should be required to justify active treatment, (2) superiority trials are more familiar, (3) placebo should not be the comparator in equivalence trials and (4) superiority trials require smaller sample sizes. Reasons identified for favouring equivalence trials were: (1) negative superiority trials represent a lack of evidence of effectiveness, not evidence of ineffectiveness and (2) the research hypothesis should not be the same as the null hypothesis. A minority of experienced researchers favour equivalence trials for ineffectiveness research, and misconceptions and lack of familiarity with equivalence trials may be contributing factors.

  • overuse
  • ineffectiveness research
  • comparative effectiveness research
  • clinical trial design

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


  • Presented at This manuscript was presented as a poster at the 2019 Society of General Internal Medicine National Meeting. This work represents the opinions of the authors and does not necessarily represent the views of the VA or the US government.

  • Contributors All authors were responsible for the conception or design of the work; KR was responsible for data analysis and writing the initial draft manuscript. All authors were responsible for interpretation of data, revising the manuscript critically for important intellectual content and final approval of the version to be published.

  • Funding KR’s work on this manuscript was funded by AHRQ grant K12 HS023009.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Ethics approval This study was determined to be exempt by the institutional review board at the University of Alabama at Birmingham.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Data are available upon reasonable request.