Call to improve transparency of trials of non-regulated interventions

Current mandates

In the United States the Food and Drug Administration is responsible for assessing all regulated interventions including drugs, biologics, and medical devices before they are licensed and commercially available. The European Medicines Agency has similar responsibilities in the European Union for drugs and biologics but not devices.

Before regulated interventions can be used in trials in the US sponsors must submit an application (an investigational new drug application or an investigational device exemption) to the FDA describing the protocols for the proposed clinical trials.8 9 In the EU trial protocols must be approved by the national regulatory agencies of the countries where the trial will take place.10 Both US and EU agencies review all clinical trial data submitted by the sponsor after the clinical development plan of the product is concluded.

Both US and EU regulations mandate registration of all phase II, III, and IV clinical trials of regulated interventions—before the trial starts in the EU and within 21 days of first participant enrolment in the US—and publication or public disclosure of trial results in a public registry with free access.10 11 12 Although initially developed to comprise trials on drugs and biologics, many registries—such as ClinicalTrials.gov and other registries belonging to the World Health Organization’s platform13—currently include trials of all types of intervention. Registration is the first step towards full trial transparency. It allows patients and clinicians to find what trials are being run for a given disease or condition. Among other benefits, registration helps to deter publication bias and outcome reporting bias.

Regulatory agencies have no role in trials with non-regulated interventions, and there are no US or EU regulations requiring the registration of these types of trial. In the absence of a mandated approach some organisations have set standards for the registration of trials of non-regulated interventions. For example, the International Committee of Medical Journal Editors,14 WHO,13 and the World Medical Association’s Declaration of Helsinki,15 as well as reporting guidelines, such as CONSORT16 and SPIRIT,17 and the AllTrials campaign,18 ask for prospective registration of trials of both regulated and non-regulated interventions. But these remain voluntary.

Quantifying trials of non-regulated interventions

The lack of transparency of trials of non-regulated interventions is evident in the pattern of trial registration on ClinicalTrials.gov, arguably the best source to investigate the registration of non-regulated intervention trials. We searched ClinicalTrials.gov on 16 August 2014 and found that only 12.4% (17 308 of 139 739) of registered intervention trials pertained to non-regulated interventions (table⇓). This is a much smaller proportion than the 39% reported in published trials.19 These data are consistent with other evidence suggesting lower registration rates for trials of non-regulated interventions than for trials of regulated interventions—61% of trials published in clinical journals were found to be registered,20 compared with only 21% and 29% of trials published in psychological or behavioural21 and physical therapy22 journals, respectively.

To estimate the number of trials of non-regulated interventions that are published we assessed a random sample of articles indexed in PubMed. A search for articles published in 2013 and tagged as “randomised controlled trial” yielded 31 772 items. We screened a 1% sample (318 items) by reading the title, abstract, and full text if needed and found that 210 of these items were indeed full text articles on randomised controlled trials. We found that 38% (80 of 210) of these trials assessed non-regulated interventions (table⇑). Although this estimate is from a rough survey, it is close to the proportion estimated by Hopewell and colleagues (39%) for trials of surgical procedures, counselling, or lifestyle interventions among randomised trials published in 2006.19 So it seems that randomised controlled trials of non-regulated interventions make a major contribution to the literature, comprising roughly 40% of published randomised controlled trials.

Small sample sizes and no involvement of regulatory agencies are known risk factors for non-publication of trials of regulated interventions.23 Trials of non-regulated interventions are typically small—for example, in our random sample of 80 randomised controlled trials, the median number of participants was 92, and 22 trials enrolled fewer than 40 participants—and, by definition, lack the involvement of regulatory agencies. Thus, it may be that among launched trials far more than 39%, and perhaps the majority, pertain to non-regulated interventions.

External incentives needed

Experience with drugs and biologics24 tells us that we cannot rely solely on the willingness of investigators to publish all trial results with non-regulated interventions. External incentives may be needed to improve the registration and publication, and therefore transparency, of non-regulated interventions. Current trial registries may cater to the registration of trials of non-regulated interventions or may need to be amended to best capture some of their peculiarities. Some stakeholders in non-regulated interventions research may already be incentivising prospective registration and publication of trials, but others may need to move in this direction.

Editors can help by requiring registration of all trials to be published in their journals. Currently 28% of medical journals request trial registration,25 but most influential journals that mandate registration for publication are dominated by trials on regulated interventions. For example, of 151 trials published in New England Journal of Medicine, Lancet, or JAMA from January to June 2014, only 43 (29%) were on non-regulated interventions. Conversely, 11 of 16 trials (69%) published in The BMJ in the same period were on non-regulated interventions.26 Lack of enforcement of the registration requirement by journals leaves scope for non-registration and non-publication. Publishers could also have a role—Elsevier, for example, is assessing how to embed trial registration numbers into the peer review process.27 Both journals and publishers could require inclusion of the registration number and date registered as a prerequisite to successful online manuscript submission. However, this will not necessarily guarantee registration of a trial before it is started.

Since September 2013 research ethics committees in the UK have required registration as a condition of granting approval for trial protocols.28 Other countries could follow this example. Research ethics committees could also require that the registration number be provided in the information sheet. This would facilitate access to important trial information for participants and physicians.

Funders of trials could mandate prospective registration of non-regulated intervention trials and public access to their results. Initial funding might be suspended until trial registration, and a portion of final funding could be withheld until a manuscript has been submitted for publication. This is current practice for research financed by the UK National Institute for Health Research.29 The National Institutes of Health in the US has recently proposed a policy for registration and publication of results for all trials that it funds (wholly or in part), including those that are not subject to FDA regulation.30 31 Funding penalties or incentives may also be used for encouraging public access to raw data.

Professional associations could encourage policies on prospective registration of all clinical trials conducted by their members and public disclosure of results and respective data. All members might express in writing their commitment to follow these policies, possibly when becoming a member or for yearly membership renewals. Professional associations may also have a role in educating their members as to the importance of prospective registration and data access from all types of trial.

Coordinated effort

Regulated intervention trials are usually conducted by teams of trialists and companies that run many trials concurrently, whereas non-regulated intervention trials are often carried out by single teams, usually from academia. We need to test such interventions with a more coordinated effort, perhaps using multicentre or multinational consortia, as occurs with many regulated interventions. In contrast to regulated interventions, where industry typically controls the research agenda, such efforts should be managed by stakeholders without conflicts of interest. For example, in 2015 Stanford University will launch the Wellness Living Laboratory, a large cohort platform where informed participants (“citizen scientists”) are invited to choose whether they want to join any among dozens of randomisations pertaining to non-regulated interventions of lifestyle and other aspects of their wellness.32

The public, biomedical scientists, and clinicians require high quality evidence irrespective of the regulatory status of assessed interventions. Given the importance of non-regulated interventions, efforts to improve their transparency are overdue.