Original articlesHow useful are unpublished data from the Food and Drug Administration in meta-analysis?☆
Introduction
The inclusion of unpublished data in systematic reviews or meta-analyses is controversial 1, 2. Proponents argue that exclusion of unpublished data may systematically alter the conclusions of meta-analyses, and therefore attempts should be made to include all available evidence. Opponents argue that unpublished data have not gone through the crucible of the peer-review process and are therefore more suspect than published data. One large survey concluded that most experts believe unpublished data should be included as long as the studies can be subjected to the same scrutiny as published data [1]. However, finding unpublished evidence presents a challenge because there does not exist a systematic way of accessing it.
One potentially large and easily accessible systematic repository of unpublished data on the efficacy and safety of pharmaceuticals is the United States Food and Drug Administration (FDA), which produces publicly available reviews of all studies submitted as part of the New Drug Application (NDA) process. However, the content and methodologic quality of these data have not been evaluated, nor has there been an assessment of how many studies in the FDA reviews also appear in the peer-reviewed literature (ie, the extent to which these data are “unpublished”). We offer the first such look, which we undertook as part of a meta-analysis assessing the complications of nonsteroidal anti-inflammatory drugs (NSAIDs).
Section snippets
Methods
As part of the regulatory process to approve drugs for use in the United States, sponsors must submit a NDA to the FDA. NDAs contain extensive information about studies conducted to support claims about the safety and efficacy of the drug for its proposed indications. FDA medical officers review these data and prepare reports summarizing that information that are subsequently used to evaluate the merits of the NDA. The original study material is considered proprietary and not available to the
Studies identified
We identified 4881 published titles that included the name of an NSAID or pertained to NSAIDs as a group (Figure 1). Among these, 2704 were rejected because the title clearly did not describe studies of NSAIDs among adults. An additional 377 articles were rejected after reviewing the abstract because they did not describe studies of NSAIDs among adults. Among the remaining 1800 articles, 1730 (96%) full text articles were obtained and screened. Among the articles screened, 1397 were rejected
Discussion
Our results provide some measure of reassurance to people who are concerned about the methodologic quality of manufacturer-supported studies submitted to the FDA as part of the NDA process. We found no meaningful difference between published studies (most of which were manufacturer sponsored) and manufacturer-sponsored unpublished studies on the most widely accepted criteria for methodologic quality of controlled trials, the Jadad scale. Although more published than FDA studies reported
Conclusions
Data from FDA reviews should not be excluded from systematic reviews and meta-analyses based solely on concerns about their methodologic integrity. Rather, these data can be considered a viable source for systematic reviews and meta-analyses but only after being subjected to the same methodologic scrutiny as published data.
References (32)
- et al.
Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?
Lancet
(1998) - et al.
Assessing the quality of reports of randomized clinical trialsis blinding necessary?
Control Clin Trials
(1996) - et al.
The uncertainty principle and industry-sponsored research
Lancet
(2000) - et al.
Funding clinical research
Lancet
(1999) - et al.
AGA technical reviewevaluation of dyspepsia
Gastroenterology
(1998) - et al.
Comprehensive geriatric assessmenta meta-analysis of controlled trials
Lancet
(1993) - et al.
Double-blind, placebo-controlled comparison of the safety and efficacy of orally administered etodolac and nabumetone in patients with active osteoarthritis of the knee
Clin Ther
(1995) - et al.
Comparison of the efficacy and safety of oxaprozin and nabumetone in the treatment of patients with osteoarthritis of the knee
Clin Ther
(1995) - et al.
Efficacy and safety of Etodolac and Naproxen in patients with osteoarthritis of the kneea double-blind, placebo-controlled study
Clin Ther
(1995) - Cook DJ, Guyatt GH, Ryan G, et al. Should unpublished data be included in meta-analyses? Current convictions and...
Negative studies
New Engl J Med
Dimensions of methodological quality associated with estimates of treatment effects in controlled trials
JAMA
The importance of quality of primary studies in producing unbiased systematic reviews
Arch Intern Med
A study of manufacturer-supported trials of nonsteroidal anti-inflammatory drugs in the treatment of arthritis
Arch Intern Med
Coefficient alpha and the internal structure of tests
Psychometrika
Cited by (0)
- ☆
Supported by a contract from Merck & Co. to RAND.
- †
Dr. Shekelle is a Senior Research Associate of the Veterans Affairs Health Services Research and Development.