The relationship between study sponsorship, risks of bias, and research outcomes in atrazine exposure studies conducted in non-human animals: Systematic review and meta-analysis
Introduction
Results from animal studies are a critical, and often the only, input to assessing potential harm from exposure to chemicals. However, the lack of reproducibility of findings from animal research has reduced public confidence in the utility of animal experiments (van der Worp et al., 2010) and led to claims that animal research is a waste of financial resources (Macleod et al., 2014). These problems with animal research have resulted in significant debate about how to assess biases in animal studies used in systematic reviews, risk assessments and other regulatory decisions (Woodruff et al., 2011, Rooney et al., 2014, National Academies of Science, 2014). A critical component of systematic review methodology is the assessment of the risks of bias of studies that are included in the review.
Risk of bias occurs when the methodological characteristics of a study produce a systematic error in the magnitude or direction of the results (Higgins and Green, 2011). Bias can shift effect estimates to be larger or smaller. For example, in controlled human clinical drug trials, studies with a high risk of bias (such as those lacking randomization, allocation concealment, or blinding of participants and outcome assessors) produce larger treatment effect sizes, thus falsely inflating the efficacy of the test interventions, compared to studies that have these design features (Schulz and Grimes, 2002a, Schulz and Grimes, 2002b). However, biased human studies assessing the harms of drugs are more likely to report smaller estimates of adverse effects (Nieto et al., 2007).
Less is known about methodological risks of bias in animal studies, although a systematic review of instruments for assessing risks of bias in animal studies identified criteria that have been shown empirically to bias effect estimates in animal models (Krauth et al., 2013). For example, analyses of animal studies examining interventions for stroke, multiple sclerosis and trauma have shown that lack of randomization, blinding, specification of inclusion and exclusion criteria, statistical power, and failure to use comorbid animals are associated with inflated effect estimates of pharmaceutical interventions (Bebarta et al., 2003, Crossley et al., 2008, Sena et al., 2010a).
Industry funding for research and industry relationships with academic researchers pose an additional risk of bias. Considerable evidence shows a strong association between industry funding, investigator financial conflicts of interest, and biased outcomes in clinical research, even when controlling for methodological characteristics of the studies (Lundh et al., 2012). There is little evidence regarding the influence of these conflicts of interest on the outcomes of animal research (Krauth et al., 2014, Bennett et al., 2010, Abdel-Sattar et al., 2014). There are conflicting results concerning the association of industry funding and research outcomes among the small cohorts of animal studies that have been examined and further research on the influence of conflicts of interest on animal studies is needed (Bennett et al., 2010, Abdel-Sattar et al., 2014). There is controversy about whether funding source should be included in risk of bias assessments for studies included in systematic reviews (Bero, 2013).
Atrazine (6-chloro-N-ethyl-N′-(1-methylethyl)-1,3,5-triazine-2,4-diamine) is used as an herbicide. Atrazine is commonly found in drinking water in the United States. The EPA has concluded that “atrazine is an endocrine disruptor” (Agency, 2007) but not that atrazine affects amphibian sexual development (Agency, 2010). As of 2013, the EPA has not altered these conclusions (Agency, 2013). Atrazine studies are a good topic for an analysis of funding bias because concerns have been raised about the influence of industry sponsorship on the design and results of studies examining the effects of atrazine on reproductive and developmental outcomes (Hayes, 2004).
The objective of this study is to determine whether industry research sponsorship is associated with the methods, conclusions, or results of animal studies examining the effect of exposure to atrazine on reproductive or developmental outcomes. We test three specific hypotheses. First, we hypothesize that industry sponsored studies will be less likely to have conclusions indicating harm from atrazine than non-industry sponsored studies. Second, we test the hypothesis that industry sponsored studies will be less likely to report statistically significant results indicating harm from atrazine than non-industry sponsored studies. Third, we test the hypothesis that industry sponsored studies will have smaller effect estimates of harm than non-industry sponsored studies. In addition, we compare the methodological risks of bias of industry sponsored vs. non-industry sponsored studies to determine if there are differences in the methods of the studies.
Section snippets
Methods
We searched for studies that addressed the following question: “Does exposure to atrazine have adverse reproductive or developmental effects in non-human animals”? We searched for studies that had non-human animal subjects that were exposed to any dose of atrazine during any life stage. Exposure levels of atrazine were classified and adverse outcomes were grouped as described below.
Results
Our searches identified 51 studies that met the inclusion criteria (Fig. 1). There were 12 industry funded studies, 29 non-industry and 10 with no disclosures. The studies were published between 1984 and 2013 with the majority (41 studies) published between 2005 and 2010. Amphibians, fish and reptiles were most commonly studied (n = 39), followed by rats (n = 8), birds/fowl (n = 3) and mice (n = 1).
Conclusions
We tested three specific hypotheses to determine whether industry sponsorship of research is associated with the results or conclusions of non-human animal studies examining the effect of atrazine exposure on reproductive or developmental outcomes. First, we tested the hypothesis that industry sponsored studies would be less likely to have conclusions indicating harm from atrazine than non-industry sponsored studies. This hypothesis was supported; 81% of industry supported studies did not
Funding source
National Institute of Environmental Health Sciences (Grant # R21ES021028) (http://www.niehs.nih.gov). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Conflict of interest
The authors declare no competing conflicts of interest.
References (28)
- et al.
Meta-analysis in clinical trials
Control. Clin. Trials
(1986) Biomedical research: increasing value, reducing waste
Lancet
(2014)- et al.
Blinding in randomised trials: hiding who got what
Lancet
(2002) - et al.
Allocation concealment in randomised trials: defending against deciphering
Lancet
(2002) The relationship between risk of bias criteria, research outcomes, and study sponsorship in a cohort of preclinical thiazolidinedione animal studies: a meta-analysis
Evid Based Preclin Med
(2014)- Agency, E.P., Atrazine chemical summary. 2007. p....
- Agency, E.P. Atrazine updates: amphibians. 2010 July 23, 2014]; Available from:...
- Agency, E.P. Atrazine updates: atrazine evaluation process. 2013 July 23, 2014]; Available from:...
- et al.
Emergency medicine animal research: does use of randomization and blinding affect the results?
Acad. Emerg. Med.
(2003) Association between pharmaceutical support and basic science research on erythropoiesis-stimulating agents
Arch. Intern. Med.
(2010)