Article Text
PDF
Abstract
Purpose Trustworthy reporting of quadrivalent human papillomavirus (HPV) vaccine trials is the foundation for assessing the vaccine’s risks and benefits. However, several pivotal trial publications incompletely reported important methodological details and inaccurately described the formulation that the control arms received. Under the Restoring Invisible and Abandoned Trials initiative (RIAT), we aim to restore the public record regarding the content and rationale of the controls used in the trials.
Methods We assembled a cohort (five randomised controlled trials) described as placebo-controlled using clinical study reports (CSRs) obtained from the European Medicines Agency. We extracted the content and rationale for the choice of control used in each trial across six data sources: trial publications, register records, CSR synopses, CSR main bodies, protocols and informed consent forms.
Results Across data sources, the control was inconsistently reported as ‘placebo’-containing aluminium adjuvant (sometimes with dose information). Amorphous aluminium hydroxyphosphate sulfate (AAHS) was not mentioned in any trial registry entry, but was mentioned in all publications and CSRs. In three of five trials, consent forms described the control as an ‘inactive’ substance. No rationale for the selection of the control was reported in any trial publication, register, consent form, CSR synopsis or protocol. Three trials reported the rationale for choice of control in CSRs: to preserve blinding and assess the safety of HPV virus-like particles as the ‘safety profile of (AAHS) is well characterised’.
Conclusions The stated rationale of using AAHS control—to characterise the safety of the HPV virus-like particles—lacks clinical relevance. A non-placebo control may have obscured an accurate assessment of safety and the participant consent process of some trials raises ethical concerns.
Trial registration numbers NCT00092482, NCT00092521, NCT00092534, NCT00090220, NCT00090285.
- paediatric infectious disease & immunisation
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
INTRODUCTION
Randomised, placebo-controlled clinical trials are considered the gold standard for evaluating new vaccines. To evaluate the vaccine’s efficacy and safety, the manufacturer of quadrivalent human papillomavirus (qHPV) vaccine conducted multiple clinical trials including around 30 000 females and males. The vaccine was approved in 2006. Today, citing the vaccine’s efficacy and extensive safety profile, increasing HPV vaccine coverage is a high priority for many national governments and the WHO.1–3
Multiple trials of qHPV vaccine are reported as ‘placebo-controlled.’4–8 However, participants in the ‘placebo’ arms of these pivotal trials received an injection-containing amorphous aluminium hydroxyphosphate sulfate (AAHS), a proprietary adjuvant. AAHS is used in qHPV vaccine to boost immune response, but the rationale for adding AAHS to the ‘placebo’ is not reported in publications of these trials4–8 and is contrary to the advice of the public health bodies and regulators. The WHO, for example, recommends control recipients in trials testing an unlicensed, experimental vaccine receive either an inert substance or an approved efficacious vaccine.9 Similarly, the European Medicines Agency (EMA), which provides specific guidance on the clinical evaluation of vaccines, recommends the use of a licensed vaccine without an effect on the target disease if a placebo cannot be used.10
In addition, the efficacy and safety analyses of these five qHPV vaccine trials were conducted as if the trials were controlled with inert placebo when they were not. None of the key publications for these trials, which have been used to inform regulatory and health decision making, appear to discuss how AAHS-containing control could affect the interpretation of results.4–8
We consider the omission in journal articles, of any rationale for the selection of AAHS-containing control, to be a form of incomplete reporting of important methodological details, and believe the rationale must be reported. We also consider that the use of the term ‘placebo’ to describe an active comparator like AAHS inaccurately describes the formulation that the control arm participants received, and constitutes an important error that requires correction. If trial participants were told they could receive ‘placebo’ (widely defined as referring to an ‘inactive’11 12 or ‘inert’9 substance) without being informed of all non-inert contents of the control arm injection, this raises ethical questions about trial conduct as well.
In January and February 2019, we publicly declared our intention to systematically correct the record for these trials13 14 in accordance with the principles of the Restoring Invisible and Abandoned Trials (RIAT) initiative, of which some of us are founders.15 RIAT was founded with the intent of restoring the text of published biomedical articles where there is clear evidence of distortion or omission (as in the case of reporting bias or non-publication of clinical trials) and the original trial authors and/or sponsors fail to take any corrective action (thereby abandoning their responsibilities). While RIAT projects to date have focused on restoring trials through a reanalysis of all study data for a given trial,16–19 this project differs in its scope in two ways: first, we are focused on multiple trials in a single manuscript; second, this restoration is restricted to a specific aspect of the trials’ methodology (the choice of control). We believe this ‘focused’ RIAT is important given the considerable implications that the choice of control has on the interpretation of trial results, and the fact that the problem is not confined to a single trial but rather spans a trial programme. Other criticisms of the reporting in qHPV trial publications (e.g., selective outcome reporting) have been made,20 but we do not address those concerns here. Consistent with the RIAT approach, we wrote to the corresponding authors of all five publications as well as the sponsor requesting that they correct their publications. None contested our concerns nor indicated any intention to correct the record. As such, we consider these trials abandoned and hereby restore the record using data obtained under freedom of information requests to the EMA according to our prespecified protocol.21
Subjects and methods
Research design
Review of text describing the content and rationale for choice of control across published and unpublished data sources.
Data sources, searches and restoration sample
Since 2010, transparency policies at the EMA have allowed independent investigators to obtain access to clinical study reports (CSRs) and other clinical data once protected as commercially confidential.22 CSRs are reports structured according to international guidelines,23 written by sponsors and submitted to regulators, generally as part of marketing authorisation applications, and provide far more detail than journal publications.24 In 2015, the EMA expanded its efforts to proactively publish CSRs to its website following regulatory decisions,25–27 a practice Health Canada also adopted as of 2019.28
The cohort of trials potentially eligible for restoration was all clinical trials of qHPV vaccine and nonavalent HPV vaccine for which we obtained CSRs from the EMA. These were received in response to a May 2014 freedom of information request. The process of obtaining CSRs has been previously described in an Index study29 and Analysis article.30 All trials evaluating qHPV vaccine and nonavalent HPV vaccine for which CSRs were obtained by 1 November 2018 were eligible for inclusion in this restoration.
Each CSR was reviewed by two independent assessors (with conflicts resolved by discussion or third party) for possible inclusion against the following criterion: trial randomised control group participants to aluminium-containing adjuvant (as a control). We excluded trials which used an approved vaccine as the control.
Identification of matching trial publications
We aimed to identify the most impactful, company-authored trial publication for all trials in our sample using the following approach:
We identified the NCT number for each trial by searching for the trial ID, obtained from the CSR, on ClinicalTrials.gov. We then compared trial characteristics (study population, intervention, comparator, name of primary outcome measure(s) and number of participants) with the CSR and a published index of HPV vaccine trial programme29 to confirm that the correct NCT record was identified.
From the ClinicalTrials.gov registry entry for each trial, we recorded all publications (in all categories) listed under the ‘Study Details’ tab.
Using the DOI of these publications, we identified the most cited publication of each trial using the Scopus database and obtained the article’s full-text PDF and supplementary material.
This process was carried out by two independent reviewers with discrepancies resolved by discussion or third-party consultation.
Determination of under-reporting and misreporting
For each trial using a concurrent aluminium-containing adjuvant as a control, we reviewed the publication to determine the potential presence of under-reporting of the rationale for using such a control and misreporting of the control intervention (i.e., describing the control as ‘placebo’).
Data extraction
For each trial using six data sources (CSR synopsis, CSR main body, trial protocol, informed consent form, trial publication and trial register entry), we recorded (1) the phrases used to describe the comparator to qHPV vaccine; (2) the rationale for using aluminum-containing adjuvant as a control, if present and (3) all listed contents (ingredients) of formulation received by intervention and control arms.
To enable an exhaustive list of all descriptions of the comparator across all document types, especially CSRs which were often thousands of pages long, we searched for the terms placebo, control, comparator, alum, AAHS, adjuvant and MAA (Merck Aluminum Adjuvant). We categorised the verbatim phrases into three broad categories: phrases that described a placebo (with no mention of adjuvant), phrases that described a placebo containing adjuvant and phrases describing a placebo containing adjuvant as well as adjuvant dose information. We recorded the frequency of terms used to describe the formulation of the control arm and its ingredients, by data source.
Extractions were independently carried out by two assessors. There were no discrepancies, and a consolidated analysis data set was created.
Results
We obtained 14 trials from the EMA (eight qHPV vaccine, six nonavalent HPV vaccine), of which five met our prespecified inclusion criteria (V501-012, V501-013, V501-015, V501-019 and V501-020; figure 1). All included trials compared qHPV vaccine to aluminium-containing adjuvant. Corresponding register and publication information for each trial are listed in table 1.
Summary of Trial Search. CSR, clinical study reports; EMA, European Medicines Agency; HPV, human papillomavirus.
Characteristics of trials being restored
Describing the control arm
There was heterogeneity in the verbatim phrases used to describe the control arm across all trials and types of documents (table 2).
Phrases used to describe the control arm (proportion of trials)
The publication abstracts and register entries for four trials only referred to the control arm as ‘placebo’ or ‘matching placebo’ (Garland et al8/NCT00092521, FUTURE II Study Group4/NCT00092534, Muñoz et al5/NCT00090220, Giuliano et al6/NCT00090285). The publication abstract and register entry for one trial (Garland et al7/NCT00092482) did not mention the existence of a control arm.
The informed consent forms for two trials (V501-019 and V501-020) did not provide participants with any information regarding the contents of the control arm. The informed consent forms for three trials (V501-012, V501-013 and V501-013) informed participants that they may be exposed to an ‘inactive solution (containing aluminium 225 mcg/dose)’.
The CSRs of all trials described the control arm as a ‘placebo’-containing 225 mcg of aluminium. While all CSRs stated the type of aluminium adjuvant (AAHS), the information was not reported in a consistent location (i.e., CSR synopsis, main body and protocol). A summary of reported ingredients, by data source, is presented in table 3.
Ingredients of control arm, by trial document type
We assessed the frequency of terms used to describe the control for the trial with the most highly cited publication in our study (V501-015). In the publication, the control was referred to as a placebo (with no mention of adjuvant) 67 times and one time as a placebo with adjuvant.4 In the CSR, the control was referred to as a placebo 1450 times versus 326 times as placebo with adjuvant, and across six informed consent forms (US and international versions of the main consent form, child assent form and pregnancy consent addendum), 54 versus 2, respectively. Finally, in the register entry, the control was referred to as a placebo 50 times versus 0 times as a placebo with adjuvant.
Rationale for aluminium-containing control
The rationale for the selection of an AAHS-containing control was reported in the CSR of three trials (V501-012, V501-013 and V501-015). No rationale was provided in any document for two trials (V501-019 and V501-020). In all five trials, the protocol, publication, register entry, informed consent form and CSR synopsis did not contain a rationale for the selection of the control (table 4).
Rationale for aluminium-containing control (verbatim quotes)
The three studies that reported a rationale for the selection of an AAHS-containing control provided identical reasons: (1) to preserve blinding and (2) to assess the safety of HPV virus-like particles given that the ‘safety profile of (AAHS) is well characterised’.
Discussion
On its website, qHPV vaccine’s sponsor defines a placebo as ‘an inactive pill, liquid, or powder that has no treatment value’.11 This definition is consistent with the decades old notion of placebos as pharmacologically inert substances used to obtain unbiased assessments in experimental research.31
The five pivotal trials of qHPV vaccine that were the subject of this restoration used a control with aluminum-containing adjuvant. This was stated in the journal publications of all trials. But contrary to what the original investigators reported, the trials were not placebo-controlled. The inclusion of AAHS in the control arm means that the control should not be reported as a placebo since AAHS is not inactive. While the original FDA-approved prescribing information in 2006 reported the AAHS-containing control as ‘placebo’, this was updated in September 2008, and all such references were removed and replaced with ‘AAHS control’.32–34
Inaccurate use of the term placebo is not confined to qHPV vaccine trials. As no regulations govern placebo composition, researchers have documented a diversity in what gets labelled a placebo in research, and that depending on the experimental arm, some so-called placebos may have effects that influence study outcomes.35
With respect to adjuvants in vaccines, the FDA has noted that ‘adjuvants have their own pharmacologic activity, which may affect both the immunogenicity and the safety of vaccines. Adverse reactions may include local reactions such as pain, swelling, injection site necrosis, and granulomas. Systemic reactions may include nausea, fever, arthritis, as well as potential immunotoxic reactions. Unexpected, rare events may also occur’.36
The FDA has also stated that ‘the evaluation of safety of an adjuvanted vaccine needs to include special safety considerations’, which may include data ‘derived by comparing the adjuvanted vaccine to a placebo or the unadjuvanted vaccine antigen, if feasible’.36 The extent to which the evaluation of qHPV vaccine included such special safety considerations is debatable given the presence of adjuvant in the control.
The implications of AAHS controlled trials on an accurate assessment of the safety and efficacy of qHPV are important. It is possible that the presence of AAHS in both arms of the trials masked AAHS’s reactogenic potential thereby obscuring any differential rates in harms, leading to an underestimate in harms. The qHPV vaccine trial results are consistent with these statements. Across the five trials using AAHS in the control, the absolute risk of injection-site adverse events was only 6%–13% higher in qHPV vaccine patients. By contrast, in study V501-018, which used a control lacking AAHS, the absolute risk increase was 25% (table 5). It is also possible that AAHS’s stimulating properties (which to our knowledge have never been studied on their own) provoked an immune response which may have been protective against endpoints studied in the trial, thereby leading to an underestimation of the effectiveness of qHPV vaccine. Because all the five trials lacked a true inert comparator arm, the trial data cannot resolve these questions.
Injection-site adverse experiences between Gardasil and control arms
It is not clear whether other trials can establish qHPV vaccine’s true safety and efficacy profile. The FDA instructed the sponsor to conduct a safety study with a control not containing aluminium, according to the company’s submission to Japanese regulators,37 and the CSR for trial V501-018 states that it was the only qHPV vaccine trial to do this. In contrast to other studies, the FDA medical officer who reviewed the sponsor’s marketing application referred to this trial’s control as a ‘true placebo’.38 However, V501-018’s short follow-up of control arm participants (30 months), small sample (597 receiving control) and younger age relative to other trials makes it inadequate to resolve safety questions.
Our study raises serious questions about the ethics of consent in the trials. The informed consent forms for the trials described the placebo as an ‘inactive solution’ and ‘a dose that contains no active ingredients’. This likely misled many trial participants who would have reasonably expected the control did not contain a pharmacologically active adjuvant.
Our restoration makes public, for the first time, the sponsor’s rationale for using AAHS as a control. The manufacturer gave two reasons: first, to ensure that the control was not visually distinguishable, and second, to assess the safety profile of the HPV virus-like particles as the ‘safety profile of Merck’s aluminium adjuvant is well characterised’.
The fact that the qHPV vaccine and nonavalent HPV vaccine clinical trial programme included two randomised, double-blind trials that did not use an AAHS-containing control demonstrates that there was no necessity to use AAHS to maintain blinding. In these trials (V501-018 and V503-006), unblinded study personnel who were otherwise not involved in the study prepared and administered injections.
Second, the safety profile of AAHS is not well characterised. We are unaware of any randomised trial directly evaluating AAHS against placebo; a network meta-analysis is underway that aims to use extant randomised controlled trial (RCT) data to determine the safety profile of the AAHS component of qHPV vaccine.39 Moreover, while other vaccines contain AAHS (e.g., Recombivax and VAQTA), such vaccines cannot be used to characterise AAHS safety as ‘it cannot be assumed that an adjuvant that is safe in one vaccine with a given antigen will be safe when added to another vaccine’.40 The unknown consequences of aluminum-containing adjuvants, more generally, is concerning. A Cochrane review is currently underway to determine whether any RCTs exist which evaluated the effects of aluminium adjuvants compared with placebo or no intervention.41
Finally, the stated rationale of using AAHS control, to characterise the safety of the HPV virus-like particles, lacks clinical relevance. The clinically relevant question is what are the effects (benefits and harms) of qHPV vaccine—the whole product, not one of its components.
As crucial documents written before trials commence, it is concerning that study protocols did not include a rationale for the use of AAHS control. Protocols for V501-015, V501-019 and V501-020 stated: ‘To provide an appropriate control for the Quadrivalent HPV (Types 6, 11, 16, 18) L1 VLP Vaccine, the placebo used in this study will be Merck standard aluminum diluent (225 µg alum) in normal saline, USP (NaCl 0.9%)’. But the documents did not explain what made the adjuvant ‘appropriate’. We were also concerned that ClinicalTrials.gov entries described the control with a single word (‘placebo’, not even mentioning AAHS), and in one case did not even list the control.
The Consolidated Standards of Reporting Trials (CONSORT) guidelines for reporting RCTs call for study interventions for each group to be described ‘with sufficient details to allow replication, including how and when they were actually administered’.42 We think CONSORT should also ask investigators to document their rationale for selecting these interventions, particularly the choice of control. Such reporting would allow for more transparency and a more informed discussion around the appropriateness of chosen controls, and is consistent with regulatory advice to industry that ‘the choice of control group is always a critical decision in designing a clinical trial’.43
Limitations
Across all study documents, the only ingredients we found listed in the control were AAHS and normal saline (NaCl 0.9%). However, there may have been other ingredients. According to qHPV vaccine’s prescribing information, each dose of vaccine contains ‘9.56 mg of sodium chloride, 0.78 mg of L-histidine, 50 mcg of polysorbate 80, 35 mcg of sodium borate, <7 mcg yeast protein/dose and water’, in addition to AAHS and HPV virus-like particles.32–34 To test HPV virus-like particles, as the manufacturer stated was its intention in using an AAHS control, the control would logically have also included these other ingredients in addition to AAHS. We recommend that reports of trials (across publications, protocols, CSRs and registers) should contain a complete list of ingredients of all trial arms (if necessary, in supplementary materials when space limitations apply).
Our study may not represent a complete list of qHPV vaccine trial publications that under-report and misreport the control. In our screening, we identified a trial (V501-005) with identical problems to the trials included in this restoration; however, this trial was excluded from our analysis as it reported on the use of a monovalent experimental HPV vaccine which was never licensed.44
Conclusions
The sponsor’s stated rationale of using AAHS control in qHPV vaccine trials—to characterise the safety of the HPV virus-like particles—lacks clinical relevance, as the clinically relevant issue is to characterise the safety profile of the whole qHPV vaccine. The non-placebo control used in qHPV vaccine trials may have obscured an accurate assessment of safety of qHPV vaccine and the participant consent process of some trials raises ethical concerns.
References
Footnotes
Contributors PD: conceptualisation; data curation; formal analysis; funding acquisition; investigation; methodology; project administration; resources; software; supervision; validation; visualisation; roles/writing: original draft; PD had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. LS: conceptualisation; data curation; methodology; data curation; writing: review and editing. TJ: conceptualisation; formal analysis; funding acquisition; investigation; methodology; roles/writing: original draft. KH: investigation; writing: review and editing. HL: investigation; writing: review and editing. FB: investigation; writing: review and editing. OS: data curation; writing: review and editing. MJ: investigation; writing: review and editing.
Funding Laura and John Arnold Foundation
Competing interests The Laura and John Arnold Foundation funds the RIAT Support Center which supports the salaries of Doshi, Bourgeois, Hong, Jefferson, Jones, Shamseer (until 2018) and Spence. In addition, PD has received travel funds from the European Respiratory Society (2012) and Uppsala Monitoring Center (2018); grants from the Laura and John Arnold Foundation (2017–2021), American Association of Colleges of Pharmacy (2015), Patient-Centered Outcomes Research Institute (2014–2016), Cochrane Methods Innovations Fund (2016–2018) and UK National Institute for Health Research (2011–2014); and is an editor at the BMJ and unpaid member of the Reagan-Udall Foundation for the FDA. TJ was a recipient of a UK National Institute for Health Research grant for a Cochrane review of neuraminidase inhibitors for influenza. In addition, TJ receives royalties from his books published by Il Pensiero Scientifico Editore, Rome and Blackwells. TJ is occasionally interviewed by market research companies about phase I or II pharmaceutical products. In 2011–2013, TJ acted as an expert witness in litigation related to the antiviral oseltamivir, in two litigation cases on potential vaccine-related damage (including the vaccine Pandemrix (2015–2017)) and in a labour case on influenza vaccines in healthcare workers in Canada. He has acted as a consultant for Roche (1997–1999), GSK (2001–2002), Sanofi-Synthelabo (2003) and IMS Health (2013). In 2014, he was retained as a scientific adviser to a legal team acting on oseltamivir. TJ has a potential financial conflict of interest in the drug oseltamivir. In 2014–2016, TJ was a member of three advisory boards for Boehringer Ingelheim. TJ was holder of a Cochrane Methods Innovations Fund grant to develop guidance on the use of regulatory data in Cochrane reviews. TJ was a member of an independent data monitoring committee for a Sanofi Pasteur clinical trial on an influenza vaccine. Between 1994 and 2013, TJ was the coordinator of the Cochrane Vaccines Field. TJ was a cosignatory of the Nordic Cochrane Centre Complaint to the European Medicines Agency (EMA) over maladministration at the EMA in relation to the investigation of alleged harms of HPV vaccines and consequent complaints to the European Ombudsman. TJ is coholder of a John and Laura Arnold Foundation grant for development of a RIAT Support Center (2017–2020) and Jean Monnet Network Grant, 2017–2020 for the Jean Monnet Health Law and Policy Network. TJ is an unpaid collaborator to the project Beyond Transparency in Pharmaceutical Research and Regulation led by Dalhousie University and funded by the Canadian Institutes of Health Research (2018–2022). TJ is a consultant to Illumina LLC (2019-current). MJ was a coinvestigator on a UK National Institute for Health Research grant for a Cochrane review of neuraminidase inhibitors for influenza; was a corecipient of a Cochrane Methods Innovations Fund grant to develop guidance on the use of regulatory data in Cochrane reviews; and is a paid consultant on a John and Laura Arnold Foundation grant for development of an RIAT Support Center (2017–2020). LS, HL, FB and KH: no competing interests to declare. OS received the Maryland CERSI Scholar award from the Food and Drug Administration (grant #1U01FD005946) and the PhRMA Foundation’s Predoctoral Fellowship in Health Outcomes.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Our completed extraction sheets are publicly available on the Open Science Framework (http://doi.org/10.17605/OSF.IO/KPE2T) and CSRs are available upon request.