The article “Drug discovery today: no molecules required” (2018) (by Panchin AY, Khromov-Borisov NN, Dueva EV) questions the imperfections in reviewing scientific publications. The authors use the example of publications on released-active drugs (RA-drugs) as a basis for the paper and their accusations against journal editorial boards. The authors postulate that these drugs do not contain any active ingredients due to the technology used and therefore come to the conclusion that the reviewers and journal editors who have published articles on RA-drugs carried out their work appallingly and made gross mistakes. So the authors state «…the vast amount of flawed publication claiming therapeutic properties and physiological effects of drugs with no active components can highlight the problems of peer-review standards and policies in biomedical journals».
This particular conclusion is based on approximate theoretical calculations and not on experimental evidence: «If one assumes a 1M initial antibodies concentration (though this is usually not disclosed in release-activity papers) and takes into account Avogadro’s constant (~6.02×10^23 mol^(−1)), then even C12 dilutions are unlikely to contain any antibody molecules». In their article, the authors present these calculations as the main argument to complain to journals. It is noteworthy that in the article the authors do not provide any data or references to any studies conducted by themselves. Neither do the authors provid...
The article “Drug discovery today: no molecules required” (2018) (by Panchin AY, Khromov-Borisov NN, Dueva EV) questions the imperfections in reviewing scientific publications. The authors use the example of publications on released-active drugs (RA-drugs) as a basis for the paper and their accusations against journal editorial boards. The authors postulate that these drugs do not contain any active ingredients due to the technology used and therefore come to the conclusion that the reviewers and journal editors who have published articles on RA-drugs carried out their work appallingly and made gross mistakes. So the authors state «…the vast amount of flawed publication claiming therapeutic properties and physiological effects of drugs with no active components can highlight the problems of peer-review standards and policies in biomedical journals».
This particular conclusion is based on approximate theoretical calculations and not on experimental evidence: «If one assumes a 1M initial antibodies concentration (though this is usually not disclosed in release-activity papers) and takes into account Avogadro’s constant (~6.02×10^23 mol^(−1)), then even C12 dilutions are unlikely to contain any antibody molecules». In their article, the authors present these calculations as the main argument to complain to journals. It is noteworthy that in the article the authors do not provide any data or references to any studies conducted by themselves. Neither do the authors provide references to outside studies. Hence, the question arises as to whether theoretical calculations may be sufficient to support the accusations made by the authors. In this regard, the available experimental data refute the authors' theoretical assumptions that in principle ultra-high dilutions cannot contain molecules. So, Chikramane PS et al. (2012) wrote in the pages of American Chemical Society journal ‘Langmuir’: «However, physicochemical studies of these solutions have unequivocally established the presence of the starting raw materials in nanoparticulate form even in these extreme (super-Avogadro, >10^23) dilutions. In this paper, we propose and validate a hypothesis to explain how nanoparticles are retained even at such enormous dilution levels» [Chikramane PS, Kalita D, Suresh AK, Kane SG, Bellare JR. Why extreme dilutions reach non-zero asymptotes: a nanoparticulate hypothesis based on froth flotation. Langmuir. 2012; 28(45):15864-75. doi: 10.1021/la303477s]. Consequently, the main thesis on which the authors base their criticism is rendered invalid at least because the subject is complicated and should not be simplified as Panchin AY et al did.
The article itself contains a number of errors. I shall provide some examples:
In Table 1, in the paragraph about the article in Nutrition & Diabetes it is stated: “There is no mention that Epstein is the CEO and founder of MMH”. This is presented as if a conflict of interest is being withheld. However, the article's “Conflict of interest” section (Nutrition & Diabetes) in fact discloses information about the employment of all the co-authors of the article: “This work was funded by (OOO ‘NPF ‘MATERIA MEDICA HOLDING’, Russian Federation). EAG, SAT, EVK and OIE have an affiliation to the commercial funders of this research study (OOO ‘NPF ‘MATERIA MEDICA HOLDING’, Russian Federation) and JN is employed by a commercial company (Zen-Bio Inc., USA).” (NOTE: OIE – Oleg I Epstein). The concealment of a conflict of interest claimed by Panchin AY et al. therefore has no basis in fact.
Furthermore, in Table 1, the paragraph concerning the same article in Nutrition & Diabetes states the following: “The authors failed to provide images of acquired blots, making it impossible to check the validity of their analysis (online supplementary letter 2)”. Actually, the Editorial Board of Nutrition & Diabetes was provided with images of acquired blots during the peer-reviewing process. However, due to the format of the article (Short Communications) and limitations in the number of figures and tables, which could be displayed in the article, the images of blots just did not get into the published article. Thus, Panchin AY et al. statement is absolutely wrong.
As for the repeated statistical analysis based on the article in the Journal of Diabetes Research (Table 1 and online supplementary letter 1) conducted by the authors, I report that this analysis contains enough errors to consider it inadequate. Regression analysis used in supplementary letter 1 makes several assumptions. There must be a linear relationship between day and plasma glucose level, residuals should be normally distributed (this information cannot be accessed from the tables in the article), independent variables are not highly correlated with each other and the variance of errors should be similar across all independent variables. Violation of these assumptions makes use of regression analysis inappropriate. Thus, the results in supplementary letter 1 are not reliable and cannot serve as evidence of the absence of significant differences. In respect to the statistical analysis presented by us in the original article (Journal of Diabetes Research), there are no statistical adjustments for multiple comparisons because the null hypothesis of the study was to compare Subetta with control at certain time point but not in dynamic and to compare effect within the group. That is why Mann-Whitney test was done for the inter group comparison and Wilcoxon test was done for the comparison with respective control. The application of the similar approach could be easily found in articles of other research groups [for instance, Shimoda M. et al. Diabetologia. 2011;54(5):1098-108; Ropero A.B. et al. PLoS One. 2012;7(4):e34650; Gruzman A. et al. J Cell Mol Med. 2012;16(3):594-604; Chen C. et al. Dis Model Mech. 2014;7(6):723-30; Kim K.S. et al. Sci Rep. 2015;5:18325]. In this case FDR adjustment and regression analysis used in supplementary letter 1 to evaluate research results are not only inappropriate but also uninformative and meaningless.
Also, there are inaccuracy in the article as well. The article states “The champion in publishing research on RADs is a Russian journal, Bulletin of Experimental Biology and Medicine (English edition is by Springer Nature)… Notably, at least 48 of them were published in issue 135 Supplementary 7 (2003) (ref.40). It might be not surprising that Epstein was the editor of this supplementary issue (ref.41)”. In the process, the authors fail to mention that it was a collection of works on studying drugs based on ultra-low doses, as indicated in the Russian edition (http://www.iramn.ru/journal/bebm/2003/bbm031p.htm; in Russian). In other words, this collection is a special edition of the journal supplement. It is not surprising that O.I. Epstein, a scientist who studies this phenomenon, was one of the editors of the special edition. This detail is critical to understanding why so many works by one author were published in the journal. However, this detail was simply swept under the carpet by Panchin AY et al.
Let me consider another example of how information is presented in a favourable light for the authors. The article states “Drug Discovery Today—one of the leading journals in the field—published a review coauthored by Epstein about the structure and dynamics of the IR16 that claims the miraculous physiological activity for the RAD subetta…”. This review is devoted to the structure and dynamics of the insulin receptor rather than the study of Subetta. This drug is listed among other drugs that target the insulin receptor and the associated signalling pathway. Therefore, the review in Drug Discovery Today does not in any way claim to be “miraculous physiological activity for the RAD Subetta”, as the authors put it.
Finally, the article does not stipulate how the six articles (presented in Table 1) were chosen for analysis. Any data analysis must be based on clear criteria according to which material is included and excluded. Otherwise, materials may be selected in order to confirm the author's point of view. As the authors themselves write, over 100 English-language articles have been published about RA-drugs. Among the publications there are many articles about the results of clinical studies of RA-drugs, including those in specialised English-language scientifically peer-reviewed journals. It should be noted that it is only the results of clinical studies that allow discussion of a particular drug's efficacy. The authors did not include an analysis of such articles into their «analysis». However, sweeping conclusions were made about RA-drugs' lack of efficacy based on the analysis of six non-clinical articles, of which:
1. one is devoted to the development of the ELISA method, and not to the evaluation of RA-drugs’ pharmacological activity;
2. one is a review of the structure of an insulin receptor, rather than an evaluation of pharmacological activity of RA-drugs;
3. two articles are in vitro studies of a possible mechanism of action, and yet again not an assessment of efficacy of RA-drugs;
4. and only two articles are devoted to the evaluation of pharmacological activity of RA- drugs.
Thus, the generalisations that the article's authors make based on their selective analysis cast doubt on their validity and impartiality.
The authors of this publication, whilst reproaching the co-authors of articles concerning RA-drugs for concealing a conflict of interest, in fact violate conflicts of interests themselves. This is impermissible in principle, as authors who have taken it upon themselves to judge other articles must fully comply with the rules of the journal, otherwise the publication resembles a caricature. The authors stated in the article: “Competing interests: None declared”.
In accordance with “ICMJE Form for Disclosure of Potential Conflicts of Interest” that all authors must complete (http://www.icmje.org/coi_disclosure.pdf), they must indicate if there are “relationships or activities that readers could perceive to have influenced, or that give the appearance of potentially influencing, what you wrote in the submitted work”.
In accordance with ICMJE uniform declaration of competing interests, authors should disclose (https://authors.bmj.com/policies/competing-interests/) “Non-financial associations that may be relevant to the submitted manuscript”.
However, one of the authors of the article, Khromov-Borisov NN, is a defendant in a defamation lawsuit brought by Materia Medica (June 28, 2018). Materia Medica has sued Khromov-Borisov NN, one of the article co-authors, for harming business reputation after he published inaccurate and discrediting information about the Company’s activities in the mass media (newspaper article in Troitsky Variant – Science; it is ‘journalistic’ but not scientific article) (А40-148616/2018) (http://kad.arbitr.ru/Card/a74a5f06-fff8-4b4a-b5c0-3f97c33d36c1; in Russian). This therefore raises serious doubts over the objectivity and impartiality of one of the authors of the article, who also carried out the data analysis presented in supplementary letter 1. So, here might be a personal dislike and personal interest in discrediting Materia Medica and its scientific activities.
In addition, according to the BMJ policy (https://authors.bmj.com/policies/competing-interests/): “Examples of competing interests include the following: … other personal or professional relationships that may influence or appear to influence”.
The authors concealed that Panchin AY and Khromov-Borisov NN are members of the Commission of the Russian Academy of Sciences on Pseudoscience and Research Fraud (http://klnran.ru/about/sostav/; in Russian) (hereinafter “The Commission”). This membership may influence or appear to influence on the content of the article taking into account that the Commission published “Memorandum No.2. Homeopathy as Pseudoscience” (2017) (hereinafter “Memorandum”) in which not only homeopathy, but also RA-drugs was criticised.
The information worth mentioning for clear understanding the status of the Commission and the Memorandum: after its release, Prosecutor's check revealed that the Memorandum is not an official position of Russian Academy of Sciences (RAS), but is a personal scientific opinion of the Memorandum's authors. Moreover, in the court case №А40-217689/2017 (http://kad.arbitr.ru/Card/c1c31039-1951-432c-9625-99a6ef871676; in Russian) regarding the Memorandum it was proved that:
1. the Commission is not a structural unit of RAS;
2. the Commission shall conduct any researches under an order of the Presidium of the RAS that did not give the order for the preparation of the Memorandum;
3. the experts' opinion, which serves as basis for the Memorandum, was signed by the Head of this Commission and by 7 members of the Commission (including Panchin AY; the full number of its members is 63);
4. all persons, who signed the Memorandum, are not employed by RAS.
Incidentally, Panchin AY is a member of the board of the Evolution Fund (https://evolutionfund.ru/about#council; in Russian), which supported the preparation of the Memorandum (http://klnran.ru/2017/02/memorandum02-homeopathy/; in Russian). The fund provides support for publishing Panchin's books (https://evolutionfund.ru/project/1; in Russian), some containing negative information regarding Materia Medica’s drugs (Panchin A. Zashchita ot temnykh iskusstv. Putevoditel' po miru paranormal'nykh yavleniy / A. Panchin — «Corpus (AST)», 2018 — 396p. (ISBN: 978-5-17-982690-3) [Library of the Evolution Fund]). Several members of the board of the Evolution Fund (https://evolutionfund.ru/about#council; in Russian) are the members of Editorial Board of the newspaper Troitsky Variant – Science (https://trv-science.ru/about/editors/; in Russian). This newspaper, as well as Khromov-Borisov NN, is a defendant in a defamation lawsuit brought by Materia Medica (June 28, 2018) (see above). Thus, the above-mentioned information should be disclosed in accordance with “ICMJE Form for Disclosure of Potential Conflicts of Interest” which indicates the requirement to disclose “Relevant financial activities outside the submitted work”: “This section asks about your financial relationships with entities in the bio-medical arena that could be perceived to influence, or that give the appearance of potentially influencing, what you wrote in the submitted work. You should disclose interactions with ANY entity that could be considered broadly relevant to the work. For example, if your article is about testing an epidermal growth factor receptor (EGFR) antagonist in lung cancer, you should report all associations with entities pursuing diagnostic or therapeutic strategies in cancer in general, not just in the area of EGFR or lung cancer”.
I would like to call attention to the fact that one of the main reason why our article was retracted from PLOS ONE [Gavrilova ES, Bobrovnik SA, Sherriff G, et al. Novel approach to activity evaluation for release-active forms of anti-interferon-gamma antibodies based on enzyme-linked immunoassay. PLoS One 2014;9:e97017.] was incomplete conflict of interest information disclosure. Moreover, I would like to remind that the affiliations of all authors were stated in the article, and the fact that the drug used for the method development is the commercial product of the pharmaceutical company is mentioned in the “Materials and Methods” section. During the review process neither the editors, nor the reviewers suggest that this information should also be mentioned in the “Conflict of interest” section. Thus, information about the authors' employment and the presence of a commercially available drug sold by the company was, in fact, provided in the article, albeit not in a separate section. At the same time, no information is presented in the Panchin AY et al. article concerning possible conflict of interests, which would allow Editor board, reviewers and readers to formulate their own position with regard to the published material and its authors.
Concluding remarks.
Thus, the article “Drug discovery today: no molecules required” (2018) (by Panchin AY, Khromov-Borisov NN, Dueva EV) is devoted to the imperfection of reviewing scientific publications. The authors use the example of publications on RA-drugs as the basis for their conclusions and accusations against the editorial boards of various journals theoretically postulating that the drugs do not contain active ingredients, making mistakes and inaccuracies in the presentation of materials, concealing a potential conflict of interest and other information that may be useful for readers to evaluate this article.
Additional information.
I do not give a detailed description of work dedicated to RA-drugs, as the main topic of the article is different. This is also due to the fact that Panchin AY et al. promote them rather successfully, not only in the scientific domain but also in the non-scientific. The article fleetingly mentions the scale of the research, which includes the entire range of works from research at the early stages of drug development to clinical trials. It reflects a systematic approach to research and development, as well as potentially supporting the conclusions of the RA-drug developers on their efficacy. It is only possible to comment on and add to some of the points:
1. Availability of RA-drugs both in Russia and abroad would be impossible without the proper evaluation of quality, efficacy and safety of the drugs carried out by Ministries of health of these countries followed by authorizing the drugs on the basis of the evaluation conducted. It is well known that evaluation of a drug is based on the main principles: legality, independence of an evaluator, objectivity, comprehensiveness and completeness of research carried out using modern science and technology. The plans to expand drug’s presence in Europe and the US are based on scientific consultations with experts from the FDA, MHRA, and EMA, as well as their positive conclusions based on the analysis of data provided by the company about the drugs.
2. Information and results of clinical trials are published openly on the specialist website clinicaltrials.gov, which is maintained by the National Library of Medicine (NLM) at the National Institutes of Health (NIH). Publishing data on this resource is not mandatory, and therefore indicates the transparency of the drug developer's policy with regard to the findings. Processing the results takes time, so it is not surprising that out of 13 completed studies conducted in accordance with all the principles of Good Clinical Practice (GCP), results have only been presented for five so far. Unsurprisingly, the drug developer ranks first among Russian pharmaceutical companies in terms of the number of clinical trials in Russia, as of the third quarter of 2018. [SYNERGY RESEARCH GROUP, Research Report. Clinical trials in Russia. Autumn 2018; https://synergycro.ru/orange_paper/Synergy_Orange_Paper_Russia_2018Q3.pdf].
3. The drug’s market position in Russia reflects demand for the products, which the numerous awards received by the company demonstrate.
4. The authors mention more than 100 English-language articles published as a result of the drugs research. These publications are based on the results of many experiments (including efficacy, safety, mechanism of action, specificity, dose-dependence, etc.) conducted by independent research teams at various experimental sites (universities, institutes and laboratories) in dozens of countries around the world. Clinical studies have been conducted with the participation of hundreds of clinical sites.
Finally, it should be mentioned that the majority of articles dedicated to RA-drugs was published many years ago. Until recently not a single reader has written any concern to the journals regarding them. Now there is a group of authors who aim at this subject. The articles criticized by Panchin AY et al. were published in 2013-2017 but the requests to the journals were sent 1-3 years later. So the question of sincerity and impartiality of those who publicly lead the work aimed at discrediting RA-drugs arises.
Sergey Tarasov responded to our article “Drug discovery today: no molecules required” with claims of mistakes in our analysis. We are sure that we did not conceal anything of relevance or make any mistakes. This is a response to the claims by Sergey Tarasov.
1. Tarasov claims that one of us has an undeclared conflict of interest: “However, one of the authors of the article, Khromov-Borisov NN, is a defendant in a defamation lawsuit brought by Materia Medica (June 28, 2018). Materia Medica has sued Khromov-Borisov NN <…> this therefore raises serious doubts over the objectivity and impartiality of one of the authors of the article, who also carried out the data analysis presented in supplementary letter 1”. However, the analysis provided by Khromov-Borisov NN was sent to the editor of International Journal of Diabetes Research on Dec 13, 2017 (Supplementary Letter 1). Then editors of Drug Discovery Today (Feb 5, 2018, Supplementary Letter 5), BMJ (Apr 2, 2018), and BMJ EBM assessed the article including analysis mentioned above. Thus Tarasov presented the chronology completely wrong. First we have submitted our criticism of release-activity (RA) to scientific journals and only then Materia Medica sued one of our co-authors.
2. We don’t see how any of our previous criticisms of RA drugs and homeopathy creates a conflict of interest or are relevant to the reception of our article.
3. We have stated that RA drugs contain no active substance. Simple...
Sergey Tarasov responded to our article “Drug discovery today: no molecules required” with claims of mistakes in our analysis. We are sure that we did not conceal anything of relevance or make any mistakes. This is a response to the claims by Sergey Tarasov.
1. Tarasov claims that one of us has an undeclared conflict of interest: “However, one of the authors of the article, Khromov-Borisov NN, is a defendant in a defamation lawsuit brought by Materia Medica (June 28, 2018). Materia Medica has sued Khromov-Borisov NN <…> this therefore raises serious doubts over the objectivity and impartiality of one of the authors of the article, who also carried out the data analysis presented in supplementary letter 1”. However, the analysis provided by Khromov-Borisov NN was sent to the editor of International Journal of Diabetes Research on Dec 13, 2017 (Supplementary Letter 1). Then editors of Drug Discovery Today (Feb 5, 2018, Supplementary Letter 5), BMJ (Apr 2, 2018), and BMJ EBM assessed the article including analysis mentioned above. Thus Tarasov presented the chronology completely wrong. First we have submitted our criticism of release-activity (RA) to scientific journals and only then Materia Medica sued one of our co-authors.
2. We don’t see how any of our previous criticisms of RA drugs and homeopathy creates a conflict of interest or are relevant to the reception of our article.
3. We have stated that RA drugs contain no active substance. Simple school arithmetic shows that the C12 dilution corresponds to the concentration of initial substance approximately 10^-24 g in 100 ml of the solvent. The mass 10^-24 g is less than the mass of a single proton or neutron (10^-24 g = 0.6 proton).
Nonetheless, Tarasov qualifies such calculations “theoretical assumptions” and states that “the available experimental data refute the authors' theoretical assumptions that in principle ultra-high dilutions cannot contain molecules.” However, the proponents of “release-activity” claim that “high dilutions no longer containing molecules of the original substance are able to cause a physiological reaction based on molecular-level effects.” Epstein O. The Spatial Homeostasis Hypothesis. Symmetry 2018, 10(4), 103. And that “Technologically processed dilutions also exert activity when they do not contain molecules of parent substance”. Epstein O. The phenomenon of release activity and the hypothesis of "spatial" homeostasis. Usp Fiziol Nauk. 2013 Jul-Sep;44(3):54-76
None of the RA articles we mention have provided concentrations of active substances, which is a flaw we discuss in our analysis. The idea that initial substances can somehow survive such dilutions (if true) works against the existence of the phenomena of “release-activity” and makes the discussed RA studies meaningless.
4. In our article we state that “There is no mention that Epstein is the CEO and founder of MMH”. Tarasov claims that “the concealment of a conflict of interest claimed by Panchin AY et al. therefore has no basis in fact.” To that we will repeat one more time: “There is no mention that Epstein is the CEO and founder of MMH”. This is conflict of interest that was not reported. And it’s a fact. We are content that Sergey Tarasov did not disagree with our concerns about concealed conflicts of interest in other RA articles.
5. In our article we state that “The authors failed to provide images of acquired blots, making it impossible to check the validity of their analysis (Supplementary Letter 2)”. Tarasov replied that: “the Editorial Board of Nutrition & Diabetes was provided with images of acquired blots during the peer-reviewing process”. However, we asked for these images both from the Editor of Nutrition & Diabetes (Sep 18, 2018) and from the corresponding author of the article (Nov 11, 2018). The Editor replied (Sep 21, 2018) but did not provide us with any data and suggested we ask for the data from the authors. The authors also did not respond to our e-mail or provide the required data. For some reason Sergey Tarasov, Director of Research & Development Department OOO “NPF “Materia Medica Holding”, also decided not to provide this data in his response. We would be happy to see it.
6. In our article we state that: “The champion in publishing research on RADs is a Russian journal, Bulletin of Experimental Biology and Medicine (English edition is by Springer Nature) <…> Notably, at least 48 of them were published in issue 135 Supplementary 7 (2003) (ref.40). It might be not surprising that Epstein was the editor of this supplementary issue (ref.41)”. Tarasov replies that “this collection is a special edition of the journal supplement” and that “It is not surprising that O.I. Epstein, a scientist who studies this phenomenon, was one of the editors of the special edition”. We don’t see how this is relevant or changes the fact that the editor who published most articles on RA drugs was both a person with a conflict of interest and coauthor of the papers.
7. Tarasov questions how we selected the articles for our analysis. We have addressed articles on RA drugs mentioned in the Drug Discovery Today review (highest impact factor journal which published article by Epstein OI). We did not analyze the article from Bulletin of Experimental Biology and Medicine due to how that journal handled the other RA drug articles (see paragraph above) and its overall low quality. We added the discussion of two more articles because we were already discussing them with the Editors when we discovered the Drug Discovery Today review. We would like to note that our article is not a meta-analysis. However, we hope that it will launch independent investigations of all published articles concerning RA drugs.
8. Tarasov claims that “one of the main reasons why our article was retracted from PLOS ONE [Gavrilova ES, Bobrovnik SA, Sherriff G, et al. Novel approach to activity evaluation for release-active forms of anti-interferon-gamma antibodies based on enzyme-linked immunoassay. PLoS One 2014;9:e97017.] was incomplete conflict of
interest information disclosure“.
However, the retraction statement of PLOS ONE (https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0197086) also states clearly: “During the journal’s post-publication assessment of this work, concerns were also raised about insufficient methodological reporting in the published article that obfuscate the assessment of the study design. The authors provided additional details about methods and reagents used in response to the journal’s queries, but even in consideration of these the fundamental concerns remain about the premise of the study, the assay used, and the overall reliability of the conclusions”.
We have provided detailed criticism of the methods, results and conclusions in this retracted study and our concerns have not been addressed by the authors (Supplementary Letter 4). Thus, we stand by our conclusion that the article is flawed and agree with the decision of PLOS ONE to retract it.
9. Sergei Tarasov states that “In respect to the statistical analysis presented by us in the original article (Journal of Diabetes Research), there are no statistical adjustments for multiple comparisons because the null hypothesis of the study was to compare Subetta with control at certain time point but not in dynamic and to compare effect within the group”.
However, the authors of the discussed paper have reported multiple (27 of 120, excluding the effects of a real drug Rosiglitazone) statistically significant effects which when using adequate and correct statistical procedures appear to be false.
The requirement to adjust the p-values for multiple comparisons is fixed in several most authoritative guidelines and checklists:
“Indicate whether and how any allowance or adjustments were made for multiple comparisons (performing multiple hypothesis tests on the same data).” Lang T.A., Altman D.G. Statistical analyses and methods in the published literature: The SAMPL guidelines. Medical Writing, 2016; 25(3):31 – 36. http://journal.emwa.org/statistics/statistical-analyses-and-methods-in-t...
“Describe adjustments were made for multiple comparisons.” Ghasemi A., Bahadoran Z., Zadeh-Vakili A., Montazeri S.A., Hosseinpanah F. The principles of biomedical scientific writing: materials and methods. Int. J. Endocrinol. Metab. 2019; 17(1):e88155. doi: 10.5812/ijem.88155. https://neoscriber.org/cdn/dl/91aee4ae-2490-11e9-ab2c-3360f3fcc2f4
Adjustment for the multiplicity testing (multiple comparisons) was used in one of the papers cited by Tarasov:
“The statistical significance of each ELISA and area under the curve (AUC) graph was measured using Mann-Whitney U test (one-tailed) or a one-way ANOVA with Bonferroni’s post hoc test.”
Kim, K.-S. et al. The aglycone of ginsenoside Rg3 enables glucagon-like peptide-1 secretion in enteroendocrine cells and alleviates hyperglycemia in type 2 diabetic mice. Sci. Rep. 5, 18325; doi: 10.1038/srep18325 (2015). https://www.nature.com/articles/srep18325.pdf
Bonferroni’s post hoc test means such an adjustment and is one of the most widely used.
We could respond to Sergei Tarasov about why we used regression analysis, but it does not really matter what analysis we use. The results are the same. For example, we carried out the ANOVA for this data. Details of this analysis with tables and figures can be found here: https://tinyurl.com/y9kbtclr. This ANOVA analysis was done by Khromov-Borisov NN.
The conclusion stands: three “release-active” preparations (AIR, AeNOS and its composition, Subetta) have no antidiabetic effects in the diabetic Goto-Kakizaki rat.
I realise there is a lot that has been written on this issue, more than it is possible for someone like me to work through in detail. However, I must point out some factual errors in the Jørgensen et al response dated 17th September.
I was rather surprised at the level of detail of some of the critique, which even drilled down to cross-checking the exact number of reported deaths in dozens of individual trials. I therefore decided to look at one of the Jørgensen et al claims, and opted for one that I could verify pretty easily, namely the number of deaths in the VIVIANE study. https://www.ncbi.nlm.nih.gov/pubmed/27373900
Jørgensen et al stated that they "found that the Cochrane HPV review gave an incorrect number of deaths for the VIVIANE study (HPV-015): 13 deaths in the HPV vaccine group and 5 deaths in the AlOH3 group; according to VIVIANE’s journal publication (8), there were 14 deaths in the HPV vaccine group and 3 deaths in the Al(OH)3 group."
A check of their cited reference (8) demonstrates that Jørgensen et al have sourced the 48 week interim 2014 data analysis for this study, rather than sourcing the correct data from the final 84 week data analysis published in 2016, which is quite properly what the Cochrane Review authors sourced.
The correct citation source states: "Data for serious adverse events related to the vaccine and deaths continued to be collect...
I realise there is a lot that has been written on this issue, more than it is possible for someone like me to work through in detail. However, I must point out some factual errors in the Jørgensen et al response dated 17th September.
I was rather surprised at the level of detail of some of the critique, which even drilled down to cross-checking the exact number of reported deaths in dozens of individual trials. I therefore decided to look at one of the Jørgensen et al claims, and opted for one that I could verify pretty easily, namely the number of deaths in the VIVIANE study. https://www.ncbi.nlm.nih.gov/pubmed/27373900
Jørgensen et al stated that they "found that the Cochrane HPV review gave an incorrect number of deaths for the VIVIANE study (HPV-015): 13 deaths in the HPV vaccine group and 5 deaths in the AlOH3 group; according to VIVIANE’s journal publication (8), there were 14 deaths in the HPV vaccine group and 3 deaths in the Al(OH)3 group."
A check of their cited reference (8) demonstrates that Jørgensen et al have sourced the 48 week interim 2014 data analysis for this study, rather than sourcing the correct data from the final 84 week data analysis published in 2016, which is quite properly what the Cochrane Review authors sourced.
The correct citation source states: "Data for serious adverse events related to the vaccine and deaths continued to be collected beyond month 48. Serious adverse events possibly related to the study vaccine were reported in five women (0·2%) of 2877 in the vaccine group and eight women (0·3%) 2870 in the control group ( appendix ). 13 women in the vaccine group and five in the control group died; no deaths were considered by the investigator to be related to study vaccination ( appendix )."
It is quite troubling that after [presumed] detailed review of all this data that Jørgensen et al should make such a basic error, and then accuse others of being incorrect.
Jørgensen et al also seem to be troubled that the Cochrane Editors in their response to them "did not address our point that the VIVIANE study (HPV-015) included in its register entry “one case of ‘Adenocarcinoma of the cervix’ and one case of ‘Cervix cancer metastatic’ … in the HPV vaccine group” (2), and that the Cochrane HPV review includes a death caused by “Cervix cancer metastatic” in the HPV vaccine group, which was not mentioned in the main text (3)."
I cannot speak for the Cochrane Editors, but perhaps they did not think this was an issue that needed to be expanded upon further, but one which had been adequately explained in the VIVIANE study publication. Indeed if one looks at the supplementary appendix of the VIVIANE study (2016) one finds this explanation:
"This woman had normal cytology but was HPV-18 DNA positive at study entry (May 2006). At the next scheduled cytology testing at month 12 (April 2007), the cytology finding was 'atypical squamous cells, cannot exclude high grade squamous intraepithelial lesion'. She was diagnosed with metastatic cervical cancer in May 2007 (approximately 7 months after receiving the third dose of vaccine and died in July 2008."
This appears to be an unfortunate case of a woman with rapidly progressive malignancy. I am unsure what mileage Jørgensen et al wish to make of it, nor how the occurence of one case can somehow contribute significantly to the safety signal data accrued on the HPV vaccine.
Finally, I must take issue with the way in which Jørgensen et al quote the WHO's statement about placebos. Jørgensen et al said this:
"The WHO states that using adjuvant or another vaccine as comparators instead of placebo makes it difficult to assess the harms of a vaccine (12)."
The inference we are meant to make from this is that the WHO do not regard a vaccine control that uses just an adjuvant, or another vaccine as a "placebo". This is a distortion of what the WHO say about placebos, and I wonder if Jørgensen's phraseology here is a deliberate attempt to mislead.
In fact the WHO say this to say:
"A true placebo is an inert substance, but in the context of vaccine research, the term placebo is also applied to other types of comparators that are not inert, but are not expected to protect against the disease of interest in a vaccine trial."
This makes it clear that they regard a control vaccine consisting of an adjuvant as a "placebo". This point is important, because the Nordic Cochrane group have criticised the descriptor of adjuvanted placebo as a "placebo" in the Cochrane review. Whilst it is true that a completely inert "true" placebo like saline is ideal for assessing safety of the vaccine comparator, and it is unfortunate that this was not used more widely in the HPV studies, it would appear that it is quite acceptable to use the term "placebo" to describe a vaccine control that does not have an active antigenic component in it (such as an adjuvant, or a different vaccine of known safety profile). So again, Jørgensen et al have cited a source that contradicts their claims rather than confirming them.
Obviously, for the sake of clarity, it should always be stated what the placebo vaccine is constituted with in every vaccine trial, so appropriate conclusions may be drawn about possible reactions reported in the control group, and Cochrane have acknowledged that.
In general there is often not enough attention paid to the difference of clinical and statistical significance. I do not understand, however, how the authors of the letter conclude that the clinical effect in CASTE-AF was "tiny".
The application of Cohen's d to a study with a discrete outcome is rather unusual. A more intuitive approach, if one is unfamilar with hazards, would be to use the relative risk as outcome measure. The risk of the primary outcome in the ablated group is 28.5%, the risk in the comparison group is 44.6%. The relative risk is 0.64. The absolute risk reduction is 16% and the number needed to treat is 6.25 (for three years). Not a tiny effect.
The problems associated with NOT using a true placebo in clinical trials - in this case GSK's trial for Cervarix - were discussed way back in 2009:
FOOD AND DRUG ADMINISTRATION CENTER FOR BIOLOGICS EVALUATION AND RESEARCH
VACCINES AND RELATED BIOLOGICAL PRODUCTS ADVISORY COMMITTEE
September 9, 2009
Pg166
DR. DEBOLD: I just think in the absence of having a true placebo in the entire study, it is very, very difficult to sort out what the effects are here, what is it that we are really dealing with, what is the baseline, versus what potentially is caused by the so-called controls.
In this particular study, not only was there two different strengths of Havrix used, there was alum used. In the pooled analysis there were a number of other vaccines used as the control, which makes it scientifically very difficult to sort things out.
I have read your response [1,2] to the paper by Jøorgensen et al. published in BMJ Evidence-Based Medicine [3] relating to the recently published Cochrane Review on HPV vaccines [4], and would like to give my feedback on this issue.
The key findings of your investigations are as follows:
1. The Cochrane Review did not miss "nearly half of the eligible trials". A small number of studies were missed due to the primary focus on peer-reviewed reports in scientific journals, but addition of these data makes little or no difference to the results of the review for the main outcomes;
2. The trials comparators were unambiguously, transparently, and accurately described;
3. The selection of outcomes for benefits was appropriate and was consistent with World Health
Organization guidance;
4. The review included published and unpublished data on serious harms, and the findings on
mortality were reported transparently and responsibly;
5. The review was compliant with Cochrane’s current conflict of interest policy;
6. Cochrane’s media coverage was cautious and balanced, but we recognize that there could be
improvements in relation to transparency where external experts are quoted;
7. The BMJ Evidence-Based Medicine article substantially overstated its criticisms.
I would like to comment on your findings 1, 2, 4 and lastly, I added comments as 8. Most observational studies neg...
I have read your response [1,2] to the paper by Jøorgensen et al. published in BMJ Evidence-Based Medicine [3] relating to the recently published Cochrane Review on HPV vaccines [4], and would like to give my feedback on this issue.
The key findings of your investigations are as follows:
1. The Cochrane Review did not miss "nearly half of the eligible trials". A small number of studies were missed due to the primary focus on peer-reviewed reports in scientific journals, but addition of these data makes little or no difference to the results of the review for the main outcomes;
2. The trials comparators were unambiguously, transparently, and accurately described;
3. The selection of outcomes for benefits was appropriate and was consistent with World Health
Organization guidance;
4. The review included published and unpublished data on serious harms, and the findings on
mortality were reported transparently and responsibly;
5. The review was compliant with Cochrane’s current conflict of interest policy;
6. Cochrane’s media coverage was cautious and balanced, but we recognize that there could be
improvements in relation to transparency where external experts are quoted;
7. The BMJ Evidence-Based Medicine article substantially overstated its criticisms.
I would like to comment on your findings 1, 2, 4 and lastly, I added comments as 8. Most observational studies neglect “healthy vaccine effect/healthy user bias”.
1. The Cochrane Review did not miss "nearly half of the eligible trials".
You explained “A small number of studies were missed due to the primary focus on peer-reviewed reports in scientific journals.” However, the abstract of the Cochrane review reports: “We searched MEDLINE ----- for reports on effects from trials. We searched trial registries and company results’ registers to identify unpublished data for mortality and serious adverse events.”
For the serious adverse events, the Cochran review described in the sensitivity analysis as follows: “We assessed the robustness of data collected for serious adverse events, all-cause mortality and pregnancy outcomes based on the source of data. The primary analysis for these outcomes included data that we considered to represent the most complete follow-up. As a sensitivity analysis we used data for these same outcomes that had only been reported in the published trial reports.”
It is evident that the Cochrane review recognized the importance of clinical study reports (CSRs) and partially gathered the unpublished data for mortality and serious adverse events.
Jørgensen et al. pointed out that there were many missed trials which Cochrane review did not included because they were not published.
As for the analysis of oseltamivir, we, the Cochrane neuraminidase inhibitor team, used all CSRs and found significantly increased psychiatric events in the 4 CSRs for prophylaxis of influenza. However, if we restricted to use peer reviewed published journals, we would not have detected the psychiatric harm even if CSRs for published trials were used. This is the most important point that Jørgensen et al. emphasized.
The review should be revised by including all CSRs identified and available. Before the revision is completed, the Cochrane review should be suspended.
2. Description of the comparators and the unavoidable toxicity of adjuvants
2-1. The safety of adjuvants is not established.
You wrote “The trials comparators were unambiguously, transparently, and accurately described”. However, the problem is not the descriptions but the real toxicity of adjuvants.
It is true that the Cochrane review by Arbyn et al. described the comparators unambiguously, transparently, and accurately. However, the problem is not the accuracy and transparency of the description of the comparators. It is that adjuvants as the comparators conceal the true harm of HPV vaccines because the safety of the vaccine adjuvant has never been established clinically [5] nor non-clinically. Instead, harm of adjuvant is rather unavoidable as indicated by laboratory tests and toxicity tests shown in the followings.
2-2. Non-clinical tests strongly suggest the harm of adjuvant.
It is revealed that true adjuvant is the DNA of the recipient which is produced by tissue damage by the administered adjuvant, such as alum adjuvant. This suggests that the bigger the damage, the stronger the stimulation of innate immunity and work as adjuvant [6,7], and autoimmune diseases [8], including those in the central nervous system [9] may be induced. GlaxoSmithKline conducted several animal toxicity tests, although they did not fulfil the standard toxicity testing method for the ordinary pharmaceutical products.
In the third toxicity test, single or repeated i.m., Cervarix, and AS04 adjuvant induced item-related local degeneration, necrosis, and regeneration of muscle fibers with hemorrhage and mild-to-moderate subacute inflammation at 4 days and 5 months after the first inoculation, unlike saline [10]. The extent and proportion of animals with these lesions were the same between Cervarix and AS04 group and immediately after the single dose but more prominent in the Cervarix group than AS04 adjuvant group after 4 repeated doses [10].
In a randomized controlled trial, saline control is appropriate as the comparator for the analysis of both efficacy and harm. For efficacy analysis, active control such as adjuvant or vaccines with/without adjuvant may not be necessarily inappropriate as the comparator. However, these active controls are definitely inappropriate for the harm analysis. The other approach should be applied for the harm analysis, if no saline control trials are available.
4. The review included published and unpublished data on serious harms, and the findings on mortality were reported transparently and responsibly;
4-1. Meta-analysis of mortality in the trials targeting mid-adults women
You wrote “The review included published and unpublished data on serious harms, and the findings on mortality were reported transparently and responsibly.” According to my meta-analysis using the extracted data from the reference peer reviewed papers in which subjects’ ages were 25 (or 26) to 45 year or older [11-14], the pooled odd ratio was 5.00 (95%CI: 1.71, 14.65), P = 0.0022 (I2 = 0%). This indicates that HPV vaccines increase mortality in woman aged 25 or older by 5 times within 4 years after the first injection.
• For the VIVIANE trial [11,12], I used 13 deaths in Cervarix group and 3 deaths in adjuvant group within 4 years, because after that period there was no difference between these groups.
• For the FUTURE III trial [13], the numbers of deaths described in the published paper were 7 in the Gardasil group and 1 in the adjuvant group as shown in its sensitivity analysis of death. However, in the Cochrane review, the numbers are 8 and 4, respectively, presumably based on the data including after 4 years from unpublished CSR.
• For the Chinese trial [14], my extraction was HPV vaccine 0 and adjuvant 0 while the Cochrane review data were 1 and 0, respectively, also in the sensitivity analysis.
Moreover, you described that after adding the mortality data from newly included trial NCT00834106 (targeted 20 to 45 years of age), RR of death increased from 1.54 (95% CI 0.73 to 3.23) to 1.65 (95% CI 0.80 to 3.38). It is highly probable that substantial difference of death risk may be reported in this added study. Please clarify the number of deaths in both groups (Data from Clinicaltrial.gov suggest that the number of deaths are estimated 2/1499 and 0/1498 for Gardasil and adjuvant group respectively).
Risk of death in women aged 25 to 45 years may be further robust by adding this trial.
Serious adverse event and mortality should be separately analysed for different age groups: namely under the age of 25 years and mid-adults (approximately 25 to 45 years old) in addition to the analysis as the whole ages.
4-2. Fluctuation of mortality and adverse events
If a vaccine does not cause harmful effects, the incidence rate of adverse events may stay at the baseline or increase only slightly as age increases. However, if the incidence rate of adverse events fluctuates substantially, then it may be the result of the harmful effect of the vaccine. Several patterns of theoretical trend of incidence rate are shown (Figure not shown).
In the pivotal RCT of Cervarix (PATRICIA), no difference in the adverse events between Cervarix group and adjuvanted HA vaccine group was observed. Hence, I calculated the overall (both groups) trend of incidence rates of adverse events, including chronic diseases (CD), autoimmune diseases (AD) and death (D) dividing 3 periods: 1: 0-1.2 year, 2: 1.2-3.4 year, 3:3.4-3.65 year). Incidence rate (/100,000 person-year) of CD, AD and D fluctuate as follows: CD: 129-55-164, AD: 25-24-88, D:22-30-135 respectively. These data show that the incidence rates or mortality rates greatly fluctuate and increase even after about 3.5 years from the first inoculation (Figure not shown).
For Gardasil studies, I calculated the incidence rates (/100,000 person-years) of various autoimmune-related adverse events (AE) among participants from both Gardasil and control (almost all received alum-adjuvant) groups (Figures not shown). Incidence rates (/100,000 person-years) of total autoimmune related adverse events are 2441 at the first period (day 1 to 6 months) and 625 at the second period (7-24 months) and that of inflammatory bowel disease are 77 and 28 respectively.
These also indicate that the incidence rates of various autoimmune diseases greatly fluctuate over time and suggest Gardasil affected these fluctuations (included studies were all phase II and III trials targeted women aged 9 to 24).
VIVIANE trial shows high mortality rate during the first 4 years and low mortality rate after 4 years in the Cervarix group, but this was not evident in the control adjuvant group (Figure not shown). These data show fluctuations are greater in the HPV vaccine group.
If the numbers of deaths for FUTURE III trial are 8 for Gardasil group and 4 for adjuvant group, 1 and 3 deaths may be observed after 4 years, respectively. Person-years at risk is not estimated, but 7 in the first 4 years and 1 thereafter indicate similar fluctuation, and less fluctuation is observed in the adjuvant group. This may be a very similar phenomenon as shown in the VIVIANE trial.
Fluctuation of serious adverse events including death should be considered in the RCTs using adjuvant as comparator. Analysis of high risk and low risk periods together may conceal true harm of HPV vaccines. Analysis should be conducted separately for the different risk periods.
4-3. Comparison of incidence rates of specific autoimmune diseases between RCT and general population of women of similar age group
The incidences rates of some autoimmune diseases examined, such as multiple sclerosis (MS), systemic lupus erythematosus (SLE), and inflammatory bowel diseases (IBD), calculated from the data in the “SBA” of Gardasil (age ranged 9–26 years, mainly 16–23 years) were all higher than those in the general female population of similar age (15 to 25 years old).
For example, the incidence rate (per 100,000 person-year) of MS in Gardasil RCTs (14.7) was about 3–15 times higher than that of general population in Italy (4.2–4.7), the United Kingdom (3.4), and France (1.0). Notably, MS incidence reported in Gardasil RCTs was even higher than the highest reported in the general population, namely, in north Sweden (8.4) and Iceland (12.5) (Figure not shown).
Incidence rates of IBD in the RCTs are also higher than that of the general female population of similar age, though the age range is a little broader (0 to 39 years old).
Annual incidence rates for typical adverse events such as specific autoimmune diseases in the RCTs should be compared with that of general population of similar age.
8. Most observational studies neglect “healthy vaccine effect/healthy user bias”
In the discussion section, the Cochrane review discussed the Adverse effects of HPV vaccines citing many observational studies.
However, most observational studies have fundamental flaws:
• Negligence of healthy vaccine effect (healthy user bias),
• Negligence of time dependent bias in the self-control case-series method and
• Confusion of incidence with prevalence.
Here I only discuss the Negligence of healthy vaccine effect (healthy user bias) which most observational studies neglected, affect the results seriously and sometimes reverse the association.
Negligence of healthy vaccinee effect (healthy user bias)
i) Theoretical basis
It is important that confounding bias is avoided in epidemiologic studies. In particular, confounding bias from “healthy vaccinee effect” always affects results in favour of an intervention, leading to overestimation of its effectiveness and safety.
People who have any diseases and/or are prone to have fever/infection tend to avoid vaccination. Autoimmune diseases often follow infection. Unless unvaccinated people who are sicklier or frailer than vaccinated people at baseline were not adequately adjusted, the results of the high incidence rate of autoimmune diseases in the vaccinated people may be offset by the disease incidence in the sickly unvaccinated people. This is “healthy-vaccinee effect,” “frailty selection bias,” or “frailty exclusion confounding bias.” [15]
Bias from “healthy-vaccinee effect” becomes more impactful as the coverage of vaccination becomes higher, namely 80% or higher [15]. Theoretically in the unvaccinated, odds ratio for having symptoms at a certain vaccine coverage (%) compared with the lowest coverage group increases as the percent coverage increases.
ii) Nagoya study
Nagoya study is a questionnaire survey on symptoms after HPV vaccination, involving about 70,000 girls (born between 2 April 1994 and 1 April 2001 (approximately aged 14 to 21 in September 2015) living in Nagoya City Japan. Of these, about 30,000 girls responded: 20,748 girls were vaccinated and 9,098 girls were unvaccinated [16].
Suzuki et al. [16] concluded “the results suggested no causal association between the HPV vaccines and their alleged harmful symptoms”.
However, according to my logistic regression analysis using the disclosed PDF data, odds ratios (ORs) of positive symptoms before vaccination were significantly (p<0.05) lower than 1.0 in 15 of 24 symptoms ranging from 0.16 (Dependent on stick or wheel chair) to 0.86 (Irregular menstrual cycle). ORs for symptoms leading to hospital visit were significantly lower than 1.0 in 7 of 24 symptoms, especially in severe symptoms, such as “Unable to walk normally” (0.22), “Dependent on stick or wheel chair” (0.21), “Sudden loss of strength” (0.28) and “Weak in the extremities” (0.30). OR for “Difficulties in calculation” is not significant (P=0.148), but point estimate of OR was very low (0.25, p=0.15).
The trends of the proportion of the frail, by coverage both in the unvaccinated and the vaccinated groups fit well to the theoretical trends expected in each group.
On the other hand, odd ratios of positive symptoms after vaccination leading to hospital visit were generally higher than 1.0 (no symptom was less than 1.0) and significantly higher than 1.0 in 11 of 24 symptoms: for example, “Weak in the extremities” (2.76, p=0.014) and “Difficult to remember Chinese characters” (8.46, p=0.047). “Dependent on stick or wheel chair leading to hospital visit” were reported in 13 girls in the vaccinated group, while none in the unvaccinated group: odds ratio = 9.61 (95%CI: 1.21- infinity, p=0.027) by the exact-like logistic regression using “R” software.
Considering the low odds ratio of health status before inoculation, each odds ratio after inoculation should be divided by the corresponding odds ratio before inoculation. Hence point odds ratio after vaccination of HPV vaccine for the symptoms leading to hospital visit are estimated as follows (those above 5.0 are shown): Dependent on stick or wheel chair: 46.7, Difficult to remember Chinese character: 24.8, Difficult to calculate: 15.5, Unable to walk normally: 11.0, Weak in the extremities: 9.2, Sudden loss of strength: 8.6 and Involuntary movement: 5.7
iii) French study: a rare study incorporating prior health status as confounder shows harm of autoimmune diseases
Among many observational studies, French study [17] is a rare study which considers the health status before vaccination, though partially. It is a retrospective cohort study utilizing database of national health system. Total 2.25 million girls aged 13 to 16 years between 2008 and 2012 were included: 0.84 million (37.3%) were exposed to HPV vaccine and 1.41 million were not exposed.
Outcome events were 14 autoimmune diseases (AD), including central nervous system (CNS) demyelinating diseases (MS: multiple sclerosis etc.), Guillain-Barres syndrome (GBS), and inflammatory bowel diseases (IBD). Hazard ratios of any of the14 ADs or of individual AD were estimated by Cox regression analysis adjusting age, year of vaccination as time dependent variable. Geometric data of residence, other diseases, level of care before inclusion, vaccination other than HPV vaccine were also adjusted. Sensitivity analysis and subgroup analysis were performed to make sure of the robustness of the results.
The adjusted factor “Level of care before inclusion” seems to be “at least one visit for medical care prior to inclusion”. Hence it is unclear how it influenced the results, and the adjustment of bias from healthy vaccinee effect may be partial.
However, a statistically significant association with HPV vaccine was shown in 2 autoimmune diseases: adjusted HR of inflammatory bowel diseases (IBD) was 1.19 (95%CI: 1.02-1.39) and that of Guillain-Barré syndrome (GBS) was 4.00 (95%CI: 1.84-8.69). The sensitivity analysis showed that the association of both autoimmune diseases were the strongest for the period from 0 to 3 months and tended to decrease subsequently without significance for IBD, but it remained significant for GBS. Risk of GBS is robust as suggested by the sensitivity and subgroup analysis. These suggest that the incidence rate of both symptoms may have fluctuated.
Moreover, using 4.7 million as the person-year (py) of unvaccinated people, incidence rates per 100,000 py are greatly different from those of the unvaccinated in the table reported by the French Agency: especially in CNS demyelinating diseases (4.7 by recalculation vs 5.8 by original report), inflammatory bowel diseases (IBD) (13.8 vs 16.9) and those having at least one autoimmune disease (AD) (63.4 vs 66.8). If our estimate of incidence rate in the unvaccinated group is correct, odds of CNS demyelinating diseases in the HPV vaccinated girls increased non-significantly (P=0.07). Odds of IBD increased more with highly significant OR up to 1.53 (95%CI: 1.33, 1.76, P<0.001), and increased odds of having at least one autoimmune disease was also significant: OR = 1.13 ((95%CI: 1.05, 1.21, P<0.001).
Moreover, incidence rate (/105 py) of CNS demyelinating diseases in the unvaccinated group (4.7 or 5.8) is very high compared with that of multiple sclerosis (0.99 [95% CI: 0.94–1.04]) in the general female population of the same age group (aged 15 to 24 years) in France between 2001 and 2007 [18]. Incidence rate (/105py) of CNS demyelinating diseases in the HPV vaccinated group in the French pharmacovigilance study [17] is far higher than that in the general population.
These suggest that unvaccinated group had included frail girls at the time of inclusion. Therefore, it should be considered that “healthy vaccinee effect” or “frailty exclusion bias” may not be completely excluded even in this French pharmacovigilance study [17].
Moreover, significantly high incidence of events after inclusion such as frequent consultation (≧4/year): Odds ratio (OR) = 2.81 (2.80,2.83) and at least one hospitalization: OR = 3.54 (3.52, 3.56) (based on person-year) should be taken into account. This is because high odds ratio needing frequent medical care and/or hospitalisation (9% of the girls per year need additional hospitalization compared with prior vaccination and unvaccinated girls) may be related to some adverse outcomes of HPV vaccination other than autoimmune diseases.
For the discussion of harm comparing the results with those from observational studies, it is essential to consider the “healthy vaccinee effect”. Unless the observational studies adjust the health status before vaccination (pseudo vaccination date for unvaccinated group), the findings from such studies should not be used as evidence of safety.
Conclusion
The review should be revised by including all identified and available data. Before the revision is completed, I consider the Cochrane review should be suspended.
In a randomized controlled trial, saline control is appropriate as the comparator for the analysis of both efficacy and harm. For efficacy analysis, active control such as adjuvant or vaccines with/without adjuvant may not be necessarily inappropriate as the comparator. However, these active controls are definitely inappropriate for the harm analysis. The other approach should be applied to the harm analysis, if no saline control trials are available.
Serious adverse event and mortality should be separately analysed for different age groups: under the age of 25 years and mid-adults (approximately 25 to 45 years old) in addition to the analysis as the whole ages.
Fluctuation of serious adverse events including death should be considered in the RCTs using adjuvant as comparator. Analysis of high risk and low risk periods together may conceal true harm of HPV vaccines. Analysis should be conducted separately for the different risk periods.
Annual incidence rate for typical adverse events such as specific autoimmune diseases in the RCTs should be compared with that of general population of similar age.
For the discussion of harm comparing the results with those from observational studies, it is essential to consider the “healthy vaccinee effect”. Unless the observational studies adjust the health status before vaccination (pseudo vaccination date for unvaccinated group), the findings from such studies should not be used as evidence of safety.
It is my sincere hope that you would consider my feedback seriously in order to further improve the quality of the Cochrane review. Should you have any questions, please feel free to contact me.
Sincerely yours,
Rokuro Hama M.D.
Chairperson, NOP Japan Institute of Pharmacovigilance
Conflict of interest
Rokuro Hama (RH) has been a member of the Cochrane neuraminidase inhibitor team headed by Tom Jefferson since January 2010. RH has met Dr. David Tovey (DT) and discussed the methods for conducting the systematic review of neuraminidase inhibitors [19] at the Oxford meeting in April 2011. RH explained DT and other members of the Cochrane neuraminidase inhibitor team how Tamiflu lowered the antibody production in the treatment trial and that ITTI population was inappropriate for the assessment of efficacy. Instead, RH proposed that ITT population should be used to assess efficacy and harm of Tamiflu. The method using ITT population was agreed by the Cochrane neuraminidase inhibitor team at the meeting in April 2011.
References
1. Cochrane’s Editor in Chief responds to BMJ EBM article criticizing HPV review https://www.cochrane.org/news/cochranes-editor-chief-responds-bmj-ebm-ar...
2. Tovey D and Soares-Weiser K. Cochrane’s Editor in Chief responds to a BMJ Evidence-Based Medicine article criticizing the Cochrane Review of HPV vaccines. https://www.cochrane.org/sites/default/files/public/uploads/cochrane_hpv...
3. Jørgensen L, Gøtzsche PC, Jefferson T. The Cochrane HPV vaccine review was incomplete and ignored important evidence of bias. BMJ Evidence-Based Medicine 2018 July 27 https://doi.org/10.1136/bmjebm-2018-111012 2.
4. Arbyn M, Xu L, Simoens C, Martin-Hirsch PPL. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Database of Systematic Reviews 2018;(5):CD009069. https://doi.org/10.1002/14651858.CD009069.pub3
5. Jefferson T, Rudin M and Pietrantonj CD, Adverse events after immunization with aluminium-containing DTP vaccines: systematic review of the evidence, Lancet Infect Dis 4, 2004, 84–90.
6. Marichal T, K. Ohata K, Bedoret D, Mesnil C, Sabatel C, Ishii KJ et al., DNA released from dying host cells mediates aluminum adjuvant activity, Nat Med 17, 2011, 996–1002.
7. Sloane JA, Blitz D, Margolin Z and Vartanian T, A clear and present danger: endogenous ligands of Toll-like receptors, Neuromolecular Med 12, 2010, 149–163.
8. Tsumiyama K, Miyazaki Y and Shiozawa S, Self-organized criticality theory of autoimmunity, PLoS One. 4, 2009, e8382.
9. Lampron A, Elali A and Rivest S, Innate immunity in the CNS: redefining the relationship between the CNS and its environment, Neuron 24 (78), 2013, 214–232.
10. GlaxoSmithKline Biologicals. Japanese Summary Basis of Approval of Cervarix. http://www.pmda.go.jp/drugs/2009/P200900052/index.html ; [accessed 18.9.4].
11. Skinner SR, Szarewski A, Romanowski B, Garland SM, Lazcano-Ponce E, Salmeron J. Efficacy, safety, and immunogenicity of the human papillomavirus 16/18 AS04- adjuvanted vaccine in women older than 25 years: 4-year interim follow-up of the phase 3, double-blind, randomised controlled VIVIANE study. Lancet 2014;384(9961): 2213–27.
12. Wheeler CM, Skinner SR, Del Rosario-Raymundo MR, Garland SM, Chatterjee A, Lazcano-Ponce E, et al. Efficacy, safety, and immunogenicity of the human papillomavirus 16/18 AS04-adjuvanted vaccine in women older than 25 years: 7-year interim follow-up of the phase 3, doubleblind, randomised controlled VIVIANE study. Lancet Infectious Diseases 2016;16(10):1154-68.
13. Castellsagué X, Munoz N, Pitisuttithum P, Ferris D, Monsonego J, Ault K. End-of-study safety, immunogenicity, and efficacy of quadrivalent HPV (types 6, 11, 16, 18) recombinant vaccine in adult women 24 to 45 years of age. British Journal of Cancer 2011;105(1):28–37.
14. Zhu FC, Li CG, Pan HX, Zhang YJ, Bi D, Tang HW. Safety and immunogenicity of human papillomavirus-16/ 18 AS04-adjuvanted vaccine in healthy Chinese females aged 15 to 45 years: a phase I trial. Chinese Journal of Cancer 2011;30(8):559–64.
15. Hama R. Symptoms after HPV vaccine: typical "frailty exclusion bias" in Nagoya City study.
MedCheckTIP 2016: 2(No5); 18-21 available at https://npojip.org/english/MedCheck/Med%20Check-TIP%2005-08-08.pdf
16. Suzuki S, Hosono A. No Association between HPV Vaccine and Reported Post-Vaccination Symptoms in Japanese Young Women: Results of the Nagoya Study. Papillomavirus Res. 2018 Feb 23. pii: S2405-8521(17)30070-8
17. French National Agency for Medicines and Health Products Safety. Vaccins anti-HPV et risque de maladies autoimmunes: étude pharmacoépidémiologique Rapport final Septembre 2015 http://ansm.sante.fr/content/download/80841/1023043/version/1/file/Ansm_...
18. Fromont A, Binquet C, Sauleau E, Fournel I, Despalins R, Moreau T et al. National estimate of multiple sclerosis incidence in France (2001-2007). Mult Scler. 2012;18:1108-15.
19. Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R et al. Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database Syst Rev. 2014;4:CD008965. http://cochranelibrary-wiley.com/doi/10.1002/14651858.CD008965.pub4/full
The authors addressed important limitations in the Cochrane HPV vaccine review and stated that the Cochrane HPV vaccine review authors should make every effort to identify all trials and the trials' limitations in their conclusion. We (members of Medwatcher Japan [YAKUGAI Ombusperson], an NGO that was launched in 1997 to monitor and prevent drug-induced disasters) strongly support their conclusions.
Medwatcher Japan released an open letter (original Japanese document) ‘Critical Opinion on Cochrane Review of HPV Vaccines’ dated on June 8th, 2018 and raised some comments concerning the content of the Cochrane HPV vaccine review and the governance of the Cochrane organisation. We have submitted the open letter (English translation of the original Japanese document) to some relevant parties in charge of Cochrane on August 13th, 2018. Furthermore, we have submitted the short version which was limited to the specifics of the contents of the Cochrane HPV vaccine review via the feedback system on The Cochrane Library on August 31st, 2018. It has been confirmed that the short version was published via the Cochrane Library feedback system. We have not, however, received any responses directly from the Cochrane.
Therefore, in addition to the points raised in this article by Jørgensen et al., we would like to address the following key comments concerning the content of the Cochrane review in this response:
1. Since the effectiveness claimed in the Cochrane revi...
The authors addressed important limitations in the Cochrane HPV vaccine review and stated that the Cochrane HPV vaccine review authors should make every effort to identify all trials and the trials' limitations in their conclusion. We (members of Medwatcher Japan [YAKUGAI Ombusperson], an NGO that was launched in 1997 to monitor and prevent drug-induced disasters) strongly support their conclusions.
Medwatcher Japan released an open letter (original Japanese document) ‘Critical Opinion on Cochrane Review of HPV Vaccines’ dated on June 8th, 2018 and raised some comments concerning the content of the Cochrane HPV vaccine review and the governance of the Cochrane organisation. We have submitted the open letter (English translation of the original Japanese document) to some relevant parties in charge of Cochrane on August 13th, 2018. Furthermore, we have submitted the short version which was limited to the specifics of the contents of the Cochrane HPV vaccine review via the feedback system on The Cochrane Library on August 31st, 2018. It has been confirmed that the short version was published via the Cochrane Library feedback system. We have not, however, received any responses directly from the Cochrane.
Therefore, in addition to the points raised in this article by Jørgensen et al., we would like to address the following key comments concerning the content of the Cochrane review in this response:
1. Since the effectiveness claimed in the Cochrane review does not coincide with the target age groups of school-based HPV vaccination programs, the Cochrane review’s authors should point out that effectiveness in the main age group for HPV vaccination has not been verified.
2. Since there is an available data for females of the study V501-018, the only study which used a saline placebo as the control, the Cochrane review’s authors should try to collect the female data of the study V501-018 and include it in the review.
3. The Cochrane review’s authors did not make mention of the characteristics of occurrence of post-HPV vaccination clinical symptoms such as multi-layered occurrence of various symptoms in the long-term after vaccination in individual adolescent girls. We suggest that the discussion on the characteristics of the multi-layered occurrence of various symptoms in the long-term after HPV vaccination should be considered.
4. Despite Japanese researchers have published many papers providing evidence of the harms of HPV vaccines, those studies were not cited by the Cochrane review’s authors. Results of those studies should be cited and discussed in relation to the review findings in the Discussion section.
5. Three of the four authors of the Cochrane review (M. Arbyn, C. Simoens and PPL. Martin-Hirsch) had received travel grants from HPV vaccine manufacturers. In light of the Cochrane conflict of interest guidelines’ criteria, we believe that a COI might exist in the Cochrane review authors.
The full contents of our open letter (English translation of the original Japanese document), including an appendix table are as given below:
(PDF available from: http://www.yakugai.gr.jp/en/topics/topic.php?id=964)
‘Critical Opinion on Cochrane Review of HPV Vaccines’
Toshihiro Suzuki, Managing Director
YAKUGAI Ombudsperson "Medwatcher Japan"
1-14-4 AM building, Shinjuku, Shinjuku-ku, Tokyo, 160-0022, Japan
URL: http://www.yakugai.gr.jp/en/
1 Introduction
In May 2018, Cochrane published the findings of a systematic review of 26 randomised controlled trials to determine the efficacy and harms of human papillomavirus (HPV) vaccines [1]. The aim of a systematic review is to provide highly reliable evidence on efficacy and harms by exhaustively collecting randomised controlled trials on the efficacy of a particular medical intervention, providing a critical appraisal of each trial’s methodology, and applying appropriate statistical methodology to synthesise the results of these trials. The Cochrane HPV vaccine review, however, falls far short of an ideal systematic review and contains numerous problems that cannot be ignored. These problems are discussed below.
2 Review of Efficacy
The HPV vaccine review claims to have confirmed a reduction in the risk of developing precancerous cervical lesions (even though the longest follow-up period was 8 years) based on an evaluation of efficacy in 26 randomised controlled trials, 25 of which were industry-funded. The risk of developing precancerous cervical lesions is a surrogate endpoint, but what the review did not confirm was a reduction in cervical cancer itself as a true endpoint of efficacy. Thus, the review does not add anything new to previous claims of efficacy by the pharmaceutical industry.
The age range for HPV vaccination in Japanese clinical practice is primarily 12 to 16 years, whereas the HPV vaccine review evaluated efficacy in females aged 15 and older, and the review authors claim in their conclusion that a prophylactic effect against precancerous lesions was confirmed in females aged 15 to 26. It should therefore be noted that the efficacy claimed in the review does not coincide with the current vaccination age group in Japan. In their discussion, the authors suggest that HPV vaccines may also be effective in girls younger than 15, noting that this age group is assumed to resemble the population of HPV-negative females 15 and older in terms of efficacy, further citing research in which girls 15 and younger showed a similar immune response to females aged 15 to 26 years. However, the review evaluated efficacy only in the population aged 15 years and above and failed to point out that efficacy has not been verified in the primary age group for clinical practice in Japan.
3 Review of Harms
The HPV vaccine review investigated harms based on the number of adverse event reports in 23 of the 26 clinical trials evaluated. It found that the HPV vaccination group had no increased risk of serious adverse events compared to the control groups, and no significant increase in deaths. However, the analysis of harms in this review contained the following problems.
(1) Adjuvanted placebos or other vaccines were used as controls
Of the 23 evaluated trials, one used an unadjuvanted hepatitis A vaccine as the control and the remaining 22 trials used an adjuvanted placebo or other adjuvanted vaccine as the control (See appendix table). Considering that there appear to be risks not only from HPV-derived ingredients (L1 protein) [2] but also from powerful adjuvants, the safety of HPV vaccines should be verified by comparison between an HPV vaccine formulation (designed to maintain a high antibody titer over a long period by adding a powerful adjuvant to L1 protein) and an unadjuvanted placebo.
In fact, study V501-018 was the only study to use a saline placebo control for the quadrivalent vaccine Gardasil. However, this trial was excluded from the HPV vaccine review’s evaluation. The review authors justified this exclusion on the grounds that although the trial included both sexes, the paper presenting the trial results did not separate male and female data, and upon inquiry the authors were told that separate data was not available for females. However, considering that study V501-018 was funded and sponsored by Merck and targeted both sexes, it is difficult to imagine that separate data was not maintained for males and females. In fact, an FDA clinical review [3] presents data on adverse reactions in 320 females vaccinated with saline placebo in study V501-018. Although the FDA review only gives data on vaccination site pain, swelling and redness, the incidence of these reactions is two to three times greater in the Gardasil group than in the saline control group, suggesting that adjuvanted HPV vaccine formulations could induce a strong immune response. This exclusion, for inexplicable reasons, of a key trial from the HPV vaccine review’s evaluation of efficacy and safety, casts doubt on the reliability of the review.
(2) Post-HPV vaccination symptoms cannot be ascertained from adverse event information in the clinical trials
The characteristics of post-HPV vaccination clinical symptoms are gradually being revealed in numerous physician-reported adverse events, showing both the complexity of these symptoms and their clinical course. There are reports of a wide range of symptoms developing in a multi-layered manner over long periods spanning several months to several years (period of maintenance of high HPV antibody titer), as well as known autoimmune diseases such as complex regional pain syndrome (CRPS), chronic fatigue syndrome (CFS) and postural orthostatic tachycardia syndrome (POTS) or similar symptoms, and even symptoms such as higher cognitive impairment.
In contrast, adverse events collected in clinical trials are reported according to predetermined reporting criteria that presuppose the occurrence of known diseases and individual symptoms. Such a reporting system cannot capture post-HPV vaccination symptoms that follow a complex course characterized by multi-layered emergence of various symptoms over a long period, and it therefore follows that the true harms of HPV vaccines cannot be detected by simply comparing the incidence of individual adverse events reported in trials.
The HPV vaccine review analysed serious adverse events through a meta-analysis based on the number of reports of individual symptoms collected during each trial’s follow-up period of several months to several years. The analysis found no difference in the incidence of adverse events between the HPV vaccine and control groups, and the review seems to conclude from this result that a certain level of safety has been demonstrated. However, as already discussed, this result is only based on the number of reports of known diseases and individual symptoms, and does not capture the multi-layered occurrence of various symptoms in the long-term after HPV vaccination in individual adolescent girls. This is a point that demands further attention.
In an analysis of adverse event reports using Vigibase, Chandler and colleagues at the WHO Uppsala Monitoring Center noted that when symptoms such as headache, dizziness and syncope, or headache, dizziness and fatigue were analyzed as symptom clusters, the result was a significantly higher proportion of adverse event reports associated with HPV vaccines than with other vaccines, and the authors point out the limitations in the conventional analytical method of only extracting individual adverse events [4, 5].
(3) Much of the literature showing evidence of harms is excluded from the Discussion
As discussed above, there are some basic limitations in the evaluation of harms associated with HPV vaccines in the systematic review of 23 trials. In addition to the paper by Chandler et al, Japanese researchers alone have published many papers providing evidence of the harms of HPV vaccines. These include case reports [6, 7, 8, 9, 10, 11] by physicians who have observed in their own patients the characteristic adverse reactions described above, papers describing objective test findings of changes in the cerebrospinal fluid, brain and nervous system that can explain patients’ symptoms [12, 13, 14, 15], an animal experiment [16], and a paper reporting a temporal relationship between HPV vaccination and onset of HPV vaccine-related symptoms in vaccinated patients [17]. At the very least, the results of these studies should have been discussed in relation to the review findings in the Discussion section, but these articles were entirely ignored and excluded from the Discussion.
(4) Critical discussion of epidemiology studies or reviews by international organisations and regulatory authorities is absent
The only results considered in the Discussion section are those that deny any evidence of harm due to HPV vaccines, such as certain epidemiology studies and reviews by the CDC, EMA and other national regulatory authorities, the WHO Global Advisory Committee on Vaccine Safety (GACVS) and other international organisations. Moreover, these conclusions are accepted without any critical scrutiny.
However, the cited epidemiology studies were designed to target predefined known conditions and specific autoimmune diseases, and did not capture the characteristic post-HPV vaccination adverse reactions described above. Furthermore, even if patients with adverse reactions to an HPV vaccine were examined by a medical facility after developing a condition or autoimmune disease defined in the research, these are fundamentally difficult diseases to diagnose, and it is easy to imagine patients being given an inaccurate diagnosis. In fact, the paper by Chandler et al reports that despite POTS, CRPS and CFS being included in 20–58% of reported adverse reactions to HPV vaccines, the disease diagnosis only included these terms in 15% of 694 cases analysed using symptom cluster analysis. The opinions of the EMA and other national regulatory authorities and the statement from GACVS also rely on this kind of limited epidemiological research as their main evidence [18, 19].
Notable bias was also apparent on the part of GACVS. Vaccine makers were in constant attendance as external experts at the regular meetings of GACVS, and a Freedom of Information request brought to light improper interference by the GACVS chairperson at a meeting of the Japanese Ministry of Health, Labour and Welfare in 2014 in order to quash concerns over HPV vaccines [20]. It is therefore inadvisable to unquestioningly accept these opinions.
4 Lessons from Review of Oseltamivir (Tamiflu)
The conclusions of a Cochrane review published in 2000 claiming the efficacy and safety of zanamivir (Relenza) and oseltamivir (Tamiflu) for the prevention and treatment of influenza [21] were revised in 2009 to acknowledge that zanamivir and oseltamivir have a small effect in reducing the time needed for alleviation of influenza symptoms, and that their use for both prevention and treatment must be decided after weighing the benefits against the risks of adverse reactions. This revision process started when the Japanese physician Keiji Hayashi requested a reappraisal of the review, pointing out that the Kaiser Study in 2000 supporting the review conclusions had used data primarily from unpublished clinical trials for its evaluation. The Kaiser Study was a meta-analysis of 10 industry-funded clinical trials conducted in the latter half of the 1990s, but while two of these trials had been published in peer-reviewed journals, the other eight were unpublished or only published as abstracts. The Cochrane review team then asked the Kaiser Study authors for the data, but because the data was not forthcoming, the meta-analysis was repeated without the Kaiser Study, resulting in a very different conclusion [22, 23]. This episode demonstrates that even Cochrane Systematic Reviews are at risk of reaching the wrong conclusion if the appropriate selection and critical examination of evaluated trials are neglected.
The influenza review was redone by Tom Jefferson of the Nordic Cochrane Centre in response to criticism of the review’s findings. Also based at the Nordic Cochrane Centre, a research group led by Peter C. Gøtzsche conducted a detailed review of the EMA evaluation of HPV vaccines, and identified failings in the basic evaluation materials and review methodology, including a lack of critical scrutiny and independent re-analysis of data provided by the vaccine makers, and omission of some clinical trials in a less-than-exhaustive review [24]. It is therefore inexplicable that the HPV vaccine review should uncritically accept the EMA evaluation without mentioning these findings from the Nordic Cochrane Centre.
5 Bias in Press Release
The way in which the HPV vaccine review was announced raises questions about bias. Cochrane’s press release introduced the first author M. Arbyn as citing the GACVS statement, “…the risk-benefit profile of prophylactic HPV vaccines remains favourable and unjustified claims of harm that lack biological and epidemiological evidence may affect the confidence of the public”. The HPV vaccine review has numerous limitations, as discussed above, but characterising expressions of doubt over the risk-benefit profile of HPV vaccines as “unjustified claims of harm” is more than enough to raise questions about the neutrality of this review.
Research on capturing the actual situation and identifying causes needs to progress in order to ascertain the true harms of pharmaceuticals, and the history of pharmaceutical scandals demonstrates that this process this takes time. Therefore, the approach required when reviewing harms should be one that seeks detection of risk signals rather than evidence of safety defined as the absence of any increased risk from an intervention. However, this basic approach is completely absent in the HPV vaccine review and press release.
In addition to the press release, the Cochrane website also presents feedback from the medical community in the form of comments from physicians welcoming the review findings, but it seems inappropriate to publish comments from physicians acting as consultants for GlaxoSmithKline and Merck (known as MSD in the US and Japan).
6 Conflict of Interest
(1) Authors’ conflict of interest
At publication of the final version of the protocol, conflicts of interest had already been identified for two of the review’s six authors. When the review results were subsequently published there were four authors (one of the two protocol authors with conflicts of interest remained as the lead author of the review). In addition, three of the remaining four authors (M. Arbyn, C. Simoens and PPL. Martin-Hirsch) had received travel grants from HPV vaccine manufacturers, which in and of itself raises questions about conflict of interest.
In particular, the lead author of the review, M. Arbyn, has continued to publish papers emphasizing the efficacy of HPV vaccines as lead author or co-author of papers on HPV vaccine trials. For example, ‘Prophylactic human papillomavirus vaccines: the beginning of the end of cervical cancer’ [25], published in the International Journal of Gynecological Cancer in 2004, was co-authored by M. Arbyn with J. Paavonen, who has written papers on clinical trials of Cervarix and Gardasil. Also, ‘Review of current knowledge on HPV vaccination’ [26], published in the Journal of Clinical Virology in 2008 was co-authored by M. Arbyn with J. Dillner, who has written papers on clinical trials of Gardasil.
M. Arbyn is also closely connected with EUROGIN, an international organisation of which MSD is the Platinum sponsor [27], and he has been selected as a program committee member for EUROGIN 2018, together with the likes of X. Bosch, J. Paavonen and J. Dillner, who have all written papers on vaccine clinical trials [28]. In 2012, M. Arbyn also wrote a review in EUROGIN 2011 which expressed an optimistic view about HPV vaccines [29].
It is notable that a 2015 Cochrane review on type 2 diabetes was withdrawn when a conflict of interest emerged between the review’s authors and the pharmaceutical industry [30]. As this withdrawal demonstrates, rather than unquestioningly accepting a review by a particular research group, Cochrane has, in the past, openly debated each review’s validity and made the necessary corrections and changes, even after publication of the review plan and review findings. Thus, it is difficult to understand Cochrane Review’s inaction in the face of these obvious and documented instances of conflict of interest.
(2) Donation of 130 million yen from the Bill & Melinda Gates Foundation
In 2016, Cochrane received roughly 130 million yen from the Bill & Melinda Gates Foundation [31], which promotes vaccination. Half of this amount was spent in the same year [32]. In 2016, Cochrane reported income of 6.8 million pounds (about one billion yen) and expenditures of 8.1 million pounds (about 1.2 billion yen). This donation constitutes more than 10% of Cochrane’s income for 2016, a significant fact that cannot be ignored.
7 Conclusion
On 24 March 2018, YAKUGI Ombudsperson ‘Medwatcher Japan’ held a symposium which had invited representatives of patient groups for victims of adverse reactions to HPV vaccines from the UK, Spain, Ireland, Colombia and Japan. It became clear from this meeting that each country had witnessed the same clinical characteristics of diverse and multi-layered adverse reactions to HPV vaccines and the same dramatically high number of postmarketing adverse event reports compared to other vaccines. It also emerged that some adverse reaction victims were told that they had ‘psychological problems’ by medical professionals and were thus unable to receive appropriate medical care. Moreover, not only has industry and government failed to put any relief measures in place due to lack of recognition of any causal relationship between HPV vaccines and adverse reactions, but some victims have been labelled ‘anti-vaccine’ and falsely accused of lying about symptoms by physicians and journalists promoting the vaccine, even though these patients had chosen to have the HPV vaccination because they believed it to be safe and effective [33].
Until these problems are recognized and understood, we are concerned that Cochrane’s HPV vaccine review, which is fundamentally flawed, will be used by people and organisations seeking to promote HPV vaccines. We urge Cochrane to live up to its stated mission of being “the benchmark for high-quality information about the effectiveness of health care.”
(English translation of the original Japanese document, ‘Critical Opinion on Cochrane Review of HPV Vaccines’; released on 7 June, 2018; revised on 8 June, 2018. http://www.yakugai.gr.jp/topics/topic.php?id=956)
References
[1] Arbyn M, Xu L, Simoens C, Martin-Hirsch PPL. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Database Syst Rev. 2018; (5): CD009069.
[2] Lenz P, Day PM, Pang YY, Frye SA, Jensen PN, Lowy DR, et al. Papillomavirus-like particles induce acute activation of dendritic cells. J Immunol. 2001; 166: 5346-55.
[3] Clinical Review of Biologics License Application Supplement STN# 125126/1297.0 – male indication for GARDASIL. https://www.fda.gov/downloads/BiologicsBloodVaccines/Vaccines/ApprovedPr... [accessed: 6 August 2018]
[4] Chandler RE, Juhlin K, Fransson J, Caster O, Edwards IR, Norén GN. Current Safety Concerns with Human Papillomavirus Vaccine: A Cluster Analysis of Reports in VigiBase®. Drug Saf. 2017; 40: 81-90.
[5] Chandler RE, Edwards IR, Lindquist M. Comment on "Safety of Human Papillomavirus Vaccines: An Updated Review". Drug Saf. 2018; 41: 537-8.
[6] Kinoshita T, Abe RT, Hineno A, Tsunekawa K, Nakane S, Ikeda S. Peripheral sympathetic nerve dysfunction in adolescent Japanese girls following immunization with the human papillomavirus vaccine. Intern Med. 2014; 53: 2185-200.
[7] Yokota S, Kinoshita Y, Nakamura I, Nakajima T, Nishioka K. General overview and discussion on HPV vaccine associated neuropathic syndrome. Japan Medical Journal (Nihon Iji Shimpou) 2015; 4758: 46-53. [in Japanese]
[8] Nishioka K. Clinical characteristics of HPV vaccination associated neuro-immunopathic syndrome. Neurological medicine 2016; 85: 512-9. [in Japanese]
[9] Yokota S, Kuroiwa Y, Nakajima T, Nishioka K. Human papillomavirus vaccine-associated neuroimmunopathic syndrome: from the view points of pediatric practice. Neurological medicine 2016; 85: 520-7. [in Japanese]
[10] Takahata K, Takashima H. A proposal for a new neurological examination for discrimination of autoimmune encephalopathy and somatoform disorders. Neurological Therapeutics 2016; 33: 9-18. [in Japanese]
[11] Takahashi Y, Matsudaira T, Nakano H, Nasu H, Ikeda H, Nakaoka K, et al. Immunological studies of cerebrospinal fluid from patients with CNS symptoms after human papillomavirus vaccination. J Neuroimmunol. 2016; 298: 71-8.
[12] Kuroiwa Y, Yokota S, Nakamura I, Nishioka K. Human papilloma virus vaccination (HPV V) - associated neuro-immunopathic syndrome (HANS) : a comparative study of the symptomatic complex occurring in Japanese and Danish young females after HPV V. The Autonomic Nervous System 2018; 55: 21-30.
[13] Hirai T, Kuroiwa Y, Hayashi T, Iguchi Y. Neurological disorder after human papilloma virus vaccination: objective findings. Neurological medicine 2016; 85: 536-46.
[14] Hirai T, Kuroiwa Y, Hayashi T, Uchiyama M, Nakamura I, Yokota S, Nakajima T, et al. Adverse effects of human papilloma virus vaccination on central nervous system: Neuro-endocrinological disorders of hypothalamo-pituitary axis, Autonomic Neuroscience: Basic & Clinical 2016; 201: 74.
[15] Matsudaira T, Takahashi Y, et al. Cognitive dysfunction and regional cerebral blood flow changes in Japanese females after human papillomavirus vaccination. Neurol Clin Neurosci. 2016; 4: 220-227. doi: 10.1111/ncn3.12083.
[16] Aratani S, Fujita H, Kuroiwa Y, Usui C, Yokota S, Nakamura I, et al. Murine hypothalamic destruction with vascular cell apoptosis subsequent to combined administration of human papilloma virus vaccine and pertussis toxin. Sci Rep. 2016; 6: 36943. doi: 10.1038/srep36943. [retracted]
[17] Ozawa K, Hineno A, Kinoshita T Ishihara S, Ikeda SI. Suspected Adverse Effects After Human Papillomavirus Vaccination: A Temporal Relationship Between Vaccine Administration and the Appearance of Symptoms in Japan. Drug Saf. 2017; 40: 1219-29.
[18] Medwatcher Japan. "Refutation of GACVS (Global Advisory Committee on Vaccine Safety) statement on Safety of HPV vaccine on December17, 2015", 2016, http://www.yakugai.gr.jp/en/topics/topic.php?id=930 [accessed: 6 August 2018]
[19] Beppu H, Minaguchi M, Uchide K, Kumamoto K, Sekiguchi M, Yaju Y. Lessons learnt in Japan from adverse reactions to the HPV vaccine: a medical ethics perspective. Indian J Med Ethics. 2017; 2: 82-8.
[20] Medwatcher Japan. Open letter on the Discussion Meeting on Cervical Cancer Prevention Vaccines submitted to MHLW. 2016, http://www.yakugai.gr.jp/en/topics/topic.php?id=929 [accessed: 6 August 2018]
[21] Jefferson T, Demicheli V, Deeks J, Rivetti D. Neuraminidase inhibitors for preventing and treating influenza in healthy adults. Cochrane Database Syst Rev. 2000; (2): CD001265.
[22] Jefferson T, Jones M, Doshi P, Del Mar C. Neuraminidase inhibitors for preventing and treating influenza in healthy adults: systematic review and meta-analysis. BMJ. 2009; 339: b5106.
[23] Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R, Thompson MJ, et al. Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database Syst Rev. 2014; (4): CD008965.
[24] Jefferson T, Jørgensen L. Human papillomavirus vaccines, complex regional pain syndrome, postural orthostatic tachycardia syndrome, and autonomic dysfunction - a review of the regulatory evidence from the European Medicines Agency. Indian J Med Ethics. 2017; 2: 30-7.
[25] Tjalma WA, Arbyn M, Paavonen J, van Waes TR, Bogers JJ. Prophylactic human papillomavirus vaccines: the beginning of the end of cervical cancer. Int J Gynecol Cancer. 2004; 14: 751-61.
[26] Arbyn M, Dillner J. Review of current knowledge on HPV vaccination: an appendix to the European Guidelines for Quality Assurance in Cervical Cancer Screening. J Clin Virol. 2007; 38: 189-97.
[27] EUROGIN 2018, Sonsors. https://www.eurogin.com/2018/360-sponsors-exhibitors.html [accessed: 6 August 2018]
[28] EUROGIN 2018, Program Committee. https://www.eurogin.com/2018/341-program-committee.html [accessed: 6 August 2018]
[29] Arbyn M, de Sanjosé S, Saraiya M, Sideri M, Palefsky J, Lacey C, et al. EUROGIN 2011 roadmap on prevention and treatment of HPV-related disease. Int J Cancer. 2012; 131: 1969-82.
[30] Hemmingsen B, Lund SS, Gluud C, Vaag A, Almdal TP, Wetterslev J. Targeting intensive glycaemic control versus targeting conventional glycaemic control for type 2 diabetes mellitus. Cochrane Database Syst Rev. 2015; (7): CD008143. [withdrawn]
[31] Cochrane, Cochrane announces support of new donor. 2016. https://www.cochrane.org/news/cochrane-announces-support-new-donor [accessed: 6 August 2018]
[32] Financial report in Cochrane 2016 Annual Review. pp18. https://www.cochrane.org/sites/default/files/public/uploads/2016_annual_... [accessed: 6 August 2018]
[33] Medwatcher Japan, Rebuilding Hope Association HPV Vaccine Victims (in Colombia), AAVP (Association of Affected People Due to the HPV Vaccine in Spain) , AHVID (UK Association of HPV Vaccine Injured Daughters), REGRET (Reactions and Effects of Gardasil Resulting in Extreme Trauma, in Ireland), National Network of Cervical Cancer Vaccine Victims in Japan, National Plaintiffs Association for the HPV Vaccines Lawsuits in Japan. Joint Statement 2018 for the Victims of HPV Vaccines", 2018. http://www.yakugai.gr.jp/en/topics/topic.php?id=953 [accessed: 6 August 2018]
1Nordic Cochrane Centre, Rigshospitalet 7811, Tagensvej 21, 2100 Copenhagen, Denmark.
Summary
In a report uploaded on the Cochrane.org website on 3 September 2018 (1), Cochrane’s Editor in Chief and Deputy Editor in Chief responded to our analysis published in BMJ Evidence-Based Medicine on 27 July 2018 (2) of the Cochrane review of the HPV vaccines published on 9 May 2018 (3).
The Cochrane editors acknowledge (1) that our analysis (2) addresses the importance of the selection of data sources for reviews, and we hope that Cochrane will take the threat posed by reporting bias (4) more seriously by using clinical study reports, rather than journal publications.
The Cochrane editors claimed that we had “substantially overstated” our criticisms and they concluded that “Jørgensen et al made allegations that are not warranted and provided an inaccurate and sensationalized report of their analysis” (1).
Here we address the Cochrane editors’ findings and present our further assessment and additional findings.
In summary, we found that our analysis (2) was appropriat...
1Nordic Cochrane Centre, Rigshospitalet 7811, Tagensvej 21, 2100 Copenhagen, Denmark.
Summary
In a report uploaded on the Cochrane.org website on 3 September 2018 (1), Cochrane’s Editor in Chief and Deputy Editor in Chief responded to our analysis published in BMJ Evidence-Based Medicine on 27 July 2018 (2) of the Cochrane review of the HPV vaccines published on 9 May 2018 (3).
The Cochrane editors acknowledge (1) that our analysis (2) addresses the importance of the selection of data sources for reviews, and we hope that Cochrane will take the threat posed by reporting bias (4) more seriously by using clinical study reports, rather than journal publications.
The Cochrane editors claimed that we had “substantially overstated” our criticisms and they concluded that “Jørgensen et al made allegations that are not warranted and provided an inaccurate and sensationalized report of their analysis” (1).
Here we address the Cochrane editors’ findings and present our further assessment and additional findings.
In summary, we found that our analysis (2) was appropriate and that the Cochrane editors substantially ignored several of our criticisms (1):
1) The Cochrane editors’ cross referencing (1) with our HPV vaccine study index (5) showed that the Cochrane HPV review was incomplete and resulted in 8% additional eligible female participants (6,191/73,428). Due to the discrepancy with our analysis (2), we assessed our index again and found that the Cochrane HPV review should have included at least 35% (25,550/73,428) additional eligible females in its meta-analyses;
2) The Cochrane editors’ considerations on harms ignored several of our criticisms including the incomplete reporting of serious adverse events in several of the Cochrane HPV review’s included studies;
3) The Cochrane editors’ considerations of the trials’ adjuvant and vaccine comparators was ambiguous, opaque, inaccurate and ignored the fact that the studies only tested the vaccine antigens—not the vaccines;
4) The Cochrane editors’ response on the Cochrane HPV review’s included composite surrogate outcomes was superficial and did not consider the substantial bias and confounding that these outcomes involve;
5) The Cochrane editors’ assessment of the Cochrane HPV review authors’ conflicts of interest was incomplete and ignored several additional important conflicts of interest;
6) The Cochrane editors’ considerations on the media coverage did not recognize that it should be balanced and free from financial conflicts of interest;
7) The Cochrane editors’ response appeared to advocate scientific censorship, which we do not approve of;
8) In conclusion, our analysis (2) was appropriate, the Cochrane editors substantially ignored several of our criticisms (1) and the Cochrane review is still incomplete and ignores important evidence of bias.
1) The Cochrane editors’ cross referencing with our HPV vaccine study index
We used our index (5) to identify additional eligible studies for the Cochrane HPV vaccine review that included 73,428 women from 26 studies (3).
From our index (5), the Cochrane editors (1) identified:
1. “five [i.e., 5/26 = 19%] eligible completed studies with available data representing 5267 women [i.e., 5,267/73,428 = 7%, that] may have been missed from the Cochrane Review, as a consequence of the search being based on bibliographic databases rather than trials registers.”
2. One “Additional 9-valent [Gardasil 9] study NCT01047345,” adding 924 women to the numerator: 6,191/73,428 = 8% additional women.
3. “Five studies awaiting classification (not recruiting, but no results available) potentially relevant for the current Cochrane Review” that included 4,044 participants.
4. “Eight ongoing studies (actively recruiting, no results available) potentially relevant for the current Cochrane Review” that included 121,531 participants.
The Cochrane editors’ analysis (1) shows that the Cochrane HPV review was incomplete. In their “Appendix A”, the Cochrane editors updated 20 of the Cochrane HPV review’s meta-analyses (20/66, 30%) with the additional data and added seven meta-analyses of the HPV vaccine Gardasil 9 (1), but as of September 14 2018 they have not updated the Cochrane review itself with the additional data (3).
It is not clear why the Cochrane editors thought that our study index (5) “did not appear to identify any important eligible studies” (6). Our index was sent to the editors on 19 January 2018. The Cochrane review was published 110 days later on 9 May 2018 (3), but it seemingly took the editors only 25 days from launching their “investigation” on 9 August 2018 (6) to updating their Cochrane review on 3 September 2018 (7) with the missing studies.
Our initial assessment of the Cochrane HPV review’s included studies:
Initially, we had cross-referenced the study IDs from our index with the 26 included study IDs in the Cochrane HPV review’s “Appendix 6.1.1. Published reports included in the Cochrane review,” and found 20 studies not included in the review. For example, we did not find any of the 20 studies included in the Cochrane HPV review’s two serious adverse events analyses: “Figure 10” of journal publication data and “Analysis 7.6” of data that the Cochrane authors “considered to represent the most complete follow-up” (3). This led us to believe that the studies were not included in the Cochrane review. When we checked again, we found some of the studies in the review’s reference list (3). The Cochrane HPV reviewers chose to use idiosyncratic referencing with study IDs such as “Phase 2 trial (ph2,2v)”, “Immunobridging (ph3,2v)” and “CVT (ph3,2v), which made the study assessment complicated. For numbers of participants, we did not subtract the male participants that were included in three of the studies, as we should have done.
As stated in our index paper (5), our detective work involved a degree of uncertainty, as we did not want to dismiss any possibly eligible studies. Therefore, the index included a “possibly exist” category for studies for which we only had one verification source. Four studies in our analysis (2) had no numbers for randomised participants; these were “probably exist” studies. We have obtained additional information for three of these studies, but we are still not sure that the fourth study exists in clinical study report form, as the manufacturer (Merck) did not answer our request for this information (2,5).
In Table 1, we list the 20 studies (plus the additional one that the Cochrane editors identified: NCT01489527) that we identified as eligible but not included in the Cochrane HPV review.
Table 1: Our reassessment of the studies we had identified as additionally eligible for the Cochrane HPV vaccine review [*Key: RCT = randomised clinical trial]
Our analysis reinforces the view that Cochrane HPV review is incomplete. We found an additional 25,550 females (and possibly up to 30,195 for the Cochrane HPV review’s serious adverse events meta-analyses) that are eligible for the Cochrane HPV review’s meta-analyses. Furthermore, we found freely available clinical study reports for 6 of the 21 studies on GlaxoSmithKline’s trial register, which the Cochrane authors used data from. Clinical study reports are far more reliable than published reports (4), in particular in relation to possible serious harms. It is therefore not merely the studies the authors of the Cochrane HPV review missed; they also missed benefits and harms data from the studies they included. In addition, the criteria that the Cochrane authors used for inclusion of data in their primary serious adverse events analysis (Analysis 7.6) are not clear: “The primary analysis for these outcomes included data that we considered to represent the most complete follow-up” (3).
2) The Cochrane editors’ considerations on harms
The Cochrane HPV review’s harms analyses ("Comparison 7”) include seven meta-analyses (four of which report injection site harms) where the three most clinically important ones—deaths (Analysis 7.7), serious adverse events (Analysis 7.6) and systemic adverse events (Analysis 7.5)—contain errors or are incomplete.
Deaths
The Cochrane HPV review’s authors found 5% (90 vs. 86) more deaths in “Analysis 7.7” (journal publication and registry data) vs. “Figure 11 (only journal publication data). We found that this discrepancy reflects the difference between the study FUTURE III’s (V501-019) journal publication and registry entry (8 vs. 12 deaths). We also found that the Cochrane HPV review gave an incorrect number of deaths for the VIVIANE study (HPV-015): 13 deaths in the HPV vaccine group and 5 deaths in the AlOH3 group; according to VIVIANE’s journal publication (8), there were 14 deaths in the HPV vaccine group and 3 deaths in the Al(OH)3 group.
The Cochrane authors state that “The deaths reported in the trials had an identified cause, and none were assessed to be due to vaccination” (3), but such judgements are biased, particularly in industry sponsored trials, and the analysis of deaths should be based on all events (4).
Serious adverse events
There are 11% (4,758/4,291) more serious adverse events in “Analysis 7.6” (journal publication and register data) compared to “Figure 10” (only journal publication data). Since there were more serious adverse events and more deaths in the register entry data, we wonder why the Cochrane authors did not include register entry data for all their outcomes.
We also wonder why three (possibly four) studies were not included in the Cochrane HPV review’s serious adverse events meta-analyses (see Table 1). The Cochrane editors write that we “claim that the [Cochrane] review authors made an error in their reporting of serious adverse events in relation to the PATRICIA [HPV-008] study. This is not the case” (1). We stated that “the Cochrane authors did not explain what the serious adverse events consisted of or whether some of them were more common in the HPV vaccine groups,” (2) and gave the example that “the PATRICIA trial publication only included two thirds (1400/2028) of the serious adverse events listed on ClinicalTrials.Gov” (2). The 2,028 individual serious adverse events listed in ClinicalTrials.Gov are listed with the total denominators of randomised women for PATRICIA, for example, “Headache: participants affected/at risk: 5/9319 [in the HPV vaccine group] (0.05%) vs. 1/9325 [in the hepatitis a vaccine group] (0.01%)” (9), suggesting that the numbers represent participants with serious adverse events.
The Cochrane editors did not consider our highly relevant observations about the incomplete reporting of serious harms (1). For example, we wrote that “FUTURE I, FUTURE II and FUTURE III, which in total included 21 441 women with up to 4 years follow-up, only reported serious adverse events occurring within 14 days post-vaccination” (2). The editors did not comment on how such reporting of serious adverse events for only about 3% of the trial periods (FUTURE I, II and III: [(14 days*3 vaccinations)/(365 days*4 years)]) resulted in the Cochrane authors’ judgements of “low risk of bias” for reporting bias (1).
Both the 1st Cochrane HPV review protocol from 2011 (10) and the 2nd protocol from 2013 (11) list the primary outcome of "serious adverse events observed after four weeks of administration of the vaccine during the trial" (emphasis added), i.e., an incomplete reporting of serious adverse events was already a criterion at protocol stage.
Systemic adverse events
The Cochrane editors (1) did not comment on our criticism of the lack of studies (2)—including PATRICIA (HPV-008)—from the Cochrane HPV reviews’ “Analysis 7.5: systemic adverse events” (3). Analysis 7.5 is incomplete—in particular, for Cervarix studies where the Cochrane review only included numbers for “solicited general adverse events” for two studies (3): HPV-009 and HPV-015, although data for such events are eligible from several additional studies, for example, HPV-001, HPV-008, HPV-013, HPV-029, HPV-030, HPV-033, HPV-035, HPV-038 and HPV-058. The inclusion of these studies could change the Cochrane review’s conclusion that “Systemic events with general mild symptoms were similarly frequent in vaccinated recipients and placebo or control vaccine recipients” (3). As we wrote, “On ClinicalTrials.gov, PATRICIA has 7129 vs 6557 systemic events listed under ‘Results: Other Adverse Events (General disorders)’, which in itself is a significantly increased risk: RR 1.09 (95% CI 1.07 to 1.11)” (2).
Furthermore, the Cochrane authors did not address that “solicited general adverse events [Cervarix]” only were reported 7-days post-vaccination and “systemic adverse events [Gardasil]” only 14-days post-vaccination (3).
Assessment of safety signals
The Cochrane editors did not comment on our safety signal section (1). Some potential HPV vaccine-related harms to the nervous system—or “autonomic dysfunction syndromes,” as the Cochrane editors described them (1)—have been reported (2). The Cochrane authors should have used trial register data to investigate such safety signals; for example, if they had summarised the nervous system disorders from PATRICIA’s (HPV-008) ClinicalTrials.gov list of serious adverse events (9), they would have found more serious nervous system disorders in the HPV vaccine arm: 39/9,319 vs. 25/9,325 in the hepatitis A vaccine arm, risk ratio 1.56 (95% CI 0.95 to 2.58). The Cochrane authors write that “All estimates of adverse effects in our review were restricted to those reported from randomised trials and therefore could not detect rare events, for which post-marketing surveillance, pharmacovigilance activities and linkage studies, joining vaccine and morbidity registries, are needed” (3), but the review’s results might have been different had the authors included serious adverse events on both an individual and organ system level.
Additional points on the Cochrane editors’ harms assessment
The Cochrane HPV review (3) did not include the following harms categories that were reported in the eligible studies’ clinical study reports and in some journal publications: “unsolicited adverse events” (Cervarix), “medically significant conditions” (Cervarix), “new onset chronic/auto-immune disease” (Cervarix) and “new medical history” (Gardasil), even though the Cochrane authors mention the two first categories (reported in “Angelo 2014”) (3).
3) The editors’ considerations of the trials’ adjuvant and vaccine comparators
The Cochrane editors stated that “The trials comparators [sic] were unambiguously, transparently, and accurately described” (1), but in the Cochrane HPV review’s “Plain language summary” intended for lay readers, the review authors state that “The risk of serious adverse events is similar in HPV and control vaccines (placebo or vaccine against another infection than HPV” (emphasis added) (3), and the word “placebo” is repeated throughout the review and all its meta-analyses, which make the review ambiguous, opaque and inaccurate, as no included trial in the review used a placebo comparator.
The WHO states that using adjuvant or another vaccine as comparators instead of placebo makes it difficult to assess the harms of a vaccine (12). The HPV vaccine trials’ adjuvant comparators—Merck’s amorphous aluminium hydroxyphosphate sulphate (AAHS) and GlaxoSmithKline’s aluminium hydroxide (Al[OH]3)—have not been tested against an inert comparator in human trials. The adjuvants’ clinical properties are largely unknown; they are not regulated on their own, as regulators do not regard them as “active ingredients” (13). For example, Merck’s AAHS has a confidential formula and its properties are variable from batch to batch and even within batches (14). Because the HPV vaccines and their adjuvants had similar harms profiles, the manufacturers and the regulators concluded that the HPV vaccines are safe. However, this is like saying that cigarettes and cigars must be safe because they have similar harms profiles.
In addition, in those trials with a non-HPV vaccine comparator, the HPV vaccine aluminium adjuvant was used in nearly all the non-HPV comparator vaccines; for example, PATRICIA’s hepatitis A (Havrix) comparator contains Al(OH)3 (only the studies HPV-032 and HPV-063 used a non-aluminium containing comparator: the hepatitis a vaccine Aimmugen). Thus, the presence of AAHS or Al(OH)3 in nearly all arms of the studies thwarted the harms assessment. The studies tested the vaccine antigens—not the vaccines.
The exclusion criteria of the Cochrane HPV review’s included trials
The Cochrane editors (1) did not consider our point that many of the Cochrane HPV review’s included studies had excluded female participants “if they had received the [aluminium] adjuvants before or had a history of immunological or nervous system disorders; for example, in the PATRICIA trial with 18 644 women and the FUTURE II trial with 12 167 women” (2). These exclusion criteria lower the external validity of the studies and suggest that there were concerns about harms caused in such participants by the adjuvants.
4) The Cochrane editors’ response on the Cochrane HPV review’s included outcomes
The Cochrane editors state that “The selection of outcomes for benefits was appropriate and was consistent with World Health Organization [WHO] guidance” (1).
In 2004, the WHO recommended the use of cervical intraepithelial neoplasia or worse: CIN2+, as the primary outcome (15). CIN2+ is a composite surrogate outcome for cervical cancer and includes CIN2, CIN3, AIS and cervical cancer. In 2014, the WHO recommended persistent HPV infection instead of CIN2+ (16). The WHO’s CIN2+ and persistent HPV infection recommendations were approved to “accelerate vaccine development and evaluation” (16). Since 2014, HPV vaccines have only been required to show benefits against persistent HPV infection for getting regulatory approval as a vaccine against HPV related cancer (16).
According to the 2004 WHO recommendations, “Representatives of industry did NOT participate in the drafting of recommendations” of the use of CIN2+ (15), but researchers with conflicts of interest did participate in the recommendations, for example, Ian Frazer—the co-inventor of the HPV vaccine who “receives royalties from sales of HPV prophylactic vaccines, and is a consultant for Merck, [and] GlaxoSmithKline” (17). In 2014, all 17 members of the WHO group that recommended persistent HPV infection instead of CIN2+ as the primary outcome had financial ties with the HPV vaccine manufacturers. For example, the group included two patent-holders of the HPV vaccines antigens (or “virus-like particles”), who are entitled “to a limited share of royalties [that] the NIH [National Institutes of Health] receives for these technologies” (16).
Outcomes such as CIN2+ can be difficult to interpret, and significant clinical differences can be hidden in the Cochrane HPV review’s meta-analyses (3). For example, as an extreme example, if there were 5 participants with CIN2+ in the HPV vaccine group and 10 in the comparator group, the 5 participants in the HPV vaccine group could theoretically all have cervical cancer while the 10 in the comparator group could have CIN2 lesions that often regress (18,19).
The Cochrane editors (1) did not address our point that the VIVIANE study (HPV-015) included in its register entry “one case of ‘Adenocarcinoma of the cervix’ and one case of ‘Cervix cancer metastatic’ … in the HPV vaccine group” (2), and that the Cochrane HPV review includes a death caused by “Cervix cancer metastatic” in the HPV vaccine group, which was not mentioned in the main text (3). The Cochrane editors (1) did not address our point that the “Cochrane review’s 26 trials mainly included women below age 30 and used frequent cervical screening (often every 6 months) that did not reflect real-life practice (often every 3–5 years),” which also lower the external validity of the studies (2).
The Cochrane HPV review’s primary analysis—Analysis 1.1 that includes four trials (CVT, FUTURE I, II and PATRICIA)—was of “High‐grade cervical lesions in hr[high risk]HPV DNA negative women at baseline: CIN2+ associated with HPV 16/18 [HPV types 16 and 18 are targeted by the HPV vaccines]”. Analysis 1.1 is affected by selection bias. Up to 15% of cervical cancers may not contain HPV (20), and many cervical cancers are infected with more than one HPV type. For example, in the clinical study report that we received for PATRICIA (HPV-008), 63 of the 102 of CIN2+ cases were co-infected with two or more HPV types. In PATRICIA, if an HPV vaccine and a comparator participant were both diagnosed with CIN2+ and positive for HPV types 31 and 33, and HPV 16/18, 31 and 33, respectively, the HPV vaccine CIN2+ case would be assigned as caused by non-vaccine types (31 and 33) and excluded from the analyses (such as Analysis 1.1), while the comparator case would be caused by HPV 16/18 (HPV vaccine types) and included in the analyses; even though HPV types 31 and 33 could have caused the CIN2+ lesions in both participants. The Cochrane HPV review’s analyses of HPV infection (“Comparison 4, 5 and 6”) include 21 meta-analyses that all analyse infection of HPV vaccine types (i.e., HPV types 6, 11, 16 and 18)—not infection of any HPV type, which would decrease confounding by HPV co-infection.
Another issue with Analysis 1.1 (3) is the large proportion of excluded females: 23,676 participants were included, but the included four studies (CVT, FUTURE I, II and PATRICIA) randomised 43,732 participants, so 46% females were excluded. The Cochrane authors did not mention that FUTURE I, II and PATRICIA—that contained 49% (36,266/73,428) of the Cochrane review’s sample (3)—were stopped early when HPV 16/18-related CIN2+ was significantly reduced for the HPV vaccine populations. Trials stopped early for benefits are known to exaggerate the effects by 29% on average compared to completed trials of the same intervention (21).
The majority (24 of 31) of the Cochrane HPV review’s meta-analyses of histological outcomes (“Comparison 1, 2 and 3”) consider cervical lesions associated with HPV vaccine types (3). A less biased meta-analysis of cervical lesions is “Analysis 3.7: High‐grade cervical lesions in women regardless of baseline HPV DNA status: Any CIN2+ irrespective of HPV types, at least 1 dose.” The Cochrane HPV review’s primary analysis—Analysis 1.1—is much more statistically significant than Analysis 3.7 (3): risk ratio 0.01 (95% CI 0.01 to 0.05) vs. 0.79 (95% CI 0.65 to 0.97), i.e., a ratio of relative risks of 0.01 (95% CI 0.006 to 0.03), which may reflect the selection bias in Analysis 1.1.
Both the 1st Cochrane HPV review protocol from 2011 (10) and the 2nd protocol from 2013 (11) list "Invasive cervical cancer" as a primary outcome. The protocols state that the Cochrane authors “will contact study authors or data owners to request data on the outcomes that were not reported” (11), which they did not do for invasive cervical cancer. Also, “If data are reported for grouped end points, we will contact trial authors or data owners to request data on the separated outcomes” (11), which the authors did not do for CIN2+ and CIN3+. But the Cochrane authors could have looked in the journal publications; for example, CIN3 irrespective of HPV type in intention-to-treat populations was reported in FUTURE I (22) (“79/2723 [in the HPV vaccine group] vs. 72/2732 [in the AAHS group]”) and FUTURE II (23) (“127/6087 [in the HPV vaccine group] vs. 161/6080 [in the AAHS group]”: in total 206/8,810 vs. 233/8,812; risk ratio 0.91 [95% CI 0.66 to 1.27]).
Furthermore, the Cochrane authors write that “No results were found for the outcomes any [sic] CIN3+ or AIS+ irrespective of HPV type” (3). If the Cochrane authors had looked in the freely available clinical study reports on GlaxoSmithKline’s trial register that the authors assessed, they would have found the outcomes CIN3+ irrespective of HPV type for PATRICIA (86/9,319 vs. 158/9,325) and HPV-032/063 combined (9/464 vs. 14/463).
5) The Cochrane editors’ assessment of the Cochrane authors’ conflicts of interest
The Cochrane editors state that “The review was compliant with Cochrane’s current conflict of interest policy” (1). If that is the case, we believe Cochrane should reconsider its policy.
The Cochrane HPV vaccines are expensive blockbuster vaccines generating billions of dollars of revenue (24), and the Cochrane review ought, therefore, to have been independent of any financial conflicts of interests.
The Cochrane editors are confident that the Cochrane authors have no relevant conflicts of interest (2). We do not agree. For example, the Cochrane HPV review’s first author, Professor Marc Arbyn—who, according to the Cochrane review (3), only “received travel grants from MSD‐Sanofi‐Pasteur and GSK, (ceased in 2008)”—was until 2008 on GlaxoSmithKline’s advisory board: “Marc Arbyn (GSK advisory Board (interrupted in 2008))”) (25); in 2011, “EUROGIN covered his [Marc Arbyn’s] travel and lodging expenses … EUROGIN conferences are financially supported by a range of pharmaceutical companies with an interest in cervical cancer” (26); in 2014, “Marc Arbyn's research unit at The Scientific Institute of Public Health received research support not exceeding 48,000 Euros from MSD-Sanofi Pasteur [co-manufacturer of Gardasil] for a surveillance study of the effects of HPV vaccination in Belgium (SEHIB study) ... His [Marc Arbyn’s] unit has also received research support from BD, Bio-Greiner, Abbot, and Cepheid for validation studies of HPV genotyping tests (through the VALGENT studies, valued at 21,000, 21,000 & 38,000€ respectively)” (27); and in 2018, Marc Arbyn is on the EUROGIN programme committee where Merck is a platinum sponsor (28). The Cochrane review’s last author, Dr. Markowitz is sponsored by Merck via Medscape (“sponsored by the manufacturer of the quadrivalent vaccine (“supported by an independent educational grant from Merck”) (29).
The Cochrane editors (1) do not think that the Costa Rica trial (“CVT”, aka HPV-009) was industry funded, and they refer to its publication in JAMA that states that the trial was "funded by the NCI (grant N01-CP-11005).” The editors write, with reference to JAMA, that “Vaccine was provided for our trial by GSK [GlaxoSmithKline] Biologicals, under a Clinical Trials Agreement with the NCI" (1). GlaxoSmithKline also provided support for aspects of the trial associated with regulatory submissions under “FDA BB-IND 7920” (30). We consider this industry funding.
6) The Cochrane editors’ assessment of the media coverage
The Cochrane editors (1) did not comment on our note that “Two of the experts had financial conflicts of interest with the HPV vaccine manufactures … [and that] No expert criticised the review” (2).
The editors (1) write that “press coverage could be made more explicit on our organizational websites and other communications, essentially noting that these opinions represent personal perspectives from a range of contributors and do not reflect the views or policies of Cochrane” (1). We agree, but stress that Cochrane’s press officer ought to only include researchers with no financial conflicts of interest.
7) The Cochrane editors appear to advocate scientific censorship
The Cochrane editors wrote that “Scientific debate is to be welcomed, and differences of opinion between different Cochrane 'voices' is not unexpected. However, public confidence may be undermined, unnecessary anxiety caused, and public health put at risk if that debate is not undertaken in an appropriate way. This is especially true when such debates take place in public. There is already a formidable and growing anti-vaccination lobby. If the result of this controversy is reduced uptake of the vaccine among young women, this has the potential to lead to women suffering and dying unnecessarily from cervical cancer.” We believe that our criticism of the Cochrane HPV review is appropriate and has general interest. We believe that providing an assessment of all the evidence reduces uncertainty and allows the public to make informed decisions based on the benefits and the harms of HPV vaccines. Debates over sources of evidence must take place in public, especially when public health interventions are at stake.
8) Conclusion
We did not “substantially overstate” (1) our criticisms of the Cochrane HPV vaccine review (2). The Cochrane editors substantially ignored several of our criticisms. The Cochrane HPV review is still incomplete and ignores important evidence of bias.
The Cochrane editors stated that “Some of the criticisms will inform the next version of this Cochrane Review and the planned review of comparative studies of HPV vaccines,” and that the editors “recognize public concerns about the aluminium-based adjuvants” (1).
The editors also stated that “reliance on the published reports in scientific journals may introduce bias due to incomplete and selective reporting” (1). We agree and remind the Cochrane editors that the Cochrane review on neuraminidase inhibitors substantially changed its conclusions after it got updated and became based on clinical study reports instead of journal publications (31).
With our analysis (2), we have contributed to a scientific debate in an area that is complex and biased. The Cochrane authors stated that they will make a “Request for non‐published available data” such as clinical study reports that “will be integrated in future updates of the review” (3). We can offer them these data, which we have used for our own systematic review that we have submitted for publication.
Article info
Conflicts of interest: LJ and PCG have no conflicts of interest to declare. TJ was a recipient of a UK National Institute for Health Research grant for a Cochrane review of neuraminidase inhibitors for influenza. In addition, TJ receives royalties from his books published by Il Pensiero Scientifico Editore, Rome and Blackwells. TJ is occasionally interviewed by market research companies about phase I or II pharmaceutical products. In 2011-13, TJ acted as an expert witness in litigation related to the antiviral oseltamivir, in two litigation cases on potential vaccine-related damage and in a labour case on influenza vaccines in healthcare workers in Canada. He has acted as a consultant for Roche (1997-99), GSK (2001-2), Sanofi-Synthelabo (2003), and IMS Health (2013).In 2014 he was retained as a scientific adviser to a legal team acting on oseltamivir. TJ has a potential financial conflict of interest in the drug oseltamivir. In 2014-16, TJ was a member of three advisory boards for Boerhinger Ingelheim. TJ was holder of a Cochrane Methods Innovations Fund grant to develop guidance on the use of regulatory data in Cochrane reviews. TJ was a member of an independent data monitoring committee for a Sanofi Pasteur clinical trial on an influenza vaccine. Between 1994 and 2013, TJ was the coordinator of the Cochrane Vaccines Field. TJ was a co-signatory of the Nordic Cochrane Centre Complaint to the European Medicines Agency (EMA) over maladministration at the EMA in relation to the investigation of alleged harms of HPV vaccines and consequent complaints to the European Ombudsman. TJ is co-holder of a John and Laura Arnold Foundation grant for development of a RIAT support centre (2017-2020) and Jean Monnet Network Grant, 2017-2020 for The Jean Monnet Health Law and Policy Network. TJ is an unpaid collaborator to the project Beyond Transparency in Pharmaceutical Research and Regulation led by Dalhousie University and funded by the Canadian Institutes of Health Research (2018-2022).
Authors' experience: PCG has co-authored 17 Cochrane reviews, including several reviews based on clinical study reports. TJ has co-authored 17 Cochrane reviews including the first Cochrane review that used clinical study reports. LJ has co-authored several articles on the HPV vaccines.
Authors' contributions: LJ wrote the first draft. LJ, PCG and TJ contributed to the conception, drafting,
critical revision for important intellectual content and the final approval of the article.
Acknowledgements: We would like to thank the Cochrane editors for taking our analysis into consideration and the BMJ-EBM editors for allowing us to respond via their journal.
Provenance and peer review: Commissioned; not externally peer-reviewed.
References
1. Tovey D, Soares-Weiser K. Cochrane’s Editor in Chief responds to BMJ EBM article criticizing HPV review. 2018; 3 Sept. https://www.cochrane.org/news/cochranes-editor-chief-responds-bmj-ebm-ar....
2. Jørgensen L, Gøtzsche PC, Jefferson T. The Cochrane HPV vaccine review was incomplete and ignored important evidence of bias. BMJ Evidence-Based Medicine. 2018 Jul 27;bmjebm-2018-111012.
3. Arbyn M, Xu L, Simoens C, Martin‐Hirsch PP. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. John Wiley & Sons, Ltd; 2018. Available from: http://cochranelibrary-wiley.com/doi/10.1002/14651858.CD009069.pub3/full
4. Golder S, Loke YK, Wright K, Norman G. Reporting of Adverse Events in Published and Unpublished Studies of Health Care Interventions: A Systematic Review. PLOS Medicine. 2016 Sep 20;13(9):e1002127.
5. Jørgensen L, Gøtzsche PC, Jefferson T. Index of the human papillomavirus (HPV) vaccine industry clinical study programmes and non-industry funded studies: a necessary basis to address reporting bias in a systematic review. Systematic Reviews. 2018 Jan 18;7:8.
6. Hawkes N. HPV vaccine safety: Cochrane launches urgent investigation into review after criticisms. BMJ. 2018 Aug 9;362:k3472.
7. Cochrane’s Editor in Chief responds to BMJ EBM article criticizing HPV review. Available from: /news/cochranes-editor-chief-responds-bmj-ebm-article-criticizing-hpv-review
8. Skinner SR, Szarewski A, Romanowski B, Garland SM, Lazcano-Ponce E, Salmerón J, et al. Efficacy, safety, and immunogenicity of the human papillomavirus 16/18 AS04-adjuvanted vaccine in women older than 25 years: 4-year interim follow-up of the phase 3, double-blind, randomised controlled VIVIANE study. Lancet. 2014 Dec 20;384(9961):2213–27.
9. Human Papilloma Virus (HPV) Vaccine Efficacy Trial Against Cervical Pre-cancer in Young Adults With GlaxoSmithKline (GSK) Biologicals HPV-16/18 - Study Results - ClinicalTrials.gov. Available from: https://clinicaltrials.gov/ct2/show/results/NCT00122681
10. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors - Arbyn, M - 2011 | Cochrane Library. Available from: https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD009069/full
11. Arbyn M, Bryant A, Martin-Hirsch PP, Xu L, Simoens C, Markowitz L. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Database of Systematic Reviews. John Wiley & Sons, Ltd; 2013. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009069.pub2/abstract
12. Expert consultation on the use of placebos in vaccine trials. World Health Organization. 2013. Available from: apps.who.int/iris/bitstream/handle/10665/94056/9789241506250_eng.pdf?sequence=1
13. Krause P. Update on Vaccine Regulation: Expediting vaccine development. FDA/CBER/OVRR. Available from: https://c.ymcdn.com/sites/www.casss.org/resource/resmgr/CMC_Euro_Speaker...
14. Thiriot DS, Ahl PL, Cannon J, Lobel GM. Method for preparation of aluminum hydroxyphosphate adjuvant. WO2013078102A1, 2013. Available from: https://patents.google.com/patent/WO2013078102A1/en
15. Pagliusi SR, Teresa Aguado M. Efficacy and other milestones for human papillomavirus vaccine introduction. Vaccine. 2004 Dec 16;23(5):569–78.
16. IARC HPV Working Group. Primary End-points for Prophylactic HPV Vaccine Trials [Internet]. Lyon (FR): International Agency for Research on Cancer; 2014. Available from: http://www.ncbi.nlm.nih.gov/books/NBK304971/
17. Trimble CL, Frazer IH. Development of therapeutic HPV vaccines. Lancet Oncol. 2009 Oct;10(10):975–80.
18. Motamedi M, Böhmer G, Neumann HH, von Wasielewski R. CIN III lesions and regression: retrospective analysis of 635 cases. BMC Infect Dis. 2015 Nov 21;15. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4654894/
19. Moscicki A-B, Ma Y, Wibbelsman C, Darragh TM, Powers A, Farhat S, et al. Rate of and Risks for Regression of CIN-2 in adolescents and young women. Obstet Gynecol. 2010 Dec;116(6):1373–80.
20. de Sanjose S, Quint WG, Alemany L, Geraets DT, Klaustermeier JE, Lloveras B, et al. Human papillomavirus genotype attribution in invasive cervical cancer: a retrospective cross-sectional worldwide study. Lancet Oncol. 2010;11:1048–56.
21. Bassler D, Briel M, Montori VM, Lane M, Glasziou P, Zhou Q, et al. Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA. 2010 Mar 24;303(12):1180–7.
22. Garland SM, Hernandez-Avila M, Wheeler CM, Perez G, Harper DM, Leodolter S, et al. Quadrivalent vaccine against human papillomavirus to prevent anogenital diseases. N Engl J Med. 2007 May 10;356(19):1928–43.
23. FUTURE II Study Group. Quadrivalent vaccine against human papillomavirus to prevent high-grade cervical lesions. N Engl J Med. 2007 May 10;356(19):1915–27.
24. Clendinen C, Zhang Y, Warburton RN, Light DW. Manufacturing costs of HPV vaccines for developing countries. Vaccine. 2016 Nov 21;34(48):5984–9.
25. De Belgische Kamer van volksvertegenwoordigers. Available from: https://www.dekamer.be/kvvcr/showpage.cfm?section=qrva&language=nl&cfm=q...
26. Information NC for B, Pike USNL of M 8600 R, MD B, Usa 20894. Declarations of interest. World Health Organization; 2013. Available from: https://www.ncbi.nlm.nih.gov/books/NBK195238/
27. Cancer Screening in the European Union (2017) Report on the implementation of the Council Recommendation on cancer screening. Health and Food Safety. Available from: https://docplayer.net/43107833-Cancer-screening-in-the-european-union-20...
28. EUROGIN 2018. Available from: https://www.eurogin.com/2018/341-program-committee.html
29. Conflicts of interests. Available from: http://www.nogracias.eu/wp-content/uploads/2013/05/vacc-HPV-Cochrane_HPV...
30. Herrero R. Human Papillomavirus (HPV) Vaccines: Limited Cross-Protection against Additional HPV Types. J Infect Dis. 2009 Apr 1;199(7):919–22.
31. Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R, Thompson MJ, et al. Neuraminidase inhibitors for preventing and treating influenza in adults and children. The Cochrane Library. John Wiley & Sons, Ltd; 2014. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD008965.pub4/full
Back in May I submitted a comment to the review by Aubyn et al, and when it was not published received an assurance from Cochrane admin that was because the site was undergoing up-date. I am dismayed but not surprised to find that it never appeared, though it is still germane:-
"It concerns me that the authors of this study failed to contest the validity of trials that did not mostly take place against genuine placebo (i.e. saline), and even use the term "placebo" repeatedly. Do they not bear a huge responsibility by not taking this issue on, as if it is all right for this kind of methodology to become a norm of modern "scientific" practice and as something which does not affect the meaning and usefulness of the safety results?
"Ordinary citizens, most of whom will have learned about good scientific practice at school, might assume that these trials were against genuine placebo and I find no clear explanation of the issue here, let alone in the "Plain Language Summary"."
I was obviously stating the issue in a simplistic manner. The public probably mostly believe that the science behind these products would have been conducted at the highest standard before marketing, but this is very far from evident - and evident at a level which they are more than capable of understanding. Yet they continue to be shielded from the truth. The authors and this journal ought to be commended for standing against the profession...
Back in May I submitted a comment to the review by Aubyn et al, and when it was not published received an assurance from Cochrane admin that was because the site was undergoing up-date. I am dismayed but not surprised to find that it never appeared, though it is still germane:-
"It concerns me that the authors of this study failed to contest the validity of trials that did not mostly take place against genuine placebo (i.e. saline), and even use the term "placebo" repeatedly. Do they not bear a huge responsibility by not taking this issue on, as if it is all right for this kind of methodology to become a norm of modern "scientific" practice and as something which does not affect the meaning and usefulness of the safety results?
"Ordinary citizens, most of whom will have learned about good scientific practice at school, might assume that these trials were against genuine placebo and I find no clear explanation of the issue here, let alone in the "Plain Language Summary"."
I was obviously stating the issue in a simplistic manner. The public probably mostly believe that the science behind these products would have been conducted at the highest standard before marketing, but this is very far from evident - and evident at a level which they are more than capable of understanding. Yet they continue to be shielded from the truth. The authors and this journal ought to be commended for standing against the professional tide.
I also note the forthcoming exhaustive documentary history 'The HPV Vaccine On Trial: Seeking Justice for a Generation Betrayed', by Mary Holland, Kim Mack Rosenberg and Eileen Iorio (with an introduction by Luc Montagnier) to be published on 2 October.
These matters must be hidden from public view any longer.
"In older women, vaccinated between 25 to 45 years of age, the effects of HPV vaccine on precancer are smaller, which may be due to previous exposure to HPV. The risk of precancer associated with HPV16/18 is probably reduced from 145/10,000 in unvaccinated women to 107/10,000 women following HPV vaccination (moderate certainty). The risk of any precancer is probably similar between unvaccinated and vaccinated women (343 versus 356/10,000, moderate certainty)."
Cochrane estimates a significant effect of the vaccine on HPV 16/18 associated precancer in the 25-45 age group, but that effect disappears when risk of precancer of all (HPV) types is calculated. It is a matter of simple logic that for the positive (desired) effect on type 16/18 precancer to be offset in this way, the vaccine must have a corresponding negative (undesired) effect on precancer associated with other HPV types.
The observation holds for all age groups, but is more pronounced in the older group. It is reasonable to assume that if one took, say, a 35 to 45 years old group, the total undesired result of the HPV vaccination could be statistically significant.
If my reading of these numbers is too simplistic, it would be good to get a clarification, since the quoted passage appears right in the Summary with no further comments.
In Europe, the vaccine is routinely recommended to women of all ages.
"In older women, vaccinated between 25 to 45 years of age, the effects of HPV vaccine on precancer are smaller, which may be due to previous exposure to HPV. The risk of precancer associated with HPV16/18 is probably reduced from 145/10,000 in unvaccinated women to 107/10,000 women following HPV vaccination (moderate certainty). The risk of any precancer is probably similar between unvaccinated and vaccinated women (343 versus 356/10,000, moderate certainty)."
Cochrane estimates a significant effect of the vaccine on HPV 16/18 associated precancer in the 25-45 age group, but that effect disappears when risk of precancer of all (HPV) types is calculated. It is a matter of simple logic that for the positive (desired) effect on type 16/18 precancer to be offset in this way, the vaccine must have a corresponding negative (undesired) effect on precancer associated with other HPV types.
The observation holds for all age groups, but is more pronounced in the older group. It is reasonable to assume that if one took, say, a 35 to 45 years old group, the total undesired result of the HPV vaccination could be statistically significant.
If my reading of these numbers is too simplistic, it would be good to get a clarification, since the quoted passage appears right in the Summary with no further comments.
In Europe, the vaccine is routinely recommended to women of all ages.
References
[1] Arbyn M, Xu L, Simoens C, Martin-Hirsch PPL. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Database Syst Rev. 2018; (5): CD009069.
The article “Drug discovery today: no molecules required” (2018) (by Panchin AY, Khromov-Borisov NN, Dueva EV) questions the imperfections in reviewing scientific publications. The authors use the example of publications on released-active drugs (RA-drugs) as a basis for the paper and their accusations against journal editorial boards. The authors postulate that these drugs do not contain any active ingredients due to the technology used and therefore come to the conclusion that the reviewers and journal editors who have published articles on RA-drugs carried out their work appallingly and made gross mistakes. So the authors state «…the vast amount of flawed publication claiming therapeutic properties and physiological effects of drugs with no active components can highlight the problems of peer-review standards and policies in biomedical journals».
Show MoreThis particular conclusion is based on approximate theoretical calculations and not on experimental evidence: «If one assumes a 1M initial antibodies concentration (though this is usually not disclosed in release-activity papers) and takes into account Avogadro’s constant (~6.02×10^23 mol^(−1)), then even C12 dilutions are unlikely to contain any antibody molecules». In their article, the authors present these calculations as the main argument to complain to journals. It is noteworthy that in the article the authors do not provide any data or references to any studies conducted by themselves. Neither do the authors provid...
Sergey Tarasov responded to our article “Drug discovery today: no molecules required” with claims of mistakes in our analysis. We are sure that we did not conceal anything of relevance or make any mistakes. This is a response to the claims by Sergey Tarasov.
1. Tarasov claims that one of us has an undeclared conflict of interest: “However, one of the authors of the article, Khromov-Borisov NN, is a defendant in a defamation lawsuit brought by Materia Medica (June 28, 2018). Materia Medica has sued Khromov-Borisov NN <…> this therefore raises serious doubts over the objectivity and impartiality of one of the authors of the article, who also carried out the data analysis presented in supplementary letter 1”. However, the analysis provided by Khromov-Borisov NN was sent to the editor of International Journal of Diabetes Research on Dec 13, 2017 (Supplementary Letter 1). Then editors of Drug Discovery Today (Feb 5, 2018, Supplementary Letter 5), BMJ (Apr 2, 2018), and BMJ EBM assessed the article including analysis mentioned above. Thus Tarasov presented the chronology completely wrong. First we have submitted our criticism of release-activity (RA) to scientific journals and only then Materia Medica sued one of our co-authors.
2. We don’t see how any of our previous criticisms of RA drugs and homeopathy creates a conflict of interest or are relevant to the reception of our article.
3. We have stated that RA drugs contain no active substance. Simple...
Show MoreI realise there is a lot that has been written on this issue, more than it is possible for someone like me to work through in detail. However, I must point out some factual errors in the Jørgensen et al response dated 17th September.
I was rather surprised at the level of detail of some of the critique, which even drilled down to cross-checking the exact number of reported deaths in dozens of individual trials. I therefore decided to look at one of the Jørgensen et al claims, and opted for one that I could verify pretty easily, namely the number of deaths in the VIVIANE study.
https://www.ncbi.nlm.nih.gov/pubmed/27373900
Jørgensen et al stated that they "found that the Cochrane HPV review gave an incorrect number of deaths for the VIVIANE study (HPV-015): 13 deaths in the HPV vaccine group and 5 deaths in the AlOH3 group; according to VIVIANE’s journal publication (8), there were 14 deaths in the HPV vaccine group and 3 deaths in the Al(OH)3 group."
A check of their cited reference (8) demonstrates that Jørgensen et al have sourced the 48 week interim 2014 data analysis for this study, rather than sourcing the correct data from the final 84 week data analysis published in 2016, which is quite properly what the Cochrane Review authors sourced.
The correct citation source states: "Data for serious adverse events related to the vaccine and deaths continued to be collect...
Show MoreIn general there is often not enough attention paid to the difference of clinical and statistical significance. I do not understand, however, how the authors of the letter conclude that the clinical effect in CASTE-AF was "tiny".
The application of Cohen's d to a study with a discrete outcome is rather unusual. A more intuitive approach, if one is unfamilar with hazards, would be to use the relative risk as outcome measure. The risk of the primary outcome in the ablated group is 28.5%, the risk in the comparison group is 44.6%. The relative risk is 0.64. The absolute risk reduction is 16% and the number needed to treat is 6.25 (for three years). Not a tiny effect.
The problems associated with NOT using a true placebo in clinical trials - in this case GSK's trial for Cervarix - were discussed way back in 2009:
FOOD AND DRUG ADMINISTRATION CENTER FOR BIOLOGICS EVALUATION AND RESEARCH
VACCINES AND RELATED BIOLOGICAL PRODUCTS ADVISORY COMMITTEE
September 9, 2009
Pg166
DR. DEBOLD: I just think in the absence of having a true placebo in the entire study, it is very, very difficult to sort out what the effects are here, what is it that we are really dealing with, what is the baseline, versus what potentially is caused by the so-called controls.
In this particular study, not only was there two different strengths of Havrix used, there was alum used. In the pooled analysis there were a number of other vaccines used as the control, which makes it scientifically very difficult to sort things out.
Dear Dr. David Tovey,
I have read your response [1,2] to the paper by Jøorgensen et al. published in BMJ Evidence-Based Medicine [3] relating to the recently published Cochrane Review on HPV vaccines [4], and would like to give my feedback on this issue.
The key findings of your investigations are as follows:
1. The Cochrane Review did not miss "nearly half of the eligible trials". A small number of studies were missed due to the primary focus on peer-reviewed reports in scientific journals, but addition of these data makes little or no difference to the results of the review for the main outcomes;
2. The trials comparators were unambiguously, transparently, and accurately described;
3. The selection of outcomes for benefits was appropriate and was consistent with World Health
Organization guidance;
4. The review included published and unpublished data on serious harms, and the findings on
mortality were reported transparently and responsibly;
5. The review was compliant with Cochrane’s current conflict of interest policy;
6. Cochrane’s media coverage was cautious and balanced, but we recognize that there could be
improvements in relation to transparency where external experts are quoted;
7. The BMJ Evidence-Based Medicine article substantially overstated its criticisms.
I would like to comment on your findings 1, 2, 4 and lastly, I added comments as 8. Most observational studies neg...
Show MoreThe authors addressed important limitations in the Cochrane HPV vaccine review and stated that the Cochrane HPV vaccine review authors should make every effort to identify all trials and the trials' limitations in their conclusion. We (members of Medwatcher Japan [YAKUGAI Ombusperson], an NGO that was launched in 1997 to monitor and prevent drug-induced disasters) strongly support their conclusions.
Medwatcher Japan released an open letter (original Japanese document) ‘Critical Opinion on Cochrane Review of HPV Vaccines’ dated on June 8th, 2018 and raised some comments concerning the content of the Cochrane HPV vaccine review and the governance of the Cochrane organisation. We have submitted the open letter (English translation of the original Japanese document) to some relevant parties in charge of Cochrane on August 13th, 2018. Furthermore, we have submitted the short version which was limited to the specifics of the contents of the Cochrane HPV vaccine review via the feedback system on The Cochrane Library on August 31st, 2018. It has been confirmed that the short version was published via the Cochrane Library feedback system. We have not, however, received any responses directly from the Cochrane.
Therefore, in addition to the points raised in this article by Jørgensen et al., we would like to address the following key comments concerning the content of the Cochrane review in this response:
1. Since the effectiveness claimed in the Cochrane revi...
Show MoreThe Cochrane HPV vaccine review was incomplete and ignored important evidence of bias: Response to the Cochrane editors
Lars Jørgensen, LJ (lj@cochrane.dk), 1
Peter C. Gøtzsche, PCG (pcg@cochrane.dk), 1
Tom Jefferson, TJ (tj@cochrane.dk), 1
1Nordic Cochrane Centre, Rigshospitalet 7811, Tagensvej 21, 2100 Copenhagen, Denmark.
Summary
In a report uploaded on the Cochrane.org website on 3 September 2018 (1), Cochrane’s Editor in Chief and Deputy Editor in Chief responded to our analysis published in BMJ Evidence-Based Medicine on 27 July 2018 (2) of the Cochrane review of the HPV vaccines published on 9 May 2018 (3).
The Cochrane editors acknowledge (1) that our analysis (2) addresses the importance of the selection of data sources for reviews, and we hope that Cochrane will take the threat posed by reporting bias (4) more seriously by using clinical study reports, rather than journal publications.
The Cochrane editors claimed that we had “substantially overstated” our criticisms and they concluded that “Jørgensen et al made allegations that are not warranted and provided an inaccurate and sensationalized report of their analysis” (1).
Here we address the Cochrane editors’ findings and present our further assessment and additional findings.
In summary, we found that our analysis (2) was appropriat...
Show MoreBack in May I submitted a comment to the review by Aubyn et al, and when it was not published received an assurance from Cochrane admin that was because the site was undergoing up-date. I am dismayed but not surprised to find that it never appeared, though it is still germane:-
"It concerns me that the authors of this study failed to contest the validity of trials that did not mostly take place against genuine placebo (i.e. saline), and even use the term "placebo" repeatedly. Do they not bear a huge responsibility by not taking this issue on, as if it is all right for this kind of methodology to become a norm of modern "scientific" practice and as something which does not affect the meaning and usefulness of the safety results?
"Ordinary citizens, most of whom will have learned about good scientific practice at school, might assume that these trials were against genuine placebo and I find no clear explanation of the issue here, let alone in the "Plain Language Summary"."
I was obviously stating the issue in a simplistic manner. The public probably mostly believe that the science behind these products would have been conducted at the highest standard before marketing, but this is very far from evident - and evident at a level which they are more than capable of understanding. Yet they continue to be shielded from the truth. The authors and this journal ought to be commended for standing against the profession...
Show MoreThe Cochrane Review of the HPV vaccine states :
"In older women, vaccinated between 25 to 45 years of age, the effects of HPV vaccine on precancer are smaller, which may be due to previous exposure to HPV. The risk of precancer associated with HPV16/18 is probably reduced from 145/10,000 in unvaccinated women to 107/10,000 women following HPV vaccination (moderate certainty). The risk of any precancer is probably similar between unvaccinated and vaccinated women (343 versus 356/10,000, moderate certainty)."
Cochrane estimates a significant effect of the vaccine on HPV 16/18 associated precancer in the 25-45 age group, but that effect disappears when risk of precancer of all (HPV) types is calculated. It is a matter of simple logic that for the positive (desired) effect on type 16/18 precancer to be offset in this way, the vaccine must have a corresponding negative (undesired) effect on precancer associated with other HPV types.
The observation holds for all age groups, but is more pronounced in the older group. It is reasonable to assume that if one took, say, a 35 to 45 years old group, the total undesired result of the HPV vaccination could be statistically significant.
If my reading of these numbers is too simplistic, it would be good to get a clarification, since the quoted passage appears right in the Summary with no further comments.
In Europe, the vaccine is routinely recommended to women of all ages.
Reference...
Show MorePages