I have read your response [1,2] to the paper by Jøorgensen et al. published in BMJ Evidence-Based Medicine [3] relating to the recently published Cochrane Review on HPV vaccines [4], and would like to give my feedback on this issue.
The key findings of your investigations are as follows:
1. The Cochrane Review did not miss "nearly half of the eligible trials". A small number of studies were missed due to the primary focus on peer-reviewed reports in scientific journals, but addition of these data makes little or no difference to the results of the review for the main outcomes;
2. The trials comparators were unambiguously, transparently, and accurately described;
3. The selection of outcomes for benefits was appropriate and was consistent with World Health
Organization guidance;
4. The review included published and unpublished data on serious harms, and the findings on
mortality were reported transparently and responsibly;
5. The review was compliant with Cochrane’s current conflict of interest policy;
6. Cochrane’s media coverage was cautious and balanced, but we recognize that there could be
improvements in relation to transparency where external experts are quoted;
7. The BMJ Evidence-Based Medicine article substantially overstated its criticisms.
I would like to comment on your findings 1, 2, 4 and lastly, I added comments as 8. Most observational studies neg...
I have read your response [1,2] to the paper by Jøorgensen et al. published in BMJ Evidence-Based Medicine [3] relating to the recently published Cochrane Review on HPV vaccines [4], and would like to give my feedback on this issue.
The key findings of your investigations are as follows:
1. The Cochrane Review did not miss "nearly half of the eligible trials". A small number of studies were missed due to the primary focus on peer-reviewed reports in scientific journals, but addition of these data makes little or no difference to the results of the review for the main outcomes;
2. The trials comparators were unambiguously, transparently, and accurately described;
3. The selection of outcomes for benefits was appropriate and was consistent with World Health
Organization guidance;
4. The review included published and unpublished data on serious harms, and the findings on
mortality were reported transparently and responsibly;
5. The review was compliant with Cochrane’s current conflict of interest policy;
6. Cochrane’s media coverage was cautious and balanced, but we recognize that there could be
improvements in relation to transparency where external experts are quoted;
7. The BMJ Evidence-Based Medicine article substantially overstated its criticisms.
I would like to comment on your findings 1, 2, 4 and lastly, I added comments as 8. Most observational studies neglect “healthy vaccine effect/healthy user bias”.
1. The Cochrane Review did not miss "nearly half of the eligible trials".
You explained “A small number of studies were missed due to the primary focus on peer-reviewed reports in scientific journals.” However, the abstract of the Cochrane review reports: “We searched MEDLINE ----- for reports on effects from trials. We searched trial registries and company results’ registers to identify unpublished data for mortality and serious adverse events.”
For the serious adverse events, the Cochran review described in the sensitivity analysis as follows: “We assessed the robustness of data collected for serious adverse events, all-cause mortality and pregnancy outcomes based on the source of data. The primary analysis for these outcomes included data that we considered to represent the most complete follow-up. As a sensitivity analysis we used data for these same outcomes that had only been reported in the published trial reports.”
It is evident that the Cochrane review recognized the importance of clinical study reports (CSRs) and partially gathered the unpublished data for mortality and serious adverse events.
Jørgensen et al. pointed out that there were many missed trials which Cochrane review did not included because they were not published.
As for the analysis of oseltamivir, we, the Cochrane neuraminidase inhibitor team, used all CSRs and found significantly increased psychiatric events in the 4 CSRs for prophylaxis of influenza. However, if we restricted to use peer reviewed published journals, we would not have detected the psychiatric harm even if CSRs for published trials were used. This is the most important point that Jørgensen et al. emphasized.
The review should be revised by including all CSRs identified and available. Before the revision is completed, the Cochrane review should be suspended.
2. Description of the comparators and the unavoidable toxicity of adjuvants
2-1. The safety of adjuvants is not established.
You wrote “The trials comparators were unambiguously, transparently, and accurately described”. However, the problem is not the descriptions but the real toxicity of adjuvants.
It is true that the Cochrane review by Arbyn et al. described the comparators unambiguously, transparently, and accurately. However, the problem is not the accuracy and transparency of the description of the comparators. It is that adjuvants as the comparators conceal the true harm of HPV vaccines because the safety of the vaccine adjuvant has never been established clinically [5] nor non-clinically. Instead, harm of adjuvant is rather unavoidable as indicated by laboratory tests and toxicity tests shown in the followings.
2-2. Non-clinical tests strongly suggest the harm of adjuvant.
It is revealed that true adjuvant is the DNA of the recipient which is produced by tissue damage by the administered adjuvant, such as alum adjuvant. This suggests that the bigger the damage, the stronger the stimulation of innate immunity and work as adjuvant [6,7], and autoimmune diseases [8], including those in the central nervous system [9] may be induced. GlaxoSmithKline conducted several animal toxicity tests, although they did not fulfil the standard toxicity testing method for the ordinary pharmaceutical products.
In the third toxicity test, single or repeated i.m., Cervarix, and AS04 adjuvant induced item-related local degeneration, necrosis, and regeneration of muscle fibers with hemorrhage and mild-to-moderate subacute inflammation at 4 days and 5 months after the first inoculation, unlike saline [10]. The extent and proportion of animals with these lesions were the same between Cervarix and AS04 group and immediately after the single dose but more prominent in the Cervarix group than AS04 adjuvant group after 4 repeated doses [10].
In a randomized controlled trial, saline control is appropriate as the comparator for the analysis of both efficacy and harm. For efficacy analysis, active control such as adjuvant or vaccines with/without adjuvant may not be necessarily inappropriate as the comparator. However, these active controls are definitely inappropriate for the harm analysis. The other approach should be applied for the harm analysis, if no saline control trials are available.
4. The review included published and unpublished data on serious harms, and the findings on mortality were reported transparently and responsibly;
4-1. Meta-analysis of mortality in the trials targeting mid-adults women
You wrote “The review included published and unpublished data on serious harms, and the findings on mortality were reported transparently and responsibly.” According to my meta-analysis using the extracted data from the reference peer reviewed papers in which subjects’ ages were 25 (or 26) to 45 year or older [11-14], the pooled odd ratio was 5.00 (95%CI: 1.71, 14.65), P = 0.0022 (I2 = 0%). This indicates that HPV vaccines increase mortality in woman aged 25 or older by 5 times within 4 years after the first injection.
• For the VIVIANE trial [11,12], I used 13 deaths in Cervarix group and 3 deaths in adjuvant group within 4 years, because after that period there was no difference between these groups.
• For the FUTURE III trial [13], the numbers of deaths described in the published paper were 7 in the Gardasil group and 1 in the adjuvant group as shown in its sensitivity analysis of death. However, in the Cochrane review, the numbers are 8 and 4, respectively, presumably based on the data including after 4 years from unpublished CSR.
• For the Chinese trial [14], my extraction was HPV vaccine 0 and adjuvant 0 while the Cochrane review data were 1 and 0, respectively, also in the sensitivity analysis.
Moreover, you described that after adding the mortality data from newly included trial NCT00834106 (targeted 20 to 45 years of age), RR of death increased from 1.54 (95% CI 0.73 to 3.23) to 1.65 (95% CI 0.80 to 3.38). It is highly probable that substantial difference of death risk may be reported in this added study. Please clarify the number of deaths in both groups (Data from Clinicaltrial.gov suggest that the number of deaths are estimated 2/1499 and 0/1498 for Gardasil and adjuvant group respectively).
Risk of death in women aged 25 to 45 years may be further robust by adding this trial.
Serious adverse event and mortality should be separately analysed for different age groups: namely under the age of 25 years and mid-adults (approximately 25 to 45 years old) in addition to the analysis as the whole ages.
4-2. Fluctuation of mortality and adverse events
If a vaccine does not cause harmful effects, the incidence rate of adverse events may stay at the baseline or increase only slightly as age increases. However, if the incidence rate of adverse events fluctuates substantially, then it may be the result of the harmful effect of the vaccine. Several patterns of theoretical trend of incidence rate are shown (Figure not shown).
In the pivotal RCT of Cervarix (PATRICIA), no difference in the adverse events between Cervarix group and adjuvanted HA vaccine group was observed. Hence, I calculated the overall (both groups) trend of incidence rates of adverse events, including chronic diseases (CD), autoimmune diseases (AD) and death (D) dividing 3 periods: 1: 0-1.2 year, 2: 1.2-3.4 year, 3:3.4-3.65 year). Incidence rate (/100,000 person-year) of CD, AD and D fluctuate as follows: CD: 129-55-164, AD: 25-24-88, D:22-30-135 respectively. These data show that the incidence rates or mortality rates greatly fluctuate and increase even after about 3.5 years from the first inoculation (Figure not shown).
For Gardasil studies, I calculated the incidence rates (/100,000 person-years) of various autoimmune-related adverse events (AE) among participants from both Gardasil and control (almost all received alum-adjuvant) groups (Figures not shown). Incidence rates (/100,000 person-years) of total autoimmune related adverse events are 2441 at the first period (day 1 to 6 months) and 625 at the second period (7-24 months) and that of inflammatory bowel disease are 77 and 28 respectively.
These also indicate that the incidence rates of various autoimmune diseases greatly fluctuate over time and suggest Gardasil affected these fluctuations (included studies were all phase II and III trials targeted women aged 9 to 24).
VIVIANE trial shows high mortality rate during the first 4 years and low mortality rate after 4 years in the Cervarix group, but this was not evident in the control adjuvant group (Figure not shown). These data show fluctuations are greater in the HPV vaccine group.
If the numbers of deaths for FUTURE III trial are 8 for Gardasil group and 4 for adjuvant group, 1 and 3 deaths may be observed after 4 years, respectively. Person-years at risk is not estimated, but 7 in the first 4 years and 1 thereafter indicate similar fluctuation, and less fluctuation is observed in the adjuvant group. This may be a very similar phenomenon as shown in the VIVIANE trial.
Fluctuation of serious adverse events including death should be considered in the RCTs using adjuvant as comparator. Analysis of high risk and low risk periods together may conceal true harm of HPV vaccines. Analysis should be conducted separately for the different risk periods.
4-3. Comparison of incidence rates of specific autoimmune diseases between RCT and general population of women of similar age group
The incidences rates of some autoimmune diseases examined, such as multiple sclerosis (MS), systemic lupus erythematosus (SLE), and inflammatory bowel diseases (IBD), calculated from the data in the “SBA” of Gardasil (age ranged 9–26 years, mainly 16–23 years) were all higher than those in the general female population of similar age (15 to 25 years old).
For example, the incidence rate (per 100,000 person-year) of MS in Gardasil RCTs (14.7) was about 3–15 times higher than that of general population in Italy (4.2–4.7), the United Kingdom (3.4), and France (1.0). Notably, MS incidence reported in Gardasil RCTs was even higher than the highest reported in the general population, namely, in north Sweden (8.4) and Iceland (12.5) (Figure not shown).
Incidence rates of IBD in the RCTs are also higher than that of the general female population of similar age, though the age range is a little broader (0 to 39 years old).
Annual incidence rates for typical adverse events such as specific autoimmune diseases in the RCTs should be compared with that of general population of similar age.
8. Most observational studies neglect “healthy vaccine effect/healthy user bias”
In the discussion section, the Cochrane review discussed the Adverse effects of HPV vaccines citing many observational studies.
However, most observational studies have fundamental flaws:
• Negligence of healthy vaccine effect (healthy user bias),
• Negligence of time dependent bias in the self-control case-series method and
• Confusion of incidence with prevalence.
Here I only discuss the Negligence of healthy vaccine effect (healthy user bias) which most observational studies neglected, affect the results seriously and sometimes reverse the association.
Negligence of healthy vaccinee effect (healthy user bias)
i) Theoretical basis
It is important that confounding bias is avoided in epidemiologic studies. In particular, confounding bias from “healthy vaccinee effect” always affects results in favour of an intervention, leading to overestimation of its effectiveness and safety.
People who have any diseases and/or are prone to have fever/infection tend to avoid vaccination. Autoimmune diseases often follow infection. Unless unvaccinated people who are sicklier or frailer than vaccinated people at baseline were not adequately adjusted, the results of the high incidence rate of autoimmune diseases in the vaccinated people may be offset by the disease incidence in the sickly unvaccinated people. This is “healthy-vaccinee effect,” “frailty selection bias,” or “frailty exclusion confounding bias.” [15]
Bias from “healthy-vaccinee effect” becomes more impactful as the coverage of vaccination becomes higher, namely 80% or higher [15]. Theoretically in the unvaccinated, odds ratio for having symptoms at a certain vaccine coverage (%) compared with the lowest coverage group increases as the percent coverage increases.
ii) Nagoya study
Nagoya study is a questionnaire survey on symptoms after HPV vaccination, involving about 70,000 girls (born between 2 April 1994 and 1 April 2001 (approximately aged 14 to 21 in September 2015) living in Nagoya City Japan. Of these, about 30,000 girls responded: 20,748 girls were vaccinated and 9,098 girls were unvaccinated [16].
Suzuki et al. [16] concluded “the results suggested no causal association between the HPV vaccines and their alleged harmful symptoms”.
However, according to my logistic regression analysis using the disclosed PDF data, odds ratios (ORs) of positive symptoms before vaccination were significantly (p<0.05) lower than 1.0 in 15 of 24 symptoms ranging from 0.16 (Dependent on stick or wheel chair) to 0.86 (Irregular menstrual cycle). ORs for symptoms leading to hospital visit were significantly lower than 1.0 in 7 of 24 symptoms, especially in severe symptoms, such as “Unable to walk normally” (0.22), “Dependent on stick or wheel chair” (0.21), “Sudden loss of strength” (0.28) and “Weak in the extremities” (0.30). OR for “Difficulties in calculation” is not significant (P=0.148), but point estimate of OR was very low (0.25, p=0.15).
The trends of the proportion of the frail, by coverage both in the unvaccinated and the vaccinated groups fit well to the theoretical trends expected in each group.
On the other hand, odd ratios of positive symptoms after vaccination leading to hospital visit were generally higher than 1.0 (no symptom was less than 1.0) and significantly higher than 1.0 in 11 of 24 symptoms: for example, “Weak in the extremities” (2.76, p=0.014) and “Difficult to remember Chinese characters” (8.46, p=0.047). “Dependent on stick or wheel chair leading to hospital visit” were reported in 13 girls in the vaccinated group, while none in the unvaccinated group: odds ratio = 9.61 (95%CI: 1.21- infinity, p=0.027) by the exact-like logistic regression using “R” software.
Considering the low odds ratio of health status before inoculation, each odds ratio after inoculation should be divided by the corresponding odds ratio before inoculation. Hence point odds ratio after vaccination of HPV vaccine for the symptoms leading to hospital visit are estimated as follows (those above 5.0 are shown): Dependent on stick or wheel chair: 46.7, Difficult to remember Chinese character: 24.8, Difficult to calculate: 15.5, Unable to walk normally: 11.0, Weak in the extremities: 9.2, Sudden loss of strength: 8.6 and Involuntary movement: 5.7
iii) French study: a rare study incorporating prior health status as confounder shows harm of autoimmune diseases
Among many observational studies, French study [17] is a rare study which considers the health status before vaccination, though partially. It is a retrospective cohort study utilizing database of national health system. Total 2.25 million girls aged 13 to 16 years between 2008 and 2012 were included: 0.84 million (37.3%) were exposed to HPV vaccine and 1.41 million were not exposed.
Outcome events were 14 autoimmune diseases (AD), including central nervous system (CNS) demyelinating diseases (MS: multiple sclerosis etc.), Guillain-Barres syndrome (GBS), and inflammatory bowel diseases (IBD). Hazard ratios of any of the14 ADs or of individual AD were estimated by Cox regression analysis adjusting age, year of vaccination as time dependent variable. Geometric data of residence, other diseases, level of care before inclusion, vaccination other than HPV vaccine were also adjusted. Sensitivity analysis and subgroup analysis were performed to make sure of the robustness of the results.
The adjusted factor “Level of care before inclusion” seems to be “at least one visit for medical care prior to inclusion”. Hence it is unclear how it influenced the results, and the adjustment of bias from healthy vaccinee effect may be partial.
However, a statistically significant association with HPV vaccine was shown in 2 autoimmune diseases: adjusted HR of inflammatory bowel diseases (IBD) was 1.19 (95%CI: 1.02-1.39) and that of Guillain-Barré syndrome (GBS) was 4.00 (95%CI: 1.84-8.69). The sensitivity analysis showed that the association of both autoimmune diseases were the strongest for the period from 0 to 3 months and tended to decrease subsequently without significance for IBD, but it remained significant for GBS. Risk of GBS is robust as suggested by the sensitivity and subgroup analysis. These suggest that the incidence rate of both symptoms may have fluctuated.
Moreover, using 4.7 million as the person-year (py) of unvaccinated people, incidence rates per 100,000 py are greatly different from those of the unvaccinated in the table reported by the French Agency: especially in CNS demyelinating diseases (4.7 by recalculation vs 5.8 by original report), inflammatory bowel diseases (IBD) (13.8 vs 16.9) and those having at least one autoimmune disease (AD) (63.4 vs 66.8). If our estimate of incidence rate in the unvaccinated group is correct, odds of CNS demyelinating diseases in the HPV vaccinated girls increased non-significantly (P=0.07). Odds of IBD increased more with highly significant OR up to 1.53 (95%CI: 1.33, 1.76, P<0.001), and increased odds of having at least one autoimmune disease was also significant: OR = 1.13 ((95%CI: 1.05, 1.21, P<0.001).
Moreover, incidence rate (/105 py) of CNS demyelinating diseases in the unvaccinated group (4.7 or 5.8) is very high compared with that of multiple sclerosis (0.99 [95% CI: 0.94–1.04]) in the general female population of the same age group (aged 15 to 24 years) in France between 2001 and 2007 [18]. Incidence rate (/105py) of CNS demyelinating diseases in the HPV vaccinated group in the French pharmacovigilance study [17] is far higher than that in the general population.
These suggest that unvaccinated group had included frail girls at the time of inclusion. Therefore, it should be considered that “healthy vaccinee effect” or “frailty exclusion bias” may not be completely excluded even in this French pharmacovigilance study [17].
Moreover, significantly high incidence of events after inclusion such as frequent consultation (≧4/year): Odds ratio (OR) = 2.81 (2.80,2.83) and at least one hospitalization: OR = 3.54 (3.52, 3.56) (based on person-year) should be taken into account. This is because high odds ratio needing frequent medical care and/or hospitalisation (9% of the girls per year need additional hospitalization compared with prior vaccination and unvaccinated girls) may be related to some adverse outcomes of HPV vaccination other than autoimmune diseases.
For the discussion of harm comparing the results with those from observational studies, it is essential to consider the “healthy vaccinee effect”. Unless the observational studies adjust the health status before vaccination (pseudo vaccination date for unvaccinated group), the findings from such studies should not be used as evidence of safety.
Conclusion
The review should be revised by including all identified and available data. Before the revision is completed, I consider the Cochrane review should be suspended.
In a randomized controlled trial, saline control is appropriate as the comparator for the analysis of both efficacy and harm. For efficacy analysis, active control such as adjuvant or vaccines with/without adjuvant may not be necessarily inappropriate as the comparator. However, these active controls are definitely inappropriate for the harm analysis. The other approach should be applied to the harm analysis, if no saline control trials are available.
Serious adverse event and mortality should be separately analysed for different age groups: under the age of 25 years and mid-adults (approximately 25 to 45 years old) in addition to the analysis as the whole ages.
Fluctuation of serious adverse events including death should be considered in the RCTs using adjuvant as comparator. Analysis of high risk and low risk periods together may conceal true harm of HPV vaccines. Analysis should be conducted separately for the different risk periods.
Annual incidence rate for typical adverse events such as specific autoimmune diseases in the RCTs should be compared with that of general population of similar age.
For the discussion of harm comparing the results with those from observational studies, it is essential to consider the “healthy vaccinee effect”. Unless the observational studies adjust the health status before vaccination (pseudo vaccination date for unvaccinated group), the findings from such studies should not be used as evidence of safety.
It is my sincere hope that you would consider my feedback seriously in order to further improve the quality of the Cochrane review. Should you have any questions, please feel free to contact me.
Sincerely yours,
Rokuro Hama M.D.
Chairperson, NOP Japan Institute of Pharmacovigilance
Conflict of interest
Rokuro Hama (RH) has been a member of the Cochrane neuraminidase inhibitor team headed by Tom Jefferson since January 2010. RH has met Dr. David Tovey (DT) and discussed the methods for conducting the systematic review of neuraminidase inhibitors [19] at the Oxford meeting in April 2011. RH explained DT and other members of the Cochrane neuraminidase inhibitor team how Tamiflu lowered the antibody production in the treatment trial and that ITTI population was inappropriate for the assessment of efficacy. Instead, RH proposed that ITT population should be used to assess efficacy and harm of Tamiflu. The method using ITT population was agreed by the Cochrane neuraminidase inhibitor team at the meeting in April 2011.
References
1. Cochrane’s Editor in Chief responds to BMJ EBM article criticizing HPV review https://www.cochrane.org/news/cochranes-editor-chief-responds-bmj-ebm-ar...
2. Tovey D and Soares-Weiser K. Cochrane’s Editor in Chief responds to a BMJ Evidence-Based Medicine article criticizing the Cochrane Review of HPV vaccines. https://www.cochrane.org/sites/default/files/public/uploads/cochrane_hpv...
3. Jørgensen L, Gøtzsche PC, Jefferson T. The Cochrane HPV vaccine review was incomplete and ignored important evidence of bias. BMJ Evidence-Based Medicine 2018 July 27 https://doi.org/10.1136/bmjebm-2018-111012 2.
4. Arbyn M, Xu L, Simoens C, Martin-Hirsch PPL. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Database of Systematic Reviews 2018;(5):CD009069. https://doi.org/10.1002/14651858.CD009069.pub3
5. Jefferson T, Rudin M and Pietrantonj CD, Adverse events after immunization with aluminium-containing DTP vaccines: systematic review of the evidence, Lancet Infect Dis 4, 2004, 84–90.
6. Marichal T, K. Ohata K, Bedoret D, Mesnil C, Sabatel C, Ishii KJ et al., DNA released from dying host cells mediates aluminum adjuvant activity, Nat Med 17, 2011, 996–1002.
7. Sloane JA, Blitz D, Margolin Z and Vartanian T, A clear and present danger: endogenous ligands of Toll-like receptors, Neuromolecular Med 12, 2010, 149–163.
8. Tsumiyama K, Miyazaki Y and Shiozawa S, Self-organized criticality theory of autoimmunity, PLoS One. 4, 2009, e8382.
9. Lampron A, Elali A and Rivest S, Innate immunity in the CNS: redefining the relationship between the CNS and its environment, Neuron 24 (78), 2013, 214–232.
10. GlaxoSmithKline Biologicals. Japanese Summary Basis of Approval of Cervarix. http://www.pmda.go.jp/drugs/2009/P200900052/index.html ; [accessed 18.9.4].
11. Skinner SR, Szarewski A, Romanowski B, Garland SM, Lazcano-Ponce E, Salmeron J. Efficacy, safety, and immunogenicity of the human papillomavirus 16/18 AS04- adjuvanted vaccine in women older than 25 years: 4-year interim follow-up of the phase 3, double-blind, randomised controlled VIVIANE study. Lancet 2014;384(9961): 2213–27.
12. Wheeler CM, Skinner SR, Del Rosario-Raymundo MR, Garland SM, Chatterjee A, Lazcano-Ponce E, et al. Efficacy, safety, and immunogenicity of the human papillomavirus 16/18 AS04-adjuvanted vaccine in women older than 25 years: 7-year interim follow-up of the phase 3, doubleblind, randomised controlled VIVIANE study. Lancet Infectious Diseases 2016;16(10):1154-68.
13. Castellsagué X, Munoz N, Pitisuttithum P, Ferris D, Monsonego J, Ault K. End-of-study safety, immunogenicity, and efficacy of quadrivalent HPV (types 6, 11, 16, 18) recombinant vaccine in adult women 24 to 45 years of age. British Journal of Cancer 2011;105(1):28–37.
14. Zhu FC, Li CG, Pan HX, Zhang YJ, Bi D, Tang HW. Safety and immunogenicity of human papillomavirus-16/ 18 AS04-adjuvanted vaccine in healthy Chinese females aged 15 to 45 years: a phase I trial. Chinese Journal of Cancer 2011;30(8):559–64.
15. Hama R. Symptoms after HPV vaccine: typical "frailty exclusion bias" in Nagoya City study.
MedCheckTIP 2016: 2(No5); 18-21 available at https://npojip.org/english/MedCheck/Med%20Check-TIP%2005-08-08.pdf
16. Suzuki S, Hosono A. No Association between HPV Vaccine and Reported Post-Vaccination Symptoms in Japanese Young Women: Results of the Nagoya Study. Papillomavirus Res. 2018 Feb 23. pii: S2405-8521(17)30070-8
17. French National Agency for Medicines and Health Products Safety. Vaccins anti-HPV et risque de maladies autoimmunes: étude pharmacoépidémiologique Rapport final Septembre 2015 http://ansm.sante.fr/content/download/80841/1023043/version/1/file/Ansm_...
18. Fromont A, Binquet C, Sauleau E, Fournel I, Despalins R, Moreau T et al. National estimate of multiple sclerosis incidence in France (2001-2007). Mult Scler. 2012;18:1108-15.
19. Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R et al. Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database Syst Rev. 2014;4:CD008965. http://cochranelibrary-wiley.com/doi/10.1002/14651858.CD008965.pub4/full
The number of systematic reviews being published each year is actually much larger than the already impressive number of 10,000 quoted. In 2017 over 20,000 were published; 20,661 are cited in the KSR Evidence database in early August 2018. KSR Evidence, includes all systematic reviews and meta-analyses published since 2015 and for many reviews provides a critical appraisal and a short, accessible bottom line. www.ksrevidence.com
The problems associated with NOT using a true placebo in clinical trials - in this case GSK's trial for Cervarix - were discussed way back in 2009:
FOOD AND DRUG ADMINISTRATION CENTER FOR BIOLOGICS EVALUATION AND RESEARCH
VACCINES AND RELATED BIOLOGICAL PRODUCTS ADVISORY COMMITTEE
September 9, 2009
Pg166
DR. DEBOLD: I just think in the absence of having a true placebo in the entire study, it is very, very difficult to sort out what the effects are here, what is it that we are really dealing with, what is the baseline, versus what potentially is caused by the so-called controls.
In this particular study, not only was there two different strengths of Havrix used, there was alum used. In the pooled analysis there were a number of other vaccines used as the control, which makes it scientifically very difficult to sort things out.
I searched PubMed for the retraction notices. The 3 retracted articles published by Elsevier have no retraction notices in PubMed. Retraction notices for the 2 articles in the Journal of the American Society of Hypertension were published in vol. 9, issue 10, this issue is indexed in PubMed and articles citations are in PubMed but the two retraction notices are missing. "Publishers of journals in PubMed must submit citation and abstract" (PubMed FAQ for publishers), these two notices should normally have been submitted by the publisher. As for the article published in the European Journal of Pharmaceutical Sciences, no retraction notice was found, information about the article being retracted is available in the original article but, as no retraction notice seems to be available, there is no information about the retracted status in PubMed. As PubMed is often the source used by researchers doing a study on retractions of articles (e.g. PMID: 28683764, PMID 26797347, PMID: 24928194) it is important to find this information in the database. Publishers should always publish retraction notices for retracted articles and submit the citations to PubMed when the journal is indexed in this database.
In their editorial, Brodersen et al. present two types of overdiagnosis. Their two types appear to be identical to two types of overdiagnosis we identified in research published in 2016 (Rogers WA and Mintzker Y. Getting clearer on overdiagnosis. J Eval Clin Prac 2016;22: 580-587). In that paper, we provide a detailed account of maldetection overdiagnosis and misclassification overdiagnosis, together with an analysis of the relevance of these two different types.
This matter, including our request for a correction, is fully explained in our letter published in this journal: Response to Brodersen et al’s ’Overdiagnosis: what it is and what it isn’t' (http://ebm.bmj.com/content/early/2018/03/29/bmjebm-2018-110948).
Andrea Giaccari asserts that I wrote in my editorial that sitagliptin in the TECOS trial "caused" 20% increase in the secondary outcome of congestive heart failure. That is not what I wrote. I wrote that "the study data remain consistent with" a 20% increase in this adverse outcome. While the wide confidence interval is also consistent with a reduction in heart failure risk, the primary goal of the TECOS tr...
Andrea Giaccari asserts that I wrote in my editorial that sitagliptin in the TECOS trial "caused" 20% increase in the secondary outcome of congestive heart failure. That is not what I wrote. I wrote that "the study data remain consistent with" a 20% increase in this adverse outcome. While the wide confidence interval is also consistent with a reduction in heart failure risk, the primary goal of the TECOS trial was to assess the safety of DPP4 inhibitors. Hence, the upper bounds of the confidence interval are of particular interest to practicing physicians interested in drug safety.
I read with interest the comments of Dr. Fenton. In his editorial (Evid Based Med. 2016 Jun;21(3):81-2) Dr Fenton stated that sitagliptin caused in TECOS "a 20% increase in the secondary outcome of congestive heart failure (intention-to-treat HR 1.00, 95% CI 0.82 to 1.20, p=0.98)". This is really misleading. With exactly the same numbers Dr. Fenton could state that sitagliptin caused an 18% reduction in hos...
I read with interest the comments of Dr. Fenton. In his editorial (Evid Based Med. 2016 Jun;21(3):81-2) Dr Fenton stated that sitagliptin caused in TECOS "a 20% increase in the secondary outcome of congestive heart failure (intention-to-treat HR 1.00, 95% CI 0.82 to 1.20, p=0.98)". This is really misleading. With exactly the same numbers Dr. Fenton could state that sitagliptin caused an 18% reduction in hospitalizations for heart failure. Obviously, both affirmations are false, since the real risk decrease/increase with sitagliptin is 1.00. That is, no effect at all.
I am really surprised how a journal named "Evidence Based Medicine" can publish such a "Subjective Data Interpretation".
Conflict of Interest:
Speaker's fee from Astra Zeneca, Boehringer, Sanofi
First, thank you for highlighting our paper. However, I do want to take issue with this commentary.
Systematic reviews in postop analgesia have been done now for over 20
years, and there is considerable methodological research to substantiate
what is done. The results are robust and trustworthy.
Single trials, however well done, are not trustworthy because while
they may be powered t...
First, thank you for highlighting our paper. However, I do want to take issue with this commentary.
Systematic reviews in postop analgesia have been done now for over 20
years, and there is considerable methodological research to substantiate
what is done. The results are robust and trustworthy.
Single trials, however well done, are not trustworthy because while
they may be powered to show direction of effect (drug better than placebo,
for example), they are not powered to measure the magnitude of effect
accurately. That typically needs about 10 times more data, hence the value
of systematic reviews and overview reviews (see Cochrane Database Syst
Rev. 2015 Sep 28;9:CD008659). Overview reviews are where you can get
indirect comparison of efficacy.
I think the authors are making a point about speed of onset with
caffeine, and that is fair, though it took some time to work that out. And
if so I am not sure that the study by Raisian helps. Apart from being
small (fewer than 40 per treatment group), it was a multiple-dose study in
patients who did not have initial moderate to severe pain, so it was more
of a pre-emptive study than one that could measure speed of onset.
Mazar and Ariely's recent paper [1] reinforces the concepts and
suggestions discussed in our recent publications: dishonesty is a human
universal, and there is no one-size-fits-all solution [2,3]. Education,
moral reminders and changing how researchers are rewarded are important
tools [1]. Most importantly, we need to reclaim the integrity, dedication
and code of honor Sir Austin Bradford Hill consider...
Mazar and Ariely's recent paper [1] reinforces the concepts and
suggestions discussed in our recent publications: dishonesty is a human
universal, and there is no one-size-fits-all solution [2,3]. Education,
moral reminders and changing how researchers are rewarded are important
tools [1]. Most importantly, we need to reclaim the integrity, dedication
and code of honor Sir Austin Bradford Hill considered essential to the
practice of Medicine [4].
References
1. Mazar N, Ariely D. Dishonesty in scientific research. J Clin Invest
2015; Nov 2; 125(11):3993-6.
2. Seshia SS, Makhinson M, Phillips DF, Young GB. Evidence-informed person
-centered healthcare part I: do 'cognitive biases plus' at organizational
levels influence quality of evidence?. J Eval Clin Pract 2014;
Dec;20(6):734-47.
3. Seshia SS, Makhinson M, Young GB. 'Cognitive biases plus': covert
subverters of healthcare
evidence. Evid Based Med 2015; Nov 26;. doi: 10.1136/ebmed-2015-
110302.[Epub ahead of
print].
4. Hill AB. Medical ethics and controlled trials. Br Med J 1963; Apr
20;1(5337):1043-9.
You are correct that protocols and improved technology have led to
reductions in radiation exposure from CT scanning at some hospitals. I
would suggest though that the resultant reduction in the risk of fatal
cancer due to imaging does not affect the conclsion of the paper. If a
laparotomy on a healthy young patient carries no risk of death and CT
scanning imposes a risk of death the decision to perfor...
You are correct that protocols and improved technology have led to
reductions in radiation exposure from CT scanning at some hospitals. I
would suggest though that the resultant reduction in the risk of fatal
cancer due to imaging does not affect the conclsion of the paper. If a
laparotomy on a healthy young patient carries no risk of death and CT
scanning imposes a risk of death the decision to perform a CT scan on a
young healthy person before proceeding to the operating room poses an
ethical dilemma for the ordering physican.
Dear Dr. David Tovey,
I have read your response [1,2] to the paper by Jøorgensen et al. published in BMJ Evidence-Based Medicine [3] relating to the recently published Cochrane Review on HPV vaccines [4], and would like to give my feedback on this issue.
The key findings of your investigations are as follows:
1. The Cochrane Review did not miss "nearly half of the eligible trials". A small number of studies were missed due to the primary focus on peer-reviewed reports in scientific journals, but addition of these data makes little or no difference to the results of the review for the main outcomes;
2. The trials comparators were unambiguously, transparently, and accurately described;
3. The selection of outcomes for benefits was appropriate and was consistent with World Health
Organization guidance;
4. The review included published and unpublished data on serious harms, and the findings on
mortality were reported transparently and responsibly;
5. The review was compliant with Cochrane’s current conflict of interest policy;
6. Cochrane’s media coverage was cautious and balanced, but we recognize that there could be
improvements in relation to transparency where external experts are quoted;
7. The BMJ Evidence-Based Medicine article substantially overstated its criticisms.
I would like to comment on your findings 1, 2, 4 and lastly, I added comments as 8. Most observational studies neg...
Show MoreThe number of systematic reviews being published each year is actually much larger than the already impressive number of 10,000 quoted. In 2017 over 20,000 were published; 20,661 are cited in the KSR Evidence database in early August 2018. KSR Evidence, includes all systematic reviews and meta-analyses published since 2015 and for many reviews provides a critical appraisal and a short, accessible bottom line. www.ksrevidence.com
The problems associated with NOT using a true placebo in clinical trials - in this case GSK's trial for Cervarix - were discussed way back in 2009:
FOOD AND DRUG ADMINISTRATION CENTER FOR BIOLOGICS EVALUATION AND RESEARCH
VACCINES AND RELATED BIOLOGICAL PRODUCTS ADVISORY COMMITTEE
September 9, 2009
Pg166
DR. DEBOLD: I just think in the absence of having a true placebo in the entire study, it is very, very difficult to sort out what the effects are here, what is it that we are really dealing with, what is the baseline, versus what potentially is caused by the so-called controls.
In this particular study, not only was there two different strengths of Havrix used, there was alum used. In the pooled analysis there were a number of other vaccines used as the control, which makes it scientifically very difficult to sort things out.
I searched PubMed for the retraction notices. The 3 retracted articles published by Elsevier have no retraction notices in PubMed. Retraction notices for the 2 articles in the Journal of the American Society of Hypertension were published in vol. 9, issue 10, this issue is indexed in PubMed and articles citations are in PubMed but the two retraction notices are missing. "Publishers of journals in PubMed must submit citation and abstract" (PubMed FAQ for publishers), these two notices should normally have been submitted by the publisher. As for the article published in the European Journal of Pharmaceutical Sciences, no retraction notice was found, information about the article being retracted is available in the original article but, as no retraction notice seems to be available, there is no information about the retracted status in PubMed. As PubMed is often the source used by researchers doing a study on retractions of articles (e.g. PMID: 28683764, PMID 26797347, PMID: 24928194) it is important to find this information in the database. Publishers should always publish retraction notices for retracted articles and submit the citations to PubMed when the journal is indexed in this database.
In their editorial, Brodersen et al. present two types of overdiagnosis. Their two types appear to be identical to two types of overdiagnosis we identified in research published in 2016 (Rogers WA and Mintzker Y. Getting clearer on overdiagnosis. J Eval Clin Prac 2016;22: 580-587). In that paper, we provide a detailed account of maldetection overdiagnosis and misclassification overdiagnosis, together with an analysis of the relevance of these two different types.
This matter, including our request for a correction, is fully explained in our letter published in this journal: Response to Brodersen et al’s ’Overdiagnosis: what it is and what it isn’t' (http://ebm.bmj.com/content/early/2018/03/29/bmjebm-2018-110948).
Dear Editor,
Andrea Giaccari asserts that I wrote in my editorial that sitagliptin in the TECOS trial "caused" 20% increase in the secondary outcome of congestive heart failure. That is not what I wrote. I wrote that "the study data remain consistent with" a 20% increase in this adverse outcome. While the wide confidence interval is also consistent with a reduction in heart failure risk, the primary goal of the TECOS tr...
Dear Editor,
I read with interest the comments of Dr. Fenton. In his editorial (Evid Based Med. 2016 Jun;21(3):81-2) Dr Fenton stated that sitagliptin caused in TECOS "a 20% increase in the secondary outcome of congestive heart failure (intention-to-treat HR 1.00, 95% CI 0.82 to 1.20, p=0.98)". This is really misleading. With exactly the same numbers Dr. Fenton could state that sitagliptin caused an 18% reduction in hos...
Dear Editor,
First, thank you for highlighting our paper. However, I do want to take issue with this commentary.
Systematic reviews in postop analgesia have been done now for over 20 years, and there is considerable methodological research to substantiate what is done. The results are robust and trustworthy.
Single trials, however well done, are not trustworthy because while they may be powered t...
Dear Editor,
Mazar and Ariely's recent paper [1] reinforces the concepts and suggestions discussed in our recent publications: dishonesty is a human universal, and there is no one-size-fits-all solution [2,3]. Education, moral reminders and changing how researchers are rewarded are important tools [1]. Most importantly, we need to reclaim the integrity, dedication and code of honor Sir Austin Bradford Hill consider...
Dear Editor,
You are correct that protocols and improved technology have led to reductions in radiation exposure from CT scanning at some hospitals. I would suggest though that the resultant reduction in the risk of fatal cancer due to imaging does not affect the conclsion of the paper. If a laparotomy on a healthy young patient carries no risk of death and CT scanning imposes a risk of death the decision to perfor...
Pages