125 e-Letters

  • Plausibility bias

    The article by Gartlehner et al. [1] is interesting because it allows the homeopathic community to elaborate on potential publication bias in clinical trials of homeopathy. There are, however, several questionable elements: in the article, and in the announcement made on the BMJ web, it is concluded that there was a high proportion of trials not preregistered, but at the same time Gartlehner acknowledges in the press that over time there has been a substantial improvement in the preregistration of trials [2]; it is mentioned that homeopaths must improve, but at the same time it is implied that "homeopathy cannot work".
    On the second point, it is worth mentioning that in the article Gartlehner et al cite two trials, one by Grimes [3] and the other by Grams [4]. These essays are based on a biased selection of literature and have elementary errors. For example, Grimes says that Jacques Benveniste's famous study was published in "1987" and that Madaleine Ennis' work was negative when in fact it was positive [5]. Grimes bases his conclusions on theoretical claims (a simple calculation of Avogadro's constant) and not on experimental studies that at the time were available (e.g. [6]). Grams, on the other hand, only cites some old articles from 1992 and 1993 without mentioning more recent studies (e.g. [7]).

    1. Gartlehner G, Emprechtinger E, Hackl M, Gartlehner J, Nonninger J, et al. (2022). Assessing the magnitude...

    Show More
  • “Bad faith” in reporting on homeopathic research

    Gartlehner et al (1) concluded that the effects of homeopathic clinical trials may be overestimated due to publication bias. Such conclusions are inaccurate based on their own statement and their evaluation of the data they investigated. The authors asserted, “the difference in effect sizes between registered and unregistered studies did not reach statistical significance.” Despite this clear statement of what their data showed, the researchers instead came to a different conclusion that sought to question the integrity of research results with homeopathy.

    To their credit, these authors acknowledge that the problem of “non-publication of trial results and selective outcome reporting …. is not a phenomenon that is limited to homeopathy.” And yet, they purposefully chose to not reference any literature that evaluated this problem in publication bias from clinical trails testing conventional medicine. A simple review of the literature would find that conventional medical trials have at least the same rate of publication bias as those reported upon that tested homeopathic medicines (2), to reviews of research that showed a much higher level of publication bias when reporting on conventional medical treatments (3).

    The fact is that several media (4)(5) that have reported on this study have come to the mistaken conclusion that the results of homeopathic clinical trials are not to be trusted, and this biased conclusion stems from the Gartlehner articl...

    Show More
  • Context is everything: Conclusions of Gartlehner et al should be interpreted with caution.

    The new study by Gartlehner et al. (1) claims that the benefits of homeopathy may have been over-estimated due to high levels of reporting bias. However, as this problem is well-known to affect all areas of medical research, context is everything.

    Although the authors state that, “non-publication of trial results and selective outcome reporting …. is not a phenomenon that is limited to homeopathy”, they failed to provide adequate context for their results by making any direct comparison to other areas of clinical research. Homeopathy is arguably out-performing conventional medicine, or, at the very least, has comparable levels of reporting bias. Using representative examples of high-impact studies on reporting bias across all medical fields, when compared with the data presented by Gartlehner et al.(1) it is clear that:
    1) half of all registered clinical trials (2) in conventional medicine fail to report their results within 12 months; whereas 62% of all registered homeopathy trials reach publication, and
    2) inconsistencies in reporting of primary outcome (3) occur in 43% of conventional medical studies; whilst this happens in only 25% of published homeopathy trials.

    The potential impact of unregistered/unpublished results on estimates of treatment effects is well known (4), yet for homeopathy, according to Gartlehner et al.(1), the impact may be minimal, or nothing at all: “the difference in effect sizes between registered and unregistered stud...

    Show More
  • Reflecting trial registration status does not change the conclusions from two previous homeopathy reviews

    In their recent paper, Gartlehner et al [1] reached the headline conclusion that ‘effect estimates of meta-analyses of homeopathy trials might substantially overestimate the true treatment effect of homeopathic remedies’. Their conclusion is based on having re-analysed one of the systematic review papers’ data published by Mathie et al [2] by taking into account the possible impact of a trial’s registration status. Gartlehner et al analysed a sub-set of 19 trials of non-individualised homeopathic treatment, comparing 6 trials that were registered with 13 trials that were not registered. They observed a statistically significant difference between homeopathy and placebo only for the non-registered trials; however, the difference in effect sizes between registered and non-registered trials did not reach statistical significance.

    In conducting their re-analysis, Gartlehner et al have failed to recognise that the meta-analysis by Mathie et al [2] was primarily based on a sensitivity analysis of trials that comprised reliable evidence (effectively, low risk of bias): the effect-size estimate collectively for those 3 trials yielded a statistically non-significant result. Those 3 trials are amongst the 6 registered trials in Gartlehner’s re-analysis, and so it is no surprise that they contributed to a non-significant pooled effect size. A majority of the other 13 trials, now defined as non-registered [1], had previously been categorised by Mathie et al as high risk of bias...

    Show More
  • Response to Letter: Delays in publishing systematic review registrations in PROSPERO

    Thank you for your Letter and concerns raised about delays to registration and failure to identify similar systematic review protocols in PROSPERO. PROSPERO, which provides registration free of charge, now receives over 47,000 submissions annually including both new submissions and resubmissions (from authors whose original submission required revision owing to low quality or incomplete information). With only 1.7 FTE administrative staff funded to manage PROSPERO, demand for registration outstrips capacity of staff to process requests.

    Previous appeals to PROSPERO users to improve the quality of submission and thus reduce the need for query, feedback and resubmission and consequently demand on the register were unsuccessful. With the intention of reducing further delays, in 2020, a decision was made to automatically publish records waiting for more than 30 days from submission, providing they pass a series of automatic checks (assessing whether a record is submitted in English language and includes information about review methodology). This allowed us to focus efforts on supporting the research endeavour surrounding the global COVID-19 pandemic. We aim to publish reviews related to covid by the end of the next working day. As this has reduced the time to registration and been met with approval by the review community, we have decided to continue automatic publishing system. Future development plans involve exploring how a more detailed automatic checking process...

    Show More
  • Level of reporting bias shown to be lower in homeopathy trials than in trials on conventional medicine

    We fully agree that „non-publication of trial results and selective outcome reporting…is not a phenomenon that is limited to homeopathy.”
    Previous reviews in conventional medicine, such as the study by Kosa et al. in 2018, report „…substantive disagreement in reporting between publications and current clinical trial registry, which were associated with several study characteristics”.[1]

    In 2019 The Lancet commented on the reporting of clinical trial data for 30 European universities that sponsor the largest number of trials governed by EU clinical trials regulation: “The report shows that 778 (83%) of 940 clinical trials sponsored by these universities due to post their results on the EU Clinical trials Register (EudraCT) had not done so”.[2]

    The International Committee of Medical Journal Editors (ICMJE) announced in 2005 that “… trials that begin enrolment of patients after 1 July 2005 must register in a public trials registry at or before the onset of enrolment to be considered for publication …”.[3] EU rules took effect in 2014, which require all clinical trials registered in EudraCT to post summary results within 12 months of study completion.[2] Hence, the inclusion of studies on homeopathy published before and in 2005 by Gartlehner et al. 2022 does not seem reasonable respectively of those published before and in 2014 is debatable.
    Notwithstanding the above, precise information on sub-groups of studies was not given by Gartlehner et al. 202...

    Show More
  • Test


  • Rapid Response, BMJ Evidence-Based Medicine

    Rapid Response, BMJ Evidence-Based Medicine
    Re: Popp M, Kranke P, Meybohm P, et al. Evidence on the efficacy of ivermectin for covid-19: another story of apples and oranges. BMJ Evidence-Based Medicine Published Online First: 20 August 2021. doi: 10.1136/bmjebm-2021-111791

    Ivermectin in Covid-19

    Andrew Bryant MSc
    Population Health Sciences Institute, Newcastle University
    Baddiley-Clark Building, Richardson Road, Newcastle upon Tyne NE2 4AX, UK
    Email: andy.bryant@ncl.ac.uk
    Theresa A Lawrie MBBCh PhD
    Edmund J Fordham PhD FInstP
    EbMCsquared, a Community Interest Company
    Northgate House, Upper Borough Walls, Bath BA1 1RG, UK
    Scott Mitchell MBChB MRCS
    Emergency Department, Princess Elizabeth Hospital, Guernsey

    To the Editor
    The sole substantive critique in this Letter[1] is the description of Bryant[2] et al. (hereafter “Bryant”) as a “bowl of colourful fruit salad”[1], because of the pre-specified comparison of “ivermectin” vs “no ivermectin”. Trials with (potentially) active comparators were indeed included. Reflection should show that any bias is conservatively against ivermectin. Helpful control interventions would dilute the apparent benefit of ivermectin, relative to inactive comparators exclusively. Efficacy will be understated, not overstated, with respect to controls.
    Unsuspected active agents in ivermectin combination therapies might...

    Show More
  • RE: Is there a smoker’s paradox in COVID-19?

    There is a hypothesis that nicotine may have a protective effect in COVID-19 (1). I present two recent reports on the association between smoking and COVID-19 infection/progression, and then presented reports regarding another mechanism of the association.

    Paleiron et al. conducted a cross-sectional study to investigate the association between smoking and COVID-19 (2). The adjusted odds ratios (ORs) (95% confidence intervals [CIs]) of current smokers and subjects aged over 50 years for the risk of developing Covid-19 were 0.64 (0.49-0.84) and 2.6 (1.2-6.9), respectively. Smoking presented a protective effect on the developing COVID-19.

    Farsalinos et al. conducted a meta-analysis to examine the effects of current smoking on adverse outcomes among hospitalized COVID-19 patients (3). Pooled OR (95% CI) of current smokers against non-current smokers and against former smokers for adverse outcomes was 1.53 (1.06-2.20) and 0.42 (0.27-0.74), respectively. Smoking relates to the progression of clinical outcomes in hospitalized COVID-19 patients, although the reason of poor clinical outcomes in former smokers should be explored by further studies.

    There is another hypothesis that lithium will limit SARS-CoV2 infections. Rudd presented a hypothesis that the repurposing of low-cost inhibitors of glycogen synthase kinase-3 (GSK-3) such as lithium will limit SARS-CoV2 infections by both reducing viral replication and potentiating the immune response against the vi...

    Show More