120 e-Letters

  • The main claim in this paper is not even wrong

    Dear Prof. Franco,

    I am writing to request further clarification on the paper “Likelihood ratio interpretation of the relative risk”. The “key messages” section of this paper states that the study adds the following to the literature:

    ⇒ It is demonstrated that the conventional interpretation of the relative risk is in conflict with Bayes’ theorem.
    ⇒ The interpretation of the relative risk as a likelihood ratio connecting prior (unconditional) intervention risk to outcome conditional intervention risk is required to avoid conflict with Bayes’ Theorem

    I will refer to the first bullet point as “Doi’s Conjecture”. Doi’s Conjecture is also stated in the second section of the main text, where it is claimed that “the usual interpretation (33% increase in the +ve outcome under treatment) contravenes Bayes Theorem”.

    No attempt is made within the text to prove Doi’s Conjecture. But perhaps more worryingly, no attempt is made to define the term “interpretation”, a term which is not defined in standard probability theory. The meaning of Doi’s Conjecture is therefore at best ambiguous. Moreover, the manuscript relies substantially on claims about how effect measures are “perceived”, another term which is defined neither in probability theory not in the manuscript.

    The relative risk is defined as the risk of the outcome under treatment, divided by the risk of the outcome under the control condition; that is, as a ratio of two probabilities. Thi...

    Show More
  • Equipoise, or not, when comparing filtering face piece respirators to surgical (medical) masks, and the futility of using a randomized control trial (RCT) for comparison. The need for evidence-based medicine+ - a paradigm shift.

    Airborne transmission and inhalation, of SARS-CoV-2 is recognized by international public health agencies (Addleman et al. 2021) from both short- and long-range aerosol transmission (Tang et al. 2021).
    When comparing filtering face piece (FFP) respirators, with a surgical (medical) masks, for protection against an inhalable (ISO 7708) airborne virus such as SARS-CoV-2, “EQUIPOISE” a central tenet for conducting a randomised control trial (RCT), does not exist. The futility of using a RCT is analogous to carrying out a study in a construction site for hard hats or seat belts in a passenger vehicle. FFP respirators used in Canada are selected and used in accordance with CAN/CSA-Z94.4-18 (Selection, use, and care of respirators). Specifications are provided in CSA Z94.4.1:21 (Performance of filtering respirators).
    At least two international studies using RCT have been initiated since the start of the COVID-19 pandemic.
    a) United Kingdom: The impact of different grades of respiratory protective equipment on sickness absence due to respiratory infections including SARS-CoV-2 for healthcare workers (The funding committee did not recommend funding).
    b) Canada, multi-centre: where nurses are randomized to either use of a medical mask or to a fit-tested N95 respirator when providing care for patients with febrile respiratory illness.
    The findings from a poorly designed RCT may also be used as improper and biased rationale to downgrade respirator (masks...

    Show More
  • EBM+ or a Trojan horse of 'science-experts'

    The clinical evidence we produce which impacts the care we deliver to patients must have the highest standards. Unlike biomedical evidence, the evidence behind our treatments, risk factors and diagnostic tests should be relevant to the patients and clinicians who practice at the coal-face and real-world of General Practice and our hospitals.
    We may have had 250,000 'peer-reviewed' articles relating to COVID-19 (whatever credibility that affords), but allowing biomedical assumptions to creep into guidelines and direct patient care, is taking a step backwards in evidence-based practice. COVID saw a multitude of 'experts' give their opinion freely in the media and on social media platforms, based on biomedical ‘evidence’, on opinion, through the platforming of academic-status and on dubious case-studies. This was self-evidently un-scientific, and arguably has set science back. Yet this article seems to advance the notion that this evidence should impact patient care.
    So much has to be done to enable EBM to flourish- yet this article will likely set EBM back. The evidence underpinning our treatments should be quantifiable in absolute terms, and the uncertainties and conflicts of interests (COIs) acknowledged. The evidence behind our risk factors characterised in absolute terms, within the context of confounders and biases from observational studies. Our diagnostic tests should be understood within the context of odds and post-test probability. O...

    Show More
  • EBM orthodoxy has had a quasi-religious feel to it.

    This refreshing article does much to help move forward from the self-imposed strictures of the EBM orthodoxy. I hope EBM+ grows as an idea as, currently, EBM feels, sometimes, quasi-religious in the fervour to reject “the wrong sort” of evidence. It leaves clinicians in fields like addiction - where RCTs are very difficult to do unless so simple in their structure that the results are clinically difficult to use or, worse, irrelevant - stuck. Meta-analyses of clinically irrelevant studies are still clinically irrelevant. Data is hard to come by that is not EBM-perfect, so why bother doing the clinical research? Hopefully this suggestion will improve that disconnect.

  • In Memoriam Ingeborg Griffioen

    To our great sadness on Wednesday, April 13th 2022, Ingeborg Griffioen, author of “Innovating in healthcare: perspective from a dual role” passed away, at the age of 50. She departed with acceptance of the inevitable and in connection to those she loved and who loved her.

    Ingeborg was founder and owner of Panton design studio, specialized in healthcare. In 2016 she started a PhD research on the use of service design to support shared decision making. Less than one year later, her husband was diagnosed with pancreatic cancer. Ingeborg incorporated their experiences with his care trajectory in her research, which led to the development of MetroMapping, a method to support shared decision making (www.metromapping.org/en). During this development process, Ingeborg herself was diagnosed with breast cancer.

    It was one of Ingeborg’s dreams that MetroMapping be further developed and implemented at a large scale. Even during her chemotherapy she contributed to the 4D PICTURE project proposal, which focuses on adapting, evaluating and implementing MetroMapping in hospitals throughout Europe. Early 2022, we received the news that Ingeborg’s dream will come true, as 4D PICTURE was selected for funding by Horizon Europe. Ingeborg wrote a beautiful testimonial for 4D PICTURE:

    “As a designer I have worked for 25 years in healthcare settings and as a researcher I studied treatment decision-making. I know the importance of...

    Show More
  • Plausibility bias

    The article by Gartlehner et al. [1] is interesting because it allows the homeopathic community to elaborate on potential publication bias in clinical trials of homeopathy. There are, however, several questionable elements: in the article, and in the announcement made on the BMJ web, it is concluded that there was a high proportion of trials not preregistered, but at the same time Gartlehner acknowledges in the press that over time there has been a substantial improvement in the preregistration of trials [2]; it is mentioned that homeopaths must improve, but at the same time it is implied that "homeopathy cannot work".
    On the second point, it is worth mentioning that in the article Gartlehner et al cite two trials, one by Grimes [3] and the other by Grams [4]. These essays are based on a biased selection of literature and have elementary errors. For example, Grimes says that Jacques Benveniste's famous study was published in "1987" and that Madaleine Ennis' work was negative when in fact it was positive [5]. Grimes bases his conclusions on theoretical claims (a simple calculation of Avogadro's constant) and not on experimental studies that at the time were available (e.g. [6]). Grams, on the other hand, only cites some old articles from 1992 and 1993 without mentioning more recent studies (e.g. [7]).

    1. Gartlehner G, Emprechtinger E, Hackl M, Gartlehner J, Nonninger J, et al. (2022). Assessing the magnitude...

    Show More
  • “Bad faith” in reporting on homeopathic research

    Gartlehner et al (1) concluded that the effects of homeopathic clinical trials may be overestimated due to publication bias. Such conclusions are inaccurate based on their own statement and their evaluation of the data they investigated. The authors asserted, “the difference in effect sizes between registered and unregistered studies did not reach statistical significance.” Despite this clear statement of what their data showed, the researchers instead came to a different conclusion that sought to question the integrity of research results with homeopathy.

    To their credit, these authors acknowledge that the problem of “non-publication of trial results and selective outcome reporting …. is not a phenomenon that is limited to homeopathy.” And yet, they purposefully chose to not reference any literature that evaluated this problem in publication bias from clinical trails testing conventional medicine. A simple review of the literature would find that conventional medical trials have at least the same rate of publication bias as those reported upon that tested homeopathic medicines (2), to reviews of research that showed a much higher level of publication bias when reporting on conventional medical treatments (3).

    The fact is that several media (4)(5) that have reported on this study have come to the mistaken conclusion that the results of homeopathic clinical trials are not to be trusted, and this biased conclusion stems from the Gartlehner articl...

    Show More
  • Context is everything: Conclusions of Gartlehner et al should be interpreted with caution.

    The new study by Gartlehner et al. (1) claims that the benefits of homeopathy may have been over-estimated due to high levels of reporting bias. However, as this problem is well-known to affect all areas of medical research, context is everything.

    Although the authors state that, “non-publication of trial results and selective outcome reporting …. is not a phenomenon that is limited to homeopathy”, they failed to provide adequate context for their results by making any direct comparison to other areas of clinical research. Homeopathy is arguably out-performing conventional medicine, or, at the very least, has comparable levels of reporting bias. Using representative examples of high-impact studies on reporting bias across all medical fields, when compared with the data presented by Gartlehner et al.(1) it is clear that:
    1) half of all registered clinical trials (2) in conventional medicine fail to report their results within 12 months; whereas 62% of all registered homeopathy trials reach publication, and
    2) inconsistencies in reporting of primary outcome (3) occur in 43% of conventional medical studies; whilst this happens in only 25% of published homeopathy trials.

    The potential impact of unregistered/unpublished results on estimates of treatment effects is well known (4), yet for homeopathy, according to Gartlehner et al.(1), the impact may be minimal, or nothing at all: “the difference in effect sizes between registered and unregistered stud...

    Show More
  • Reflecting trial registration status does not change the conclusions from two previous homeopathy reviews

    In their recent paper, Gartlehner et al [1] reached the headline conclusion that ‘effect estimates of meta-analyses of homeopathy trials might substantially overestimate the true treatment effect of homeopathic remedies’. Their conclusion is based on having re-analysed one of the systematic review papers’ data published by Mathie et al [2] by taking into account the possible impact of a trial’s registration status. Gartlehner et al analysed a sub-set of 19 trials of non-individualised homeopathic treatment, comparing 6 trials that were registered with 13 trials that were not registered. They observed a statistically significant difference between homeopathy and placebo only for the non-registered trials; however, the difference in effect sizes between registered and non-registered trials did not reach statistical significance.

    In conducting their re-analysis, Gartlehner et al have failed to recognise that the meta-analysis by Mathie et al [2] was primarily based on a sensitivity analysis of trials that comprised reliable evidence (effectively, low risk of bias): the effect-size estimate collectively for those 3 trials yielded a statistically non-significant result. Those 3 trials are amongst the 6 registered trials in Gartlehner’s re-analysis, and so it is no surprise that they contributed to a non-significant pooled effect size. A majority of the other 13 trials, now defined as non-registered [1], had previously been categorised by Mathie et al as high risk of bias...

    Show More
  • Response to Letter: Delays in publishing systematic review registrations in PROSPERO

    Thank you for your Letter and concerns raised about delays to registration and failure to identify similar systematic review protocols in PROSPERO. PROSPERO, which provides registration free of charge, now receives over 47,000 submissions annually including both new submissions and resubmissions (from authors whose original submission required revision owing to low quality or incomplete information). With only 1.7 FTE administrative staff funded to manage PROSPERO, demand for registration outstrips capacity of staff to process requests.

    Previous appeals to PROSPERO users to improve the quality of submission and thus reduce the need for query, feedback and resubmission and consequently demand on the register were unsuccessful. With the intention of reducing further delays, in 2020, a decision was made to automatically publish records waiting for more than 30 days from submission, providing they pass a series of automatic checks (assessing whether a record is submitted in English language and includes information about review methodology). This allowed us to focus efforts on supporting the research endeavour surrounding the global COVID-19 pandemic. We aim to publish reviews related to covid by the end of the next working day. As this has reduced the time to registration and been met with approval by the review community, we have decided to continue automatic publishing system. Future development plans involve exploring how a more detailed automatic checking process...

    Show More