Article Text
PDF
Abstract
Objective We aimed at providing a systematic overview of randomised trials assessing non-pharmaceutical interventions (NPIs) to prevent COVID-19.
Design Scoping review.
Methods We included all randomised trials assessing NPIs to prevent COVID-19 in any country and setting registered in ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform using the COVID-evidence platform (until 17 August 2021). We searched for corresponding publications in MEDLINE/PubMed, Google Scholar, the Living Overview of Evidence platform, and the Cochrane COVID-19 registry as well as for results posted in registries (until 14 November 2021). Descriptive statistics using numbers and percentages were used in the narrative synthesis of the results.
Results We identified 41 randomised trials. Of them, 12 were completed (29.3%) including 9 with published results. The 41 trials planned to recruit a median of 1700 participants (IQR 588–9500, range 30–35 256 399) with a median planned duration of 8 months (IQR 3–14, range 1–24). Most came from the USA (n=11, 26.8%). The trials mostly assessed protective equipment (n=11, 26.8%), COVID-19-related information and education programmes (n=9, 22.0%), access to mass events under specific safety measures (n=5, 12.2%), testing and screening strategies (n=5, 12.2%) and hygiene management (n=5, 12.2%).
Conclusions Worldwide, 41 randomised trials assessing NPIs have been initiated with published results available to inform policy decisions for only 9 of them. A long-term research agenda including behavioural, environmental, social and systems level interventions is urgently needed to guide policies and practices in the current and future public health emergencies.
- public health
- public health
- global health
- infectious diseases
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. Not applicable.
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
Summary box
What is already known about this subject?
Non-pharmaceutical interventions (NPIs) were taken worldwide to contain the COVID-19 pandemic.
The need for randomised trials on NPIs has been highlighted.
What are the new findings?
After 18 months of the pandemic and more than 4000 registered trials worldwide, we identified only 41 randomised trials assessing NPIs to prevent COVID-19.
While 12 trials (29.3%) were completed with 9 of them that published results as of November 2021, most of the trials are ongoing or not yet recruiting (n=26, 63.4%).
NPIs were diverse with approximately 1 of 4 trials assessing protective equipment.
How might it impact clinical practice in the foreseeable future?
Recommendations on best strategies to prevent COVID-19 in schools, workplaces, nursing homes and other settings cannot be made due to a lack of randomised evidence.
A long-term research agenda including behavioural, environmental, social and systems level interventions is urgently needed to guide policies and practices in the current and future public health emergencies.
Introduction
As of November 2021, 5 million deaths associated with SARS-CoV-2 infection and almost 250 million confirmed cases have been reported worldwide.1 Numerous non-pharmaceutical interventions (NPIs) were taken worldwide to contain the pandemic.2 NPIs are considered crucial to prevent infections, in particular for non-vaccinated populations and new virus variants.3 Such interventions include drastic social distancing and lockdown measures impacting billions of people around the globe.4 The need for randomised evidence on NPIs has been highlighted on numerous occasions.5 6 There is a major debate on the comparative effectiveness of public health measures such as quarantine, school and workplace closures, travel restrictions and different testing regimens to prevent COVID-19.7 8 The evidence for the benefits and harms of NPIs is mostly based on observational and modelling studies.9 Authors of high-quality evidence syntheses have concluded that the evidence for quarantine and travel restriction measures is limited facing a lack of randomised real-world evidence and that most of the studies use mathematical calculations based on diverging assumptions and model parameters.10 11
Given the extremely complex interrelationship and parallel introduction of different interventions in a situation with strong time-dependent changes, including those of the virus itself, it is difficult and almost impossible to clearly attribute outcomes to individual interventions without parallel controls.9 Furthermore, with no valid causal models capturing complex interrelationship between interventions and regional settings, environments, population, cultural, social and economic factors nor high-quality, high-granular data reflecting these factors and allowing for statistical adjustments, there is a substantial risk of confounding bias, even for comparisons with parallel controls.12–14 Only a randomised trial may provide high quality evidence on comparative effects of interventions without the need to understand complex mechanisms and measure complex data.14–16
Randomised evaluations have been used in the past to measure causal effects of a wide range of diverse and complex behavioural, social, environmental interventions and policy strategies. For example, such randomised evaluations helped to determine how health insurance affects health outcomes,17 assessed impact of school class sizes on academic performance and health (including mortality),18 and explored which human papillomavirus screening strategy for cervical cancer is favourable,19 which hygiene interventions reduce hand bacterial counts,20 how financial incentives improve physical activity21 or which infection control measures reduce spread of respiratory infections in children.22
Conversely, no randomised trials assessing any NPIs were initiated early in the COVID-19 pandemic23 and it is likely their number have remained sparse or even lacking.10 11 24 We aimed to provide a systematic overview of the major characteristics and results of randomised trials assessing NPIs to prevent COVID-19. Mapping the trial landscape helps to get an overview of the current trials worldwide, what evaluations on which NPIs are being conducted and what are the unmet needs.
Methods
A scoping review was performed to collect the current randomised trials on NPIs and to summarise their status and characteristics. We did not register this study or published a specific study protocol. We used the ‘Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews’) framework to provide a comprehensive study report, where applicable.25
Eligibility criteria
This scoping review included all registered randomised trials assessing NPIs to prevent COVID-19 in any country and setting without any language restriction. NPIs were defined as any interventions to prevent COVID-19 that did not contain pharmaceutical substances or regimens. We considered any NPIs as single measures or in combination involving, for example, information and education interventions, protective equipment, disinfection, social distancing, isolation, quarantine, testing or community-wide containment strategies. We excluded interventions based on drugs, biologicals, vaccines, herbals, traditional medicine and homoeopathy. Trials assessing NPIs without any health outcome related to SARS-CoV-2-infections were excluded (eg, interventions aiming to improve vaccination rates).
Search and selection of trials
We searched the COVID-evidence platform (www.covid-evidence.org,26) for eligible trials as of 17 August 2021. COVID-evidence is a freely available continuously updated database that contains information about worldwide planned, ongoing, and completed randomised trials on any intervention to treat or prevent SARS-CoV-2-infections. Trials registered in ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform (ICTRP) are retrieved on a weekly basis. COVID evidence uses a multimethod approach combining peer-reviewed search strategies of study registries, continuous automated extraction of search results, automated classifications combined with manual screening and data extraction (details on the methodology are published elsewhere27).
For all eligible trials identified in COVID-evidence, we searched for corresponding publications (peer-reviewed and preprints) in MEDLINE/PubMed, Google Scholar, the Living Overview of Evidence platform, and the Cochrane COVID-19 registry using trial registration numbers (until 14 November 2021). We also recorded if trial results were reported in the registries. All searches were done by one researcher (JH or PJ). The selection of trials followed a three-step approach and was done by one researcher (JH or PJ). First, records within COVID-evidence were automatically screened based on keyword searches to identify randomised trials related to COVID-19. Second, we automatically screened for trials that were clearly assessing intervention containing pharmaceutical substances using the following keyword search: (drug OR biological OR vaccine) NOT (behavioral OR mask OR school OR distancing OR non-drug OR “non drug” OR non-pharma OR “non pharma”). Finally, we manually screened the remaining trials. Unclear cases were discussed and resolved within the study team (JH, PJ and LH).
Data items and charting process
One researcher (JH) extracted data on trial status, start to end date, country, number of enrolled individuals and clusters, if applicable, population and setting, intervention and comparison, health-related COVID-19 outcome(s), and design features (randomisation unit, number of arms and outcome measurement type). Data extraction was based on trial registry information and, if available, published trial results. Extractions from registries took place in August 2021. Last update of trial and publication status was conducted on 14 November 2021. Extractions were tabulated and verified by two other researchers (PJ and LH).
Synthesis of results
Descriptive statistics using numbers and percentages were used in the narrative synthesis of the results. For calculations, we used R (V.4.1).
Patient and public involvement
We did not involve patients or members of the public in selecting the research question, designing the study, interpreting the results or writing the manuscript.
Results
We identified 41 eligible randomised trials (figure 1, table 1).
Retrieval and trial selection process. Abbreviations: ICTRP, International Clinical Trials Registry Platform; NPI, non-pharmaceutical interventions.
Characteristics of all 41 randomised trials on non-pharmaceutical interventions to prevent COVID-19 as of August 2021
As of August 2021, 12 trials were completed (29.3%) including 9 with published results, 17 ongoing (41.5%), 9 not yet recruiting (22.0%) and 3 terminated early or withdrawn (7.3%). Most came from the USA (n=11, 26.8%), UK (n=4, 9.8%) and France (n=4, 9.8%). Of note, none of the completed and terminated trials had made their results available on the registries.
The types of NPIs were diverse. Of the 41 trials, 11 trials (26.8%) assessed protective equipment such as various masks, face shields or goggles; 9 trials (22.0%) assessed diverse COVID-19-related information and education programmes (eg, text messages, newsfeeds, personalised health education, Facebook advertisements, nurse-led home visits); 5 trials (12.2%) assessed the impact of attending a musical mass event/concert under specific safety measures; 5 trials (12.2%) assessed diverse testing and screening strategies in nursing homes, schools, universities and the emergency department; 5 trials (12.2%) assessed multicomponent behavioural interventions on hygiene management in the community. The other six trials assessed access to fitness centres, school opening, special housing of homeless individuals, direct income support for individuals with COVID-19-related financial disruptions, yoga sessions for healthcare professionals and COVID-19-related symptom tracking in outpatients with cancer.
The trials planned to recruit a median of 1700 participants (IQR 588–9500, range 30–35 256 399) with a median planned duration of 8 months (IQR 3–14, range 1–24). The largest trial assessed the impact of Facebook advertisements to prevent travelling and family reunions during Thanksgiving and Christmas with >35 million participants,28 the shortest trial assessed the audience infection rate 8 days after an indoor music event.29 A typical randomised trial on NPIs randomly allocated individuals (n=23, 56.1%), rather than clusters (n=18, 43.9%), to two comparison groups (n=38, 92.7%) and assessed a COVID-19-related primary outcome (n=33, 80.5%) that was laboratory confirmed (n=30, 73.2%).
Discussion
Worldwide, 41 randomised trials assessing NPIs have been initiated with published results available to inform policy decisions for only 9 of them. The number of trials assessing NPIs to prevent COVID-19 is a tiny fraction of more than 4000 randomised trials on COVID-19 registered globally as of August 2021.27 Given the unprecedented public health impact worldwide and the wide debates about benefits and harms of NPIs, this number seems disproportionate in light of other interventions that have been the focus of much more randomised trials.8 For example, over 300 randomised trials included the highly debated hydroxychloroquine in their intervention arm30 which has been found to be associated with increased mortality in COVID-19 patients.31 The reasons for such an imbalance remain speculative and require urgent investigation. However, trials assessing NPIs face some challenges. One hurdle is the need for large sample sizes to detect an effect of public health measures in populations with low event rates. For example, it was shown that about 1000 schools in Norway would have been needed to participate in a trial assessing school closures to demonstrate a meaningful risk reduction in a situation with low incidences.32 However, this number is put into perspective when one considers that there are more than 80 000 primary schools in France, Germany, UK alone and more than 1.5 billion students were deprived of schooling worldwide in the early days of the pandemic.33–36 The fact that 117 million students are still affected by full school closure as of September 2021 underlines the continuous need to determine best strategies.37 A randomised assessment would have helped to find optimal ways for schools to safely teach millions of students globally. One other potential hurdle is costs. However, worldwide, governments have made astronomical funds available for preventive measures. For example, it has been reported that Germany made available 700 million Euro for mobile air filters.38 Such implementation could have been directly assessed through randomised roll-outs providing access to the preventive measures all the while providing robust evidence on their effectiveness.39 Another barrier may be that as vaccines become more available and accessible, the need for randomised evaluations of NPIs may be perceived to be decreasing.32 The fatal course of the disease, with the dramatic increase in deaths in the early days of the pandemic has probably also shifted the research focus from prevention to treatments for COVID-19 to the detriment of evaluating NPIs.9 30 Overall, the specific role and relevance of these factors is not clear and requires an evidence-based assessment with sound meta-research.
The list of available treatments for COVID-19 remains limited,40–42 with vaccination rates plateauing in many countries43 and increasing numbers of confirmed cases during the summer and autumn 2021,1 NPIs and their effectiveness will remain a central debate.9 Our analysis clearly indicates that randomised trials are generally feasible to assess the benefits and harms of NPIs during a pandemic situation, in a very short time using the most reliable method to provide reliable evidence for optimal health policy decision making. WHO is committed to improve the evidence base on effectiveness of public health and social measures,44 and this overview may inform further developments in this important area of evidence generation.9
This study is limited as our sample relies on accurate trial registration and reporting. We cannot exclude that some NPI randomised trials might have been missed because they were not registered, and outcomes related to SARS-CoV-2-infections were incorrectly or not reported in the registries. Searching the grey literature (eg, via web search engine or institutional searches) or contacting key informants outside academia could have retrieved eligible trials that were not registered nor published in an academic format. In particular, NPI trials with inconclusive results or not supporting their use may be at risk of not being published potentially causing a publication bias.45 Non-published results can be expected since some trials should have been completed according to their completion date given in the registry. Furthermore, we could have missed eligible trials because of our single-reviewer screening approach. However, it is unlikely that we have been unaware of pertinent major results of further NPI trials, given their substantial impact on current debates and scarcity of the evidence.46
To provide continuous guidance for policy and practice, we will continuously track randomised trials assessing NPIs and provide all information on their key design characteristics and their status, freely accessible via covid-evidence.org.26
Conclusions
Overall, during the first 18 months of the pandemic, the worldwide clinical research agenda failed to provide urgently needed evidence determining best strategies to prevent COVID-19 in schools, workplaces, nursing homes and other settings substantially affected. A long-term research agenda including behavioural, environmental, social and systems level interventions is urgently needed to guide policies and practices in the current and future public health emergencies.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information. Not applicable.
Ethics statements
Patient consent for publication
References
Footnotes
Twitter @LGHemkens
Correction notice This article has been corrected since it first published. Title has been amended to remove "as of August 2021". The open access licence has also been updated to CC BY.
Contributors PJ acts as guarantor and accepts full responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish. Concept and design: all authors. Acquisition, analysis or interpretation of data: all authors. Drafting of the manuscript: all authors. Critical revision of the manuscript for important intellectual content: all authors. Statistical analysis: JH. Obtained funding: LGH. Administrative, technical or material support: n/a. Supervision: LGH and PJ.
Funding The COVID-evidence project is supported by the Swiss National Science Foundation, project ID 31CA30_196190.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.