Overarching measures | | |
Registration of studies and analysis plans | Allows to know what studies and analysis plans were preconceived and adhered to original plans and reduces degrees of freedom for data dredging | Most observational studies are non-registered or are registered after the analyses are done; there is debate on whether retrospective designs should/could be meaningfully registered; analytical plans are rarely registered in sufficient detail |
Sharing of raw data and code | Allows independent validation of analyses and optimises the use of the data in overarching syntheses of data from multiple studies | Sharing has been limited for various reasons (privacy, consent and legal issues, as well as reluctance of primary investigators) |
Better data collection | | |
Background collection of reliable information on seroprevalence, exposures, testing, disease risk factors, risk profiles on hospital admission and use of treatments | Allows for better adjustments and exploration of effect modification | Some of this information may be biased or very difficult to collect reliably |
Blinded assessment of outcomes, for example, death causes | Allows removing some outcome misclassification biases | Blinding records requires time and resources and a committed effort |
Better designs | | |
Use of maximal/best information in properly matched studies, multivariable analyses, propensity analyses and other models | Designs that consider and hopefully address more biases are better | Observational studies are unlikely to ever eliminate all possible biases |
Performing randomised trials, whenever possible, for suitable questions (eg, use of booster doses, comparative effectiveness of different vaccination strategies) | Removes many of the biases | Reluctance to perform randomised trials when data suggest large efficacy (but this may be less of a concern for comparative effectiveness), randomised trials also have biases |
Systematic review | | |
Living reviews and meta-analyses | Provide bird’s eye view of evolving evidence | Meta-analyses have their own, long list of biases |
Better communication | | |
Use of both relative and absolute metrics of risk reduction and presentation of uncertainty | Allows better comprehension of the magnitude of the benefit | Poor ability of many/most people to understand risks and other quantitative metrics |
Avoidance of exaggeration in communicating results to the general public | Minimises misconceptions, confusion, panic (eg, from misleading claims of loss of vaccine effectiveness) or dangerous behaviour changes (eg, from misleading claims of retaining high effectiveness even with high exposures) | There is an avid market seeking immediate information on what is new on the pandemic and vaccines and sensationalism is prominent; the anti-vax movement makes confusion worse by adding extra misinformation |