Objectives Network meta-analysis (NMA) provides plots of interventions networks per outcome. Such plots also demonstrate all possible pairwise comparisons in a network. This paper discusses the gap between evidence expected from a given network and observed evidence from trials by use of a simple formula.

Method Published network meta-analyses in BMJ, JAMA, Lancet and New England Journal of Medicine were collected (2003-2019). The potential number of comparisons per network plot was calculated using (n*(n-1))/2¹ and the number of direct comparisons from trials was deducted to obtain the ‘evidence gap’ per network plot. We also compared evidence gap in the networks with low or high number of intervention nodes. All steps were conducted by GA, repeated by FS and supervised by CEA.

Results We excluded four NMAs because of mixing study designs but were able to include 41 NMAs of randomised controlled trials (RCTs). We identified 77 network plots from NMAs. The plots consisted of between 2 and 31 intervention nodes. Only four plots were complete and based on only direct comparisons. The evidence gap was between 0.06 and 0.88 for the remainder - between 10% and 90% of the comparisons in the network have not yet been reported in RCTs. There is a positive and strong correlation between the number of interventions and the number of indirect comparisons (R² = 0.894) highlighting the considerable gap in the certainty of body of evidence as presented in NMAs. The evidence gap is filled statistically without real-world evidence from trials and the results of NMA are particularly problematic when there are many interventions in the plot.

Conclusions Researchers who undertake NMAs should report network plots and a list of missing comparisons from the trials. They should also report the evidence gap to emphasise the proportion of the NMA which is based on data derived from real world experiments and the proportion from statistic-based inference. The findings of our research call for an update of the PRISMA for Network Meta-Analyses reporting guideline.

Reference

1. Shokraneh F, Adams CE. A simple formula for enumerating comparisons in trials and network meta-analysis. F1000Research. 2019 Jan 9; 8:38. DOI 10.12688/f1000research.17352.1