Meta Analyses Series
Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial

https://doi.org/10.1016/j.jclinepi.2010.03.016Get rights and content

Abstract

Objective

To present some simple graphical and quantitative ways to assist interpretation and improve presentation of results from multiple-treatment meta-analysis (MTM).

Study Design and Setting

We reanalyze a published network of trials comparing various antiplatelet interventions regarding the incidence of serious vascular events using Bayesian approaches for random effects MTM, and we explore the advantages and drawbacks of various traditional and new forms of quantitative displays and graphical presentations of results.

Results

We present the results under various forms, conventionally based on the mean of the distribution of the effect sizes; based on predictions; based on ranking probabilities; and finally, based on probabilities to be within an acceptable range from a reference. We show how to obtain and present results on ranking of all treatments and how to appraise the overall ranks.

Conclusions

Bayesian methodology offers a multitude of ways to present results from MTM models, as it enables a natural and easy estimation of all measures based on probabilities, ranks, or predictions.

Introduction

For several medical questions of interest, many treatment options exist for the same indication, and these treatments may have been compared against each other and against placebo/no treatment in various clinical trials. Knowing whether one specific treatment is better than placebo or some other specific comparator is only a fragment of the big picture. Ideally, one would like to know how all the different options rank against each other and how big the differences are in effect size between all the available options. Multiple-treatment meta-analysis (MTM) offers a quantitative method of integrating all the data from all the available comparisons [1], [2], [3]. The methodology and its caveats have been extensively discussed in the statistical literature [4], [5], [6], [7], [8]. MTM applications using a variety of relevant methods appear increasingly in major medical journals on influential medical topics [9], [10], [11], [12], [13], [14]. However, the presentation and interpretation of MTM results remain challenging, especially when there are numerous competing treatments.

As MTM methodology becomes popular, one needs efficient and clinically relevant ways to present the results. Until now, results from such analyses are usually presented with display of all pairwise odds ratios or using a common reference treatment. Effect sizes and uncertainty are important for clinical purposes, and their use is recommended for interpreting results when two treatments are compared. However, for MTM, these effect sizes may be difficult to interpret and use for decision-making purposes, because a large number of such effect sizes is generated for each MTM. It would, thus, be useful to provide alternative summary displays of the data that would be more readily interpretable. Decision making often requires probabilities. Some important questions on probabilities for decision making are as follows: What is the probability that each treatment is the best? What is the probability that each treatment is among the n best options? What is the probability that a specific treatment has an effect that is within an acceptable range of the best treatment?

In some cases, it may suffice to know what the best treatment is. One can then obtain the “probability that each treatment is the best.” However, this may not convey the entire picture. For example, the most effective treatment may be unavailable (e.g., in some developing countries), difficult to apply (e.g., many physicians and medical centers may not be familiar with its use), too expensive, or associated with serious toxicity that outweighs its benefits. Moreover, a treatment may have low probability to be the best but very high probability of being second or third best. Also, two treatments may have similar probabilities to be the best but may have very different ranking thereafter. One would like to see, therefore, what the other alternatives are and how they rank. Furthermore, one would often like to know whether these alternatives still have a high probability of being within an acceptable range from the best in terms of effectiveness. Appropriate and comprehensive presentation of MTM results might informally facilitate decision making and assist in trading effectiveness against applicability, cost, or toxicity.

In this article, we aim to discuss simple graphical and quantitative ways to facilitate presentation and interpretation of results from MTM. We first describe a data set as reported and analyzed by the authors of the original publication. In the Methods section, we present in detail quantitative summaries that can be used to complement the conventional way of presentation using summary effect sizes. We show how to obtain and present results on ranking of all treatments and how to appraise the overall ranks and probabilities of being within an acceptable range. We advocate that Bayesian methodology is the natural way to fit MTM models, as it enables a natural and easy estimation of all measures based on probabilities, ranks, or predictions. In the Results section, the methodology is exemplified by reanalyzing the published network. Finally, we discuss how more informative presentation of the results could improve clarity and interpretation.

Section snippets

Data set

We selected a previously published network of interventions with readily available data that generated some discussion and commentaries on publication [15]. It is a network of antiplatelet regimens given after transient ischemic attack or stroke in preventing serious vascular events. The authors found that “the most powerful antiplatelet regimen […] is the combination of aspirin and dipyridamole,” outperforming placebo, aspirin, thienopyridines, and their combination, after summarizing data

Multiple-treatment meta-analysis model for binary outcomes

The idea underlying MTM methodology is that, for any given comparison between two treatments A and B, direct evidence (coming from studies directly comparing A to B) and indirect evidence (coming from combining studies through an intermediate comparator, e.g., A vs. C and B vs. C studies) can be synthesized into a single effect size. The idea is generalized for a network of comparisons where multiple sources of indirect evidence are synthesized achieving maximum precision for the resulting

Software

The Bayesian model presented earlier is fitted using the WinBUGS software (Medical Research Council UK; http://www.mrc-bsu.cam.ac.uk/bugs) [17]. A version of the code used can be found in https://www.bris.ac.uk/cobm/research/mpes/mixed-treatment-comparisons.html. Additional pieces of the code to estimate the presented numerical summaries, the S-plus software code used to create barplots and cumulative ranking curves and to estimate SUCRA and the data can be found in //www.dhe.med.uoi.gr/software.htm

Numerical and graphical summaries of multiple-treatment meta-analysis results based on pairwise comparisons of effects

We reanalyzed the network for serious vascular events with antiplatelet regimens. Conventional presentation of results in terms of mean odds ratios and 95% credible intervals compared with placebo are summarized in Table 2. These are the means of the random effects distribution. The median (to convey possible skewness) common heterogeneity τ and its 95% credible interval are 0.03 (0, 0.13). Additionally, we estimated all pairwise odds ratios and their respective credible intervals presented by

Discussion

Consideration of many treatments in meta-analysis is becoming increasingly popular, as statisticians, clinicians, and researchers try to consider together and understand the results from multiple trials and comparisons on the same topic. However, the methodology of MTM still remains the realm of expert statisticians, as it requires specialized software and careful consideration of the assumptions underlying a joint analysis of the treatment network. Interpretation of the results from MTM

References (24)

  • G. Lu et al.

    Assessing evidence inconsistency in mixed treatment comparisons

    J Am Stat Assoc

    (2006)
  • S.G. Baker et al.

    The transitive fallacy for randomized trials: if A bests B and B bests C in separate trials, is A better than C?

    BMC Med Res Methodol

    (2002)
  • Cited by (2512)

    View all citing articles on Scopus
    View full text