Article Text

This article has a correction. Please see:

PDF

Numbers needed to treat derived from meta-analysis: a word of caution
  1. Arthur Marx, MD, MPH,
  2. Heiner C Bucher, MD, MPH
  1. Basel Institute for Clinical Epidemiology, Basel, Switzerland

    Statistics from Altmetric.com

    In clinical trials, treatment effects from binary outcomes, such as “alive” or “dead”, can be presented in various ways (eg, relative risk reduction [RRR] and absolute risk reduction [ARR]).1–2 (See glossary for definitions and calculations). Alternatively, the number needed to treat (NNT) is an expression of the number of patients who need to be treated to prevent one additional adverse event.2–4 Mathematically, the NNT equals the reciprocal of the ARR. Many journals now report results from clinical trials using the NNT, along with 95% confidence intervals (CIs).5

    Since its introduction,3 a debate has ensued whether reporting NNTs from meta-analyses is misleading.4,6,7 ACP Journal Club and Evidence-Based Medicine have devoted attention to this debate, and methodological questions have been raised in various review discussions and commentaries.8–9 The objective of this editorial is to provide Evidence-Based Medicine readers with caveats and suggestions when applying NNTs from a meta-analysis to an individual patient. We highlight 2 problems. Firstly, NNTs from a meta-analysis are subject to variation in risk differences among the studies included in the meta-analysis as well as in baseline risks. Secondly, applying NNTs to an individual requires adjustment for their baseline risk.

    NNTs derived from meta-analyses are affected by variations in risk differences among the studies, as well as baseline event rates in control groups of randomised controlled trials.1,10 Summary estimates of NNTs assume constant risk differences between trials, a problematic assumption because of inevitable variation in baseline event rates between trials, differences in outcomes considered, effects of secular trends on disease risk, and differences in clinical setting as well as duration of follow up (ie, time horizon).2,10 In primary prevention of chronic disease, such as cardiovascular disease, the effect of time trends will become noticeable.

    Several approaches have been introduced to derive or present NNTs from summary risk estimates of meta-analysis. Evidence-Based Medicine readers have become familiar with standard summary presentations of meta-analytic research that provides NNTs based on pooled risk differences. However, in meta-analysis, the assumption underlying the pooling of any effect estimate is that measures are reasonably homogeneous across trials and that any variation can be attributed to random chance. In the case of the NNT, this means that absolute risk differences should be constant across studies. Growing empirical evidence suggests that relative risks and odds ratios in most instances provide more homogeneous estimates than absolute risk differences.1 For this reason, clinicians should use NNTs that are derived from pooled estimates of relative risks instead of absolute risk differences.

    Application of NNTs from meta-analyses to individual patients also requires attention to baseline risks, subject to the following conditions: knowledge of the baseline risk of the study patients and the estimated risk of the individual patient to whom the NNT will be applied, and information on the time horizon of the study (eg, the median follow up time).

    We illustrate these points using the meta-analysis of randomised controlled trials investigating the effect of n-3 polyunsaturated fatty acids on clinical endpoints in patients with coronary heart disease as an example.11–12 The meta-analysis found a 19% (CI 10% to 27%) RRR of overall mortality favouring the use of n-3 polyunsaturated fatty acids. In the Evidence-Based Medicine structured summary of this meta-analysis, the NNT derived from a pooled risk difference, and based on an average baseline risk of 9.5%, is 73 (CI 49 to 147).11 However, this estimate does not reflect the high variability of baseline risk of the included trials. As discussed by the authors of the original meta-analysis,12 the baseline risk of included trials ranged from 2–22%. Based on the pooled relative risk estimate, the corresponding NNT for an average follow up of 18 months is 263 (CI 185 to 500) for a baseline risk of 2% and 24 (CI 17 to 45) for a baseline risk of 22%. This example illustrates that if risk differences and baseline event rates in meta-analysis vary considerably across trials, the NNT derived from pooled absolute risk differences is not very informative and may be misleading if used to assess the benefit of a given intervention for an individual patient. Only if the baseline risk does not vary considerably across trials may the NNT derived from pooled risk differences be applied to determine the potential benefit for an individual patient.

    Cook and Sackett4 and Straus13 have proposed a simple method to apply estimates of RRRs from single trials or a meta-analysis to individual baseline risk. They suggest comparing the patient’s baseline risk with that of a typical patient in the published trial. If the baseline risk of an individual patient is a factor f compared with the baseline risk of a typical study patient and the relative risk is constant, the ARR for the patient is scaled according to the same factor f. The estimated NNT corresponding to patients at the revised baseline risk is therefore simply the study NNT multiplied by f. Thus, in our example, if a high risk patient is judged to have baseline risk 11 times higher than that of a low risk patient, then f = 11 and the corresponding NNT is 24 × 11 = 264. Confidence intervals can also easily be obtained by multiplying the limits of the corresponding confidence interval from the original study with the factor f. However, as we have mentioned, this method should not be used with NNTs that are derived from pooled estimates of absolute risk differences. For example, if a clinician used an NNT of 73 based on absolute risk differences and a weighted control event rate of 9.5% to assess the benefit of n-3 polyunsaturated fatty acids for a patient with a baseline risk of 2%, the resulting NNT would be 347 (compared with 263 based on relative risk estimates). Therefore, clinicians should be wary when interpreting NNTs derived from pooled risk differences of a meta-analysis if baseline risks or absolute risk differences vary across trials. In this case, it makes no sense to use an NNT for the “average” single trial patient or the “average” meta-analysis patient.

    Acta Obstet Gynecol ScandArch Pediatr Adolesc MedGutJ Vasc Surg
    Age AgeingArch SurgHeartLancet
    Am J CardiolArthritis RheumHypertensionMed Care
    Am J MedBJOGJAMAMed J Aust
    Am J Obstet GynecolBMJJ Am Coll CardiolN Engl J Med
    Am J PsychiatryBr J Gen PractJ Am Coll SurgNeurology
    Am J Public HealthBr J PsychiatryJ Am Geriatr SocObstet Gynecol
    Am J Respir Crit Care MedBr J SurgJ Clin EpidemiolPain
    Ann Emerg MedCMAJJ Fam PractPediatrics
    Ann Intern MedChestJ Gen Intern MedRheumatology
    Ann SurgCirculationJ Infect DisSpine
    Arch Dis ChildCochrane LibraryJ Intern MedStroke
    Arch Gen PsychiatryCrit Care MedJ Neurol Neurosurg PsychiatrySurgery
    Arch Intern MedDiabetes CareJ PediatrThorax
    Arch NeurolGastroenterology

    Journals reviewed for this issue*

    Acknowledgments

    We thank Matthias Egger and Brian Haynes for their critical review and contributions to this paper.

    References

    View Abstract

    Request permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Linked Articles

    • Letters
      Arthur Marx Heiner Bucher