Is meta-analysis the platinum standard of evidence?

Stud Hist Philos Biol Biomed Sci. 2011 Dec;42(4):497-507. doi: 10.1016/j.shpsc.2011.07.003.

Abstract

An astonishing volume and diversity of evidence is available for many hypotheses in the biomedical and social sciences. Some of this evidence-usually from randomized controlled trials (RCTs)-is amalgamated by meta-analysis. Despite the ongoing debate regarding whether or not RCTs are the 'gold-standard' of evidence, it is usually meta-analysis which is considered the best source of evidence: meta-analysis is thought by many to be the platinum standard of evidence. However, I argue that meta-analysis falls far short of that standard. Different meta-analyses of the same evidence can reach contradictory conclusions. Meta-analysis fails to provide objective grounds for intersubjective assessments of hypotheses because numerous decisions must be made when performing a meta-analysis which allow wide latitude for subjective idiosyncrasies to influence its outcome. I end by suggesting that an older tradition of evidence in medicine-the plurality of reasoning strategies appealed to by the epidemiologist Sir Bradford Hill-is a superior strategy for assessing a large volume and diversity of evidence.

MeSH terms

  • Biomedical Research / standards
  • Decision Making
  • Evidence-Based Practice
  • Humans
  • Meta-Analysis as Topic*
  • Randomized Controlled Trials as Topic
  • Research / standards*
  • Sociology / methods