Perhaps the greatest threat to the validity of a meta-analysis is the possibility of publication bias, where studies that are interesting or statistically significant are more likely to be published than those with less encouraging results. In particular, there is the concern that this bias might be 'one-sided', where studies indicating that the treatment is beneficial have a greater probability of publication. The impact that this type of bias has on the estimate of treatment effect has received a great deal of attention but this also has implications for estimates of between-study variance. Using step functions to model the bias it can be demonstrated that it is impossible to make generalizations concerning how we should revise estimates of between-study variance when presented with the possibility of publication bias. To determine this, assumptions must be made concerning the form that the bias takes, which is unknown in practice.
Copyright 2005 John Wiley & Sons, Ltd.