Applying the GRADE approach when evidence for an effect is summarised narratively (a meta-analysis is not available)
| GRADE domain | How to apply the GRADE domain to evidence that has been summarised narratively |
|---|---|
| Methodological limitations of the studies | Make a judgement on the risk of bias across studies for an individual outcome. A sensitivity analysis is not possible to determine if the effect changes when studies at high risk of bias are excluded. It is possible to consider the size of a study, its risk of bias and the impact it would have on the summarised effect. |
| Indirectness | Make a global judgement on how dissimilar the research evidence is to the clinical question at hand (in terms of population, interventions and outcomes across studies). |
| Imprecision | Consider the optimal information size (or the total number of events for binary outcomes and the number of participants in continuous outcomes) across all studies. A threshold of 400 or less is concerning for imprecision.15 Results may also be imprecise when the CIs of all the studies or of the largest studies include no effect and clinically meaningful benefits or harms. |
| Inconsistency | Judge inconsistency by evaluating the consistency of the direction and primarily the difference in the magnitude of effects across studies (since statistical measures of heterogeneity are not available). Widely differing estimates of the effects indicate inconsistency. |
| Likelihood of publication bias | Publication bias can be suspected when the body of evidence consists of only small positive studies or when studies are reported in trial registries but not published. Statistical evaluation of publication bias is not possible in this case. Publication bias is more likely if the search of the systematic review is not comprehensive. |
Factors that can raise certainty in evidence:
| If one of the three domains that can increase certainty in a body of evidence (typically from non-randomised studies) is noted, consider rating up the grade of certainty, particularly if it is noted in the majority of studies. |