A pragmatic strategy for the review of clinical evidence

Background: Systematic reviews (SR) of clinical evidence are rightfully considered the basis for developing recommendations to support decisions in current practice. To avoid bias, SRs are expected to be systematic in their research strategy and are exhaustive. The drawback of the latter criteria relies in the substantial work needed to conduct and keep SRs updated. The objective of this paper is to compare a research strategy based on the review of a selected number of core journals, which we consider a 'pragmatic review' (PR), with that derived by an SR in estimating the efficacy of treatments.

Methods: Five clinical areas were considered for the comparison between the two strategies: chronic obstructive pulmonary disease, dermatology, heart failure, renal diseases and stroke. We extracted a systematic sample from all the Cochrane SRs pertaining to each area and were published before April 2010. Two groups of journals were considered in the PR: six general journals that commonly published research for the five clinical areas, and five specialist journals with the highest impact factor in each area. To assess the agreement in the findings of SRs and PRs, we considered both the direction of the estimates and P-values.

Results: A sample of 27 SRs included 171 overall analyses and 259 subgroup analyses related to primary outcomes. The PR captured one or more clinical trials in 24 of the 27 SRs (89%), and 118 of the 171 overall analyses (69%) were replicated. The PR supported the recommendations to use (or not) the study treatment in 11 of the 13 SRs (85%), which ended with a clinical recommendation.

Conclusions: We verified in a sample of SRs that the conclusion of a research strategy based on a pre-defined set of general and specialist medical journals is able to replicate almost all the clinical recommendations of a formal SR.