Article Text

PDF

Redefining rapid reviews: a flexible framework for restricted systematic reviews
  1. Annette Plüddemann,
  2. Jeffrey K Aronson,
  3. Igho Onakpoya,
  4. Carl Heneghan,
  5. Kamal R Mahtani
  1. Nuffield Department of Primary Care Health Sciences, Centre for Evidence-Based Medicine, University of Oxford, Oxford, UK
  1. Correspondence to Dr Annette Plüddemann, Nuffield Department of Primary Care Health Sciences, Centre for Evidence-Based Medicine, University of Oxford, Oxford OX2 6GG, UK; annette.pluddemann{at}phc.ox.ac.uk

Statistics from Altmetric.com

Having suggested that so-called ‘rapid’ reviews have been misnamed and should instead be called ‘restricted reviews’, Members of the Oxford Centre for Evidence Based Medicine now suggest ways in which such reviews should be restricted.

In our previous article, we suggested that systematic reviews that have been called ‘rapid reviews’ have been misnamed, that rapidity is not their cardinal feature, and that they would be better called ‘restricted systematic reviews’. This is because when they are performed, certain elements that are required in full systematic reviews are simplified or omitted. Here we contrast full systematic reviews with restricted reviews and show how the former can be modified when performing the latter.

So-called ‘rapid reviews’ have been published in a variety of clinical areas, including digital communication,1 children’s consumption of energy drinks,2 sexual health interventions,3 and cannabis cessation.3 Guideline developers have also used them. For example, the UK’s National Institute for Health and Care Excellence4 commissioned three rapid reviews to inform guidance on midlife approaches to delay or prevent the onset of dementia, disability, and frailty in later life.4

Several methods have been developed and described for conducting restricted reviews. However, there is little consistency among the different approaches.5 6 They may, for example, include updating literature searches of previous reviews; limiting search strategies by database selection and time frame; and screening, data extraction and/or quality assessment by a single reviewer.7 A few studies have compared results from restricted reviews and full systematic reviews on the same topic; some have shown that results from the former did not substantially differ from those of the corresponding full systematic reviews, while others have noted differences. For example, in a comparison of a systematic review and a review in which checking by a second reviewer was restricted to 20% (referred to as an ‘enhanced rapid review’), the restricted review identified all 10 of the studies in the systematic review.8 The restrictions imposed meant that 2126 fewer titles and 129 fewer full texts required screening by the second reviewer. However, quality assessments generated more ‘unclear’ ratings in restricted reviews than in full systematic reviews.

An operational definition of a restricted review in a flexible framework

Here we propose a flexible framework within an operational definition of restricted reviews (see figure 1), taking into consideration factors such as the available time and resources, while highlighting some core steps that are minimum requirements for all systematic reviews.

Figure 1

A flexible framework outlining the core steps and minimum requirements for completing a restricted systematic review (left-hand column), and additional steps that can be taken at each stage to reduce bias, when time and resources allow (right-hand column).

We consider that restricted reviews should include as part of the team a reviewer with experience in the design and conduct of full systematic reviews. Additional steps, highlighted in figure 1, should be included in order to reduce the risk of bias in the methods. Several reviewers may be involved in a restricted review, which may help in completing the review more quickly. The more reviewers involved, the nearer to a complete review a restricted review can become, enhancing quality without sacrificing time.

Figure 1 shows that all restricted reviews should include a clearly formulated clinical research question and a clear rationale for the needs of the research to inform decision-making. They should also, when possible, involve policymakers, patients, and the public in defining and/or refining the research question. The search strategy should include at least one major scientific database (eg, PubMed) and one other source (eg, another topic-specific database or Google Scholar). To reduce publication bias, additional elements, such as unpublished literature and no language restrictions, can optionally be added. Risk of bias assessment could take a simple approach, such as using the OCEBM Levels of Evidence9 or rapid appraisal tools. For a more complete review, detailed appraisal tools, such as the Cochrane Risk of Bias tools, can be considered. Determination of study eligibility, data extraction, risk of bias assessment, and data analysis either can be performed by a single reviewer or can, as an additional step, include partial verification by a second or more reviewers, to reduce selection bias and data error bias. The latter would depend on the time available and might also reflect the complexity of the review question. In all cases, the limitations, and especially sampling bias, should be outlined and addressed.

Conclusions

Restricted systematic reviews will probably result in more cautious interpretations and recommendations than full reviews. A survey of decision makers and guideline developers about how much uncertainty they would accept when using restricted reviews suggests that there is a willingness to accept some possible trade-off in internal validity in exchange for timeliness.10 It should be noted that systematic reviews have their own limitations, on which we do not reflect here, and these limitations therefore also apply to restricted reviews.11

If restricted reviews are conducted well, the information obtained could be used by policymakers and clinicians in making decisions about healthcare interventions. Restricted reviews may also have an important role in assessing whether full systematic reviews requiring additional steps are warranted.

Acknowledgments

The authors thank Jack O’Sullivan and Rafael Perera for helpful discussions.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
View Abstract

Footnotes

  • Contributors All authors conceived and designed the study, and contributed critically to subsequent revisions and approved the final manuscript. AP, JKA, and IO wrote the first draft of the manuscript.

  • Funding AP, KRM, IO, and CH are co-investigators of The Evidence Synthesis Working Group, which is funded by the National Institute for Health Research School for Primary Care Research (NIHR SPCR) (project number 390).

  • Disclaimer The views expressed in this commentary represent the views of the authors and not necessarily those of the host institution, the NHS, the NIHR, or the Department of Health.

  • Competing interests AP reports grants from NIHR, grants from NIHR School of Primary Care Research, during the conduct of the study; and occasionally receives expenses for teaching Evidence-Based Medicine. CH reports that he has received expenses and fees for media work, expenses from WHO, and holds grant funding from the NIHR, the NIHR School of Primary Care Research, The Wellcome Trust and WHO. He has received financial remuneration from an asbestos case. He has also received income from the publication of a series of toolkit books published by Blackwells. On occasion, he receives expenses for teaching EBM and is also paid for his GP work in NHS out of hours and as editor in Chief of the BMJ EBM Journal. CEBM jointly runs the EvidenceLive Conference with the BMJ and the Overdiagnosis Conference with some international partners; these conferences are run on a non-profit making model. IO reports funding from NIHR School of Primary Care Research, during the conduct of the study. KRM receives funding from the NHS NIHR including the Health Technology Assessment programme as Primary Care panel Chair. He is also Director of an MSc in Systematic Reviews. JKA is a member of the Centre for Evidence Based Medicine.

  • Patient consent Not required.

  • Provenance and peer review Commissioned; internally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles