The Pandora’s Box of Evidence Synthesis and the case for a living Evidence Synthesis Taxonomy
===============================================================================================

* Zachary Munn
* Danielle Pollock
* Timothy Hugh Barker
* Jennifer Stone
* Cindy Stern
* Edoardo Aromataris
* Holger J Schünemann
* Barbara Clyne
* Hanan Khalil
* Reem A Mustafa
* Christina Godfrey
* Andrew Booth
* Andrea C Tricco
* Alan Pearson

*   Systematic Reviews as Topic
*   Evidence-Based Practice

## Introduction

Have we, as an evidence-based health community, opened the Pandora’s box of evidence synthesis? There now exists a plethora of overlapping evidence synthesis approaches and duplicate, redundant and poor-quality reviews.1–4 After years of advocating for the need for systematic reviews of the evidence, there is a risk that this message been disseminated too widely and has been misinterpreted in this process. We have reached a point where in some fields more reviews exist than clinical trials, where same topic reviews are being conducted in parallel, and evidence syntheses possess limited utility for decision-making because of their poor quality or poor reporting.To paraphrase the late Douglas Altman,5 it is possible we are now at a stage where we need less reviews, better reviews and reviews done for the right reason—as opposed to the current state of mass production (approximately 80 reviews per day).6 

How have we arrived at this point - and is it a point of no return? One obvious reason is that over time, systematic review methods (and evidence synthesis more broadly) have become increasingly demanding, complex and multifaceted. This evolution in review methods has caused, and continues to cause, great confusion for both novice and experienced synthesisers of evidence.7–10 For example, confusion persists between scoping reviews and systematic reviews and the correct application of these approaches to evidence synthesis. This, in turn, results in scoping reviews assessing the effects of interventions when they are neither intended nor equipped to do so.11 This confusion is evident with other evidence synthesis methodologies, including scoping reviews and mapping reviews, where further guidance may be needed to help with the appropriate choice of methodologies.12 Indeed, it has already been argued that the current proliferation of review types is creating challenges for the terminology for describing such reviews, creating fundamental issues for evidence synthesisers.7 Although the utility of an encompassing toolkit for evidence synthesis to assist in answering an array of complex and multifaceted questions is to be welcomed, the resultant confusion (and complexity) associated with this expansion cannot be overlooked. With supervisors encouraging increasing numbers of novice researchers to undertake systematic reviews (fuelled by the misplaced idea of a quick or easy publication that requires little [if any] infrastructure support [or ethics approval] or as a requirement of doctorate studies or grant applications), and funders and frameworks rightly promoting interventional research be developed based on the evidence, we are likely to encounter further proliferation of misplaced, misconducted and redundant evidence synthesis projects.13 

We are concerned about this current state of affairs within the field of evidence synthesis, but believe we have not yet reached the point of no return. As such, this article discusses some of the pitfalls associated with an ever-expanding toolkit for evidence synthesis (likened to the opening of Pandora’s Box) and discuss potential solutions for improving the cohesiveness of evidence synthesis.

## Confusion regarding the many evidence synthesis approaches

Within the family of systematic reviews there are different approaches, including reviews addressing interventions, prognosis, test accuracy, values and other overarching types of systematic reviews.14 Despite the usefulness of these systematic reviews to synthesise research, circumstances persist where they are unwieldy or not the tool of choice to meet the requirements of knowledge users.14–16 This recognition has led to many alternative approaches to evidence synthesis, including realist reviews, scoping reviews, umbrella reviews, concept analyses and others.7–10 17 In addition, numerous methods have been proposed for the synthesis of qualitative research,18 19 including thematic synthesis, realist synthesis, content analysis, meta-ethnography and meta-aggregation.18–20 These developments have expanded the array of methods to conduct evidence syntheses as described in the literature.

Although the differentiation of evidence synthesis types has made it possible to ensure that methods are tailored to particular questions, one pitfall has been the emergence of other (perhaps) less useful and/or rigorous approaches. These include (among others) the numerous sets of products collectively referred to as ‘rapid’ reviews and integrative reviews. Rapid reviews can be critiqued for their lack of uniformity or agreement on an ideal approach, and often appear haphazard in their conduct. However, guidance on the conduct of rapid reviews is being produced21 and they have played a key role in providing evidence in emergency situations such as during the COVID-19 pandemic,22 and when done appropriately (and transparently) rapid reviews do have a place in the evidence synthesis ecosystem when requested by decision-makers. Integrative reviews, which emerged as a reaction to the early practice of only including randomised controlled trials in systematic reviews, represent a diffuse methodology which now appear redundant given the appearance of more rigorous approaches to incorporating different study designs (such as non-randomised and randomised studies in intervention reviews23 or quantitative and qualitative studies in a single review24 and synthesising different types of evidence (such as qualitative evidence synthesis), in addition to statistical methods to meta-analyse non-randomised studies.

Other evidence synthesis types possess clearly defined methods but are yet to realise their full potential within the context of evidence based practice. For example, concept analyses,25 26 ‘a process whereby concepts are logically and systematically investigated to form clear and rigorously constructed conceptual definitions,’26 share at least one similarity with scoping reviews (which are more robust in their approaches) in their intention to clarify concepts in the literature.27 However, concept analyses lack the immediately apparent utility of scoping reviews. A case can be made, given that concept analyses have been critiqued for having no impact on practice,25 to revisit and reconsider whether a separate role exists outside of the scoping review.

It is now possible that almost any question can be answered by some application of an appropriate evidence synthesis process. However, it can be daunting and difficult for researchers to determine what type of synthesis approach is required, how to conceptualise and phrase the synthesis question, inclusion criteria and select the appropriate methods for analysis and synthesis. Typologies, taxonomies and other evidence synthesis classification systems may play a role here.

## Typologies and taxonomies of evidence synthesis

Typologies of evidence synthesis types offer a starting reference point for researchers, policy makers and funders when deciding on a review approach.8–10 These individual typologies provide useful guidance on different review types but remain static—largely as traditional publications in a journal format that are not up-to-date and with minimal links to supporting materials. Furthermore, the development processes for many of these taxonomies are largely unclear with non-transparent methods. In a discussion paper by Gough *et al*,7 the authors distinguished between various review designs and methods but stopped short of providing a taxonomy of review types. The rationale for this was that in the field of evidence synthesis ‘the rate of development of new approaches to reviewing is too fast and the overlap of approaches too great for that to be helpful.’7 Instead, the authors provide a useful description of how reviews may differ and more importantly, an explanation why this may be the case. We concur that evidence synthesis methodology is a rapidly developing field, and that within different systematic review types there may be many different subsets and complexities that need to be addressed. These classifications can be considered to lie at the roots of a much larger family tree.

## The evidence synthesis taxonomy initiative

Previous taxonomies and typologies are useful to categorise and direct researchers to the ideal synthesis approach, however limitations regarding their comprehensiveness, development methods and their currency exist. As the research and policy communities become increasingly aware of the need for evidence-informed choices, the breadth of evidence synthesis approaches also continues to expand. As this enthusiasm for evidence synthesis continues to propagate, we need measures in place to ensure: (1) we are making informed choices in the design and conduct of evidence syntheses; (2) we are producing these reviews as efficiently and accurately as possible, using reproducible and transparent methods and (3) all results and implications from these syntheses are trustworthy. As such, the value of a resource that guides and drives appropriate synthesis approaches in response to relevant clinical and policy needs, such as a research-informed evidence synthesis taxonomy, becomes essential. The authors of this paper are now collaborating with like-minded researchers on such an initiative, including the shared commitment to keep this a living taxonomy, continuously updated (such as via a living wiki platform) and refined alongside advances in the field. This work complements a decision tool that currently steers researchers and decision-makers towards the right choice when conducting a review—aptly named ‘Right Review’.28 In addition, this work will align neatly with the position statement from Evidence Synthesis International, in recognising the need to develop and share standards, terminology and methodology consistently across the field while acknowledging the need for ‘fit for purpose’ approaches for diverse evidence requirements.29 

We envisage a number of challenges when attempting this work. First, we will need to ensure we commit to a truly living approach given the rate of development of new methods and approaches. Second, given our collaborative approach, it is likely there will be strong views among the various contributors and organisations, which will require the use of formal consensus development methods. Third, we want to ensure all stakeholders are included, which will result in challenges organising a diverse group of people and opinions, but we see this as critical to ensuring the utility of the final taxonomy. The inclusion of these diverse stakeholders will be critical to ensure the taxonomy addresses research waste and overproduction of reviews. As an example, by ensuring the inclusion of editors, we hope that journals and publishers may be able to better identify redundant reviews or reviews applying inappropriate methodology prior to publication, in addition to updating author guidelines and requirements as a means to prevent these problems.

## Conclusion

We suggest that the current proliferation of evidence synthesis types is creating challenges for the terminology for describing such reviews and contributing to confusion in the field and has contributed to the mass production of redundant, misleading and conflicted evidence base. Currently, no well-developed, continuously updated framework exists to name and categorise the different approaches to evidence synthesis. As such, a rigorous, empirically derived taxonomy is required that can comprehensively identify extant methods; clarify distinctive nomenclature; and can provide a classification system of methods and approaches. We hope that these efforts will reduce confusion within evidence synthesis, leading reviewers to select the best approach for their question and purpose among all current evidence synthesis options. Despite the challenges expected with developing this work (including achieving consensus across such diverse fields), we remember that at the bottom of Pandora’s box, there was hope, after all. We, too, remain hopeful regarding the continued utility of appropriate evidence synthesis to inform high quality decisions.

## Ethics statements

### Patient consent for publication

Not applicable.

## Footnotes

*   Twitter @ZacMunn

*   Contributors ZM conceived the article, wrote the first draft, incorporated feedback, finalised the manuscript. DP, THB, JS, CS, EA, HJS, BC, HK, RAM, CG, AB, ACT, AP all contributed to discussions and initial ideas in the article, reviewed drafts, provided feedback and approved.

*   Competing interests ZM is employed by JBI, an evidence-based healthcare research and development organisation situated within the University of Adelaide and is supported by an NHMRC Investigator Grant 1195676. ACT is funded by the Tier 2 Canada Research Chair in Knowledge Synthesis.

*   Provenance and peer review Not commissioned; externally peer reviewed.

[http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/)
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/).

## References

1.   Møller MH , Ioannidis JPA , Darmon M . Are systematic reviews and meta-analyses still useful research? we are not sure. Intensive Care Med 2018;44:518–20.[doi:10.1007/s00134-017-5039-y](http://dx.doi.org/10.1007/s00134-017-5039-y) pmid:http://www.ncbi.nlm.nih.gov/pubmed/29663048 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

2.   Ioannidis JPA . The mass production of redundant, misleading, and Conflicted systematic reviews and meta-analyses. Milbank Q 2016;94:485–514.[doi:10.1111/1468-0009.12210](http://dx.doi.org/10.1111/1468-0009.12210) pmid:http://www.ncbi.nlm.nih.gov/pubmed/27620683 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1111/1468-0009.12210&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=27620683&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

3.   Page MJ , Shamseer L , Altman DG , et al . Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med 2016;13:e1002028.[doi:10.1371/journal.pmed.1002028](http://dx.doi.org/10.1371/journal.pmed.1002028) pmid:http://www.ncbi.nlm.nih.gov/pubmed/27218655 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1371/journal.pmed.1002028&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=27218655&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

4.   Campbell JM , Kavanagh S , Kurmis R , et al . Systematic reviews in burns care: poor quality and getting worse. J Burn Care Res 2017;38:e552–67.[doi:10.1097/BCR.0000000000000409](http://dx.doi.org/10.1097/BCR.0000000000000409) pmid:http://www.ncbi.nlm.nih.gov/pubmed/28253213 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

5.   Altman DG . The scandal of poor medical research. BMJ 1994;308:283–4.[doi:10.1136/bmj.308.6924.283](http://dx.doi.org/10.1136/bmj.308.6924.283) pmid:http://www.ncbi.nlm.nih.gov/pubmed/8124111 
    
    [FREE Full Text](http://ebm.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMDgvNjkyNC8yODMiO3M6NDoiYXRvbSI7czoyMDoiL2VibWVkLzI4LzMvMTQ4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

6.   Hoffmann F , Allers K , Rombey T , et al . Nearly 80 systematic reviews were published each day: observational study on trends in epidemiology and reporting over the years 2000-2019. J Clin Epidemiol 2021;138:1–11.[doi:10.1016/j.jclinepi.2021.05.022](http://dx.doi.org/10.1016/j.jclinepi.2021.05.022) pmid:http://www.ncbi.nlm.nih.gov/pubmed/34091022 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2021.05.022&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

7.   Gough D , Thomas J , Oliver S . Clarifying differences between review designs and methods. Syst Rev 2012;1:28.[doi:10.1186/2046-4053-1-28](http://dx.doi.org/10.1186/2046-4053-1-28) pmid:http://www.ncbi.nlm.nih.gov/pubmed/22681772 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1186/2046-4053-1-28&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=22681772&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

8.   Grant MJ , Booth A . A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J 2009;26:91–108.[doi:10.1111/j.1471-1842.2009.00848.x](http://dx.doi.org/10.1111/j.1471-1842.2009.00848.x) pmid:http://www.ncbi.nlm.nih.gov/pubmed/19490148 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1111/j.1471-1842.2009.00848.x&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=19490148&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

9.   Tricco AC , Tetzlaff J , Moher D . The art and science of knowledge synthesis. J Clin Epidemiol 2011;64:11–20.[doi:10.1016/j.jclinepi.2009.11.007](http://dx.doi.org/10.1016/j.jclinepi.2009.11.007) pmid:http://www.ncbi.nlm.nih.gov/pubmed/20189767 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2009.11.007&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=20189767&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

10.  Munn Z , Stern C , Aromataris E , et al . What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol 2018;18:5.[doi:10.1186/s12874-017-0468-4](http://dx.doi.org/10.1186/s12874-017-0468-4) 
    
    

11.  Khalil H , Peters MD , Tricco AC , et al . Conducting high quality scoping reviews-challenges and solutions. J Clin Epidemiol 2021;130:156–60.[doi:10.1016/j.jclinepi.2020.10.009](http://dx.doi.org/10.1016/j.jclinepi.2020.10.009) pmid:http://www.ncbi.nlm.nih.gov/pubmed/33122034 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

12.  Khalil H , Tricco AC . Differentiating between mapping reviews and scoping reviews in the evidence synthesis ecosystem. J Clin Epidemiol 2022;26.[doi:10.1016/j.jclinepi.2022.05.012](http://dx.doi.org/10.1016/j.jclinepi.2022.05.012) 
    
    

13.  Skivington K , Matthews L , Simpson SA , et al . A new framework for developing and evaluating complex interventions: update of medical Research Council guidance. BMJ;2021:n2061.[doi:10.1136/bmj.n2061](http://dx.doi.org/10.1136/bmj.n2061) 
    
    

14.  Munn Z , Stern C , Aromataris E , et al . What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol 2018;18:5.[doi:10.1186/s12874-017-0468-4](http://dx.doi.org/10.1186/s12874-017-0468-4) pmid:http://www.ncbi.nlm.nih.gov/pubmed/29316881 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1186/s12874-017-0468-4&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

15.  Pearson A , Wiechula R , Court A , et al . The JBI model of evidence-based healthcare. Int J Evid Based Healthc 2005;3:207–15.[doi:10.1111/j.1479-6988.2005.00026.x](http://dx.doi.org/10.1111/j.1479-6988.2005.00026.x) pmid:http://www.ncbi.nlm.nih.gov/pubmed/21631749 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1111/j.1479-6988.2005.00026.x&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=21631749&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

16.  Pearson A . Balancing the evidence: incorporating the synthesis of qualitative data into systematic reviews. JBI Reports 2004;2:45–64.[doi:10.1111/j.1479-6988.2004.00008.x](http://dx.doi.org/10.1111/j.1479-6988.2004.00008.x) 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1111/j.1479-6988.2004.00008.x&link_type=DOI) 

17.  Sutton A , Clowes M , Preston L , et al . Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J 2019;36:202–22.[doi:10.1111/hir.12276](http://dx.doi.org/10.1111/hir.12276) pmid:http://www.ncbi.nlm.nih.gov/pubmed/31541534 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1111/hir.12276&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

18.  Thorne S , Jensen L , Kearney MH , et al . Qualitative metasynthesis: reflections on methodological orientation and ideological agenda. Qual Health Res 2004;14:1342–65.[doi:10.1177/1049732304269888](http://dx.doi.org/10.1177/1049732304269888) pmid:http://www.ncbi.nlm.nih.gov/pubmed/15538004 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1177/1049732304269888&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=15538004&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 
    
    [Web of Science](http://ebm.bmj.com/lookup/external-ref?access_num=000225082400003&link_type=ISI) 

19.  Barnett-Page E , Thomas J . Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol 2009;9:59.[doi:10.1186/1471-2288-9-59](http://dx.doi.org/10.1186/1471-2288-9-59) pmid:http://www.ncbi.nlm.nih.gov/pubmed/19671152 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1186/1471-2288-9-59&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=19671152&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

20.  Booth A , Noyes J , Flemming K , et al . Structured methodology review identified seven (retreat) criteria for selecting qualitative evidence synthesis approaches. J Clin Epidemiol 2018;99:41–52.[doi:10.1016/j.jclinepi.2018.03.003](http://dx.doi.org/10.1016/j.jclinepi.2018.03.003) pmid:http://www.ncbi.nlm.nih.gov/pubmed/29548841 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

21.  Garritty C , Gartlehner G , Nussbaumer-Streit B , et al . Cochrane rapid reviews methods group offers evidence-informed guidance to conduct rapid reviews. J Clin Epidemiol 2021;130:13–22.[doi:10.1016/j.jclinepi.2020.10.007](http://dx.doi.org/10.1016/j.jclinepi.2020.10.007) pmid:http://www.ncbi.nlm.nih.gov/pubmed/33068715 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2020.10.007&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=33068715&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

22.  Fretheim A , Brurberg KG , Forland F . Rapid reviews for rapid decision-making during the coronavirus disease (COVID-19) pandemic, Norway, 2020. Euro Surveill 2020;25.[doi:10.2807/1560-7917.ES.2020.25.19.2000687](http://dx.doi.org/10.2807/1560-7917.ES.2020.25.19.2000687) pmid:http://www.ncbi.nlm.nih.gov/pubmed/32431291 
    
    

23.  Cuello-Garcia CA , Santesso N , Morgan RL , et al . Grade guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines. J Clin Epidemiol 2022;142:200–8.[doi:10.1016/j.jclinepi.2021.11.026](http://dx.doi.org/10.1016/j.jclinepi.2021.11.026) pmid:http://www.ncbi.nlm.nih.gov/pubmed/34800676 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

24.  Stern C , Lizarondo L , Carrier J , et al . Methodological guidance for the conduct of mixed methods systematic reviews. JBI Evid Synth 2020;18:2108–18.[doi:10.11124/JBISRIR-D-19-00169](http://dx.doi.org/10.11124/JBISRIR-D-19-00169) pmid:http://www.ncbi.nlm.nih.gov/pubmed/32813460 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.11124/JBISRIR-D-19-00169&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

25.  Draper P . A critique of concept analysis. J Adv Nurs 2014;70:1207–8.[doi:10.1111/jan.12280](http://dx.doi.org/10.1111/jan.12280) pmid:http://www.ncbi.nlm.nih.gov/pubmed/24467340 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1111/jan.12280&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=24467340&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

26.  Ream E , Richardson A . Fatigue: a concept analysis. Int J Nurs Stud 1996;33:519–29.[doi:10.1016/0020-7489(96)00004-1](http://dx.doi.org/10.1016/0020-7489(96)00004-1) pmid:http://www.ncbi.nlm.nih.gov/pubmed/8886902 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1016/0020-7489(96)00004-1&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=8886902&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 
    
    [Web of Science](http://ebm.bmj.com/lookup/external-ref?access_num=A1996VJ74400007&link_type=ISI) 

27. 1.  
    2.   Booth A , 
    3.   Sutton A , 
    4.   Clowes M  
    
    ed.  Booth A ,  Sutton A ,  Clowes M  . Systematic approaches to a successful literature review. Third. London: Sage, 2021.
    
    

28.  Amog K , Pham Ba' , Courvoisier M , et al . The web-based "Right Review" tool asks reviewers simple questions to suggest methods from 41 knowledge synthesis methods. J Clin Epidemiol 2022;147:42–51.[doi:10.1016/j.jclinepi.2022.03.004](http://dx.doi.org/10.1016/j.jclinepi.2022.03.004) pmid:http://www.ncbi.nlm.nih.gov/pubmed/35314349 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom) 

29.  Gough D , Davies P , Jamtvedt G , et al . Evidence synthesis International (ESI): position statement. Syst Rev 2020;9:155.[doi:10.1186/s13643-020-01415-5](http://dx.doi.org/10.1186/s13643-020-01415-5) pmid:http://www.ncbi.nlm.nih.gov/pubmed/32650823 
    
    [CrossRef](http://ebm.bmj.com/lookup/external-ref?access_num=10.1186/s13643-020-01415-5&link_type=DOI) 
    
    [PubMed](http://ebm.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Febmed%2F28%2F3%2F148.atom)