Article Text
PDF
Abstract
Developing a clinical practice guideline (CPG) is expensive and time-consuming and therefore often unrealistic in settings with limited funding or resources. Although CPGs form the cornerstone of providing synthesised, systematic, evidence-based guidance to patients, healthcare practitioners and managers, there is no added benefit in developing new CPGs when there are accessible, good-quality, up-to-date CPGs available that can be adapted to fit local needs. Different approaches to CPG development have been proposed, including adopting, adapting or contextualising existing high-quality CPGs to make recommendations relevant to local contexts. These approaches are attractive where technical and financial resources are limited and high-quality guidance already exists. However, few examples exist to showcase such alternative approaches to CPG development. The South African Guidelines Excellence project held a workshop in 2017 to provide an opportunity for dialogue regarding different approaches to guideline development with key examples and case studies from the South African setting. Four CPGs represented the topics: mental health, health promotion, chronic musculoskeletal pain and prehospital emergency care. Each CPG used a different approach, however, using transparent, reportable methods. They included advisory groups with representation from content experts, CPG users and methodologists. They assessed CPGs and systematic reviews for adopting or adapting. Each team considered local context issues through qualitative research or stakeholder engagement. Lessons learnt include that South Africa needs fit-for-purpose guidelines and that existing appropriate, high-quality guidelines must be taken into account. Approaches for adapting guidelines are not clear globally and there are lessons to be learnt from existing descriptions of approaches from South Africa.
- public health
- primary care
- mental health
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Background
Clinical practice guideline (CPG) development tends to be expensive, skills-intensive and time-consuming and therefore often unrealistic in resource-constrained settings. Although CPGs form the cornerstone of providing synthesised, systematic, evidence-based guidance to patients, healthcare practitioners and managers, it is not good use of time or resources to develop new CPGs when there are accessible, good-quality, up-to-date CPGs available that can be adapted to fit local needs. Furthermore, the higher burden of disease in low-income and middle-income countries also arguably makes the focus on evidence-based guidelines even more urgent, to minimise wastage and ensure the best patient care for optimal cost.1 2
As such, alternative approaches to de novo (new) CPG development have been proposed, some of which either adopt or adapt existing guidelines to local settings,2–4 some use the Grading of Recommendations Assessment, Development and Evaluation (GRADE), termed adolopment,4 while others accelerate certain steps in the guideline development process.5 These approaches are attractive where resources are limited and high-quality guidance already exists.6 7 These methods provide a key vehicle for formal guideline teams, clinicians and decision makers to produce contextually relevant and robust guidance for their setting. To date, there are limited examples in the literature showcasing alternative CPG development methods and standards for teams in resource-constrained settings, whether in high-income or low-income countries.3 8–10 As such, resulting clinical guidance in these settings often varies in quality and applicability.10
In order to address this gap, we present four purposefully selected case studies from South Africa, displaying different approaches for adapted CPG development. This draws from the South African Guidelines Excellence project, a multipartner research initiative aimed at supporting the understanding of standards of national CPG development, adaptation, implementation and capacity building.11 We also suggest future considerations and lessons learnt for CPG teams that choose to adapt a guideline.
Case studies
Case study 1: national CPG for the management of people with serious mental illness and co-occurring substance-use disorders in South African psychiatric settings
The South African National Department of Health (NDOH) commissioned the University of Cape Town’s Department of Psychiatry to draft a ‘policy guide’ for managing people with serious mental illness and co-occurring substance-use disorders (dual diagnosis). This CPG’s target users were mental health practitioners practising in psychiatric settings. The NDOH requested the first draft to be available for stakeholder input within 4 months of project start, with a final version presented at 12 months. The CPG panel included one methodologist and three content experts. In addition, there was a plan for consultation with stakeholders representing psychiatrists, psychologists, social workers, addiction counsellors and service administrators in the field.
The CPG development followed these steps:
The team applied the WHO approach for CPG development.12 They agreed on the outcomes (voted on and discussed a priori by the panel) and used a PICO (Population, Intervention, Comparison and Outcome) framework to formulate health questions.
Values and preferences were prespecified and were aimed to minimise cost in the event of small clinical effects.
A comprehensive search was conducted in PubMed and the Cochrane Library for CPGs and systematic reviews published in the past 5 years.
Available CPGs and systematic reviews were appraised with the Appraisal of Guidelines for Research & Evaluation II (AGREE-II) tool (CPGs) and the (A MeaSurement Tool to Assess systematic Reviews) AMSTAR tool (systematic reviews).13 14
Where systematic reviews were available, each health question was reassessed using the GRADE methodology.15 Re-GRADEing was necessary as the systematic reviews differed in their assessment of imprecision where the panel used a clinical threshold approach.16 Recommendations were based on the GRADE quality of evidence profiles.
There were several challenges. The limited time from inception to first draft did not allow for training of all panel members in GRADE methodology. Consequently, with one methodologist, this meant non-duplicated search and selection, and assessments using the appraisal and GRADE tools. The GRADE process was difficult and time-consuming, necessitating revision of all imprecision ratings from the original systematic reviews due to the guideline panel’s use of a clinical threshold method. At times, this required retrospective power analyses. Furthermore, where no systematic reviews were available, existing guidelines were used and needed to be carefully scrutinised, as they often did not use GRADE. The absence of systematic reviews of randomised controlled trials had to be considered in making final recommendations.
The methodologist was also a psychiatrist working in this field, and therefore tensions existed between an advocacy-orientated stance versus an objective stance, necessitating careful reflection to minimise potential bias. Decisions regarding inclusion of systematic reviews and CPGs were based on arbitrary classification into ‘high’ versus ‘low-quality’ categories using AMSTAR and AGREE-II, an approach not recommended by the tool developers.
Ethical considerations influenced the formulation of recommendations, as equity plays an important role, where psychiatric patients have been historically marginalised. Quality of evidence as per GRADE, risk:benefit ratios, equity, resource implications, acceptability and feasibility were considered in making recommendations, including all aspects from the GRADE evidence to decision (EtD) framework.17
Following CPG finalisation, four stakeholder workshops were held to share results and clarify contextual issues. Conveying results of GRADE evidence assessments proved challenging and required substantial preliminary information and teaching to non-research stakeholder audiences. Most often workshop participants wanted simple messages regarding ‘what works for dual diagnosis?’ and grappled with the nature of options for treatment. Use of the wording ‘weak’ to qualify GRADE recommendations based on considerable uncertainty provoked concern from participants, and this led to the adoption of the alternative wording ‘conditional’, framed as recommendations conditional on enhanced staffing and resources.
Case study 2: ‘Health for All’, a clinical tool for health promotion in primary care
To minimise the burden of chronic disease, a health promotion approach is required in the delivery of primary healthcare (PHC) in South Africa. A CPG was developed for use by PHC practitioners alongside an adult primary care guideline that is already available.18 The core aim of the CPG was to enable people to take control over and improve their health and its determinants, through a healthier lifestyle and greater self-efficacy.
CPG development was commissioned by the PHC Directorate of the South African NDOH and led by an independent public health specialist with experience in primary care practice and guideline development. The guideline panel formed included five health professionals, who jointly had experience in primary care, health education, research including evidence synthesis and the development of evidence-based CPGs.
The CPG development followed these steps:
Definition of the concept, theories of health promotion and social marketing to guide the process.
Conceptualisation of the look and feel of the CPG and how it would be best used in practice in conjunction with adult primary care CPG. This included defining attributes such as language, illustration and relevance to clinical situations. This followed the design of an algorithmic approach that aligned risk assessment and delivery of health promotion alongside clinical assessments of patients.
Regular consultation with NDOH and PHC-relevant, condition-specific NDOH programmes, and presentation of drafts at NDOH national and provincial fora to ensure agreement between the developers and NDOH regarding the specific risks and conditions to be included.
Population of each selected risk and condition sections of the framework with accurate user-friendly clinical information with active health messages using the PICO framework.
This was followed by a hierarchical approach to evidence selection consisting of (1) a search for WHO graded guidelines from 2010, in the absence of which (2) a search of the Cochrane Library from 2010, failing which (3) a search for non-Cochrane, high-quality systematic reviews, or if there was still no evidence (4) a systematic search for evidence by the Cochrane Library (see figure 1). A training package with a guide and additional tools was designed and piloted by the team in three South African provinces, followed by focus group discussions with users. This provided feedback for the final CPG.
Finalisation of the CPG which was signed off by the Director General for Health for implementation.
Case 2 search strategy.
CPG development was completed within 15 months, while endorsement took an additional 18 months. The CPG process, including travel and initial printing, was funded by a non-governmental agency. The national engagement forum was made possible by other sources of non-governmental funding. The NDOH is progressing with dissemination and training for PHC health professionals.
Case study 3: prehospital CPG for South African emergency care providers
South African prehospital emergency care providers have been practising based on protocols that are more than a decade old.19 Consequently, in August 2015, the Health Professions Council of South Africa Professional Board of Emergency Care awarded a bid to develop the first evidence-based CPG for the South African emergency care profession.20 This CPG was developed under the direction of the African Federation for Emergency Medicine collaborating with other research institutions and emergency care departments. The primary aim was to develop a contextually appropriate evidence-based CPG for prehospital emergency care providers and managers. The CPG needed to be patient-centred and realistic and ensure continuation of care through the emergency system from prehospital to patient discharge.21 22
Due to limitations in time and funding, de novo CPG development was not possible.2 Thus, the approach started with engagement with an advisory board of key stakeholders, including methodologists, prehospital providers and various medical specialists, followed by the CPG panel identifying and appraising existing CPGs and using these to develop contextually appropriate evidence-based CPGs.
Key steps in the process included the following:
Clarifying the clinical questions, followed by searching for existing CPGs.
We used systematic review methods, including comprehensive searching of the literature, critical appraisal and synthesis.23
Full CPGs were critically appraised using the AGREE-II tool.24 The AGREE-II scores were used to assess and prioritise CPGs for use, particularly if there were two or more on similar topics.
Within priority areas, different recommendations often overlapped; in this case the most current and unambiguous recommendation was accepted.
High-quality, relevant and up-to-date CPGs were prioritised through consensus by the panel. Where possible, only one guideline per recommendation was used.
Then, the process of adopting, adapting or contextualising existing CPGs for local use was based on an approach used by Dizon et al in the Philippines.2 Decisions were made by the CPG panel following review by the advisory board. Where applicable, ‘practice points’ were added; these included more specific guidance to practitioners regarding performance of particular interventions or clarified clinical steps (eg, how to prepare and administer a medicine related to a particular recommendation).
Overall, the steps and processes are similar to those for de novo CPG development. However, the key difference was identifying and synthesising high-quality CPGs for emergency care, instead of use of primary research22 (table 1).
De novo versus case 3 alternative guideline development approach
The project was completed within 1 year. The next steps include creating an end-user document (protocol) for use by paramedics, further integration, updating and realignment of prehospital scopes of practice, based on CPG recommendations and planning for CPG updates. The CPG is currently being implemented nationally for South African prehospital care.
Case study 4: a CPG for the management of chronic musculoskeletal pain in South African PHC settings
Globally, and in South Africa, musculoskeletal conditions contribute significantly to the years lived with disability.25 The prevalence of chronic pain is high and there are indications that the prevalence of chronic pain may be higher in developing countries.26
The aim of the CPG was to provide contextually relevant, evidence-informed guidance on the assessment and management of chronic musculoskeletal pain (CMSP), to optimise the health outcomes of patients. Since CMSP is a multidimensional phenomenon, the CPG needed to be holistic and multimodal, to include pharmacological and non-pharmacological interventions. The target users were healthcare practitioners involved with the management of chronic pain in PHC settings.
This CPG was developed through the process of contextualisation of existing high-quality CPGs.3 The CPG panel included methodologists, a diverse group of healthcare practitioners, researchers, educators and healthcare managers. Patient input was sought as part of development, along with broader stakeholder consultation. The process of development took approximately 18 months.
The CPG contextualisation method followed these steps:
We conducted qualitative research with the aim to develop a framework of local context factors relevant for framing CPG recommendations. The perspectives of patients and healthcare practitioners about the factors influencing pain care were explored.
A systematic review was conducted to identify existing CPGs on the topic. The included CPGs were appraised using AGREE-II.12 27 Only CPGs with high-quality methodology were included.
Clinical recommendations were extracted from the CPGs and synthesised using a specific writing guide to form a core set of recommendations.
We used a formal consensus process in which a multidisciplinary team of experts evaluated the proposed recommendations and endorsed them as relevant for the local primary care context.
The expert group developed specific criteria (context and practice points) using the framework of contextual factors that were developed to enhance the implementability of recommendations. The recommendations were aligned with a typical patient journey as extracted from the qualitative data.
An external review of the recommendations and proposed clinical pathway was done by additional stakeholders to evaluate the acceptability of the recommendations for the intended setting.
An end-user document with an implementation plan is currently being developed.
The advantage of the contextualising method is the integration of multiple stakeholder perspectives and the consideration of local context factors. However, CPG contextualisation is dependent on the availability of good-quality and up-to-date existing CPGs.
Lessons learnt: challenges and opportunities
Across the case studies, access to funding and dedicated human resources were a significant challenge, and infrastructure, agreed standards and technical staff to support processes were lacking. Support often came from academics or public health specialists responding to a particular request, additional to their regular working hours. Furthermore, additional training was required for most involved in the CPG development process, with a focus on using GRADE and critical appraisal with the AGREE-II tool. Various opportunities exist, such as providing appropriate training for existing and up-and-coming CPG developers in de novo and alternative development methods and providing appropriate resources, such as a toolkit to guide development for novice and experienced CPG developers.28 Training and capacity building would be most useful, where the need is greatest, for example in ministry technical teams, professional societies and university departments, where CPG quality is often lacking when compared with international groups or national bodies such as the National Institute for Health and Care Excellence (NICE) and Scottish Intercollegiate Guidelines Network (SIGN).29
Across the four cases studies, recommendations often originated from CPGs developed in high-income settings, and significant changes were required in order to implement CPGs in a resource-constrained setting. Research in generating tiered recommendations, based on available resources, for example in high-resource versus low-resource settings, is a potential opportunity that can assist where methods for how to contextualise recommendations are still unclear and variable.
In summary, key learnings revolved around navigating funding and human resource challenges, whereas opportunities include addressing guideline training gaps and investing in strengthening adaptation and contextualisation of guideline recommendations through stakeholder engagement for efficient guideline development and enhanced uptake.
Discussion
CPG development teams in resource-constrained settings often work with significant technology, human resource and budget restrictions, and therefore de novo CPG development is not always feasible or efficient. Adapted CPG methods therefore may bypass de novo methods, by efficiently using existing high-quality evidence and streamlining CPG development steps. However, adapted methods must still be rigorous, transparent and adhere to the same standards as de novo methods. These amendments to standard CPG development methods, by definition, should thus be responsive, considering the needs of the local CPG development team, topic and setting, without compromising rigour and transparency. Unlike de novo methods, adapted methods have less guidance available on development standards such as those published by the Institute of Medicine30 or Guidelines International Network.31 With the anticipated steady increase in CPGs that use alternative methods, developing quality checklists and adapted CPG standards warrants attention, especially since AGREE-II, the go-to appraisal tool, does not address alternative CPG development issues.
In our experience, alternative development methods should aim to create fit-for-purpose CPGs that consider local contexts with a focus on strengthening CPG implementation and uptake. Such fit-for-purpose CPGs, as we have shown, can be moulded to the unique needs of the setting without compromising on rigour. Even though there was significant variation in methods between the cases presented, all CPGs included important aspects of standard CPG development, from priority setting, comprehensive searches and quality appraisal, to stakeholder input. However, approaches in developing guidance varied across the cases; some used the GRADE EtD or the adopt-adapt-contextualise model in generating recommendations, whereas others focused on implementation through making contextual recommendations or a user-friendly end-user product. We found that case studies provide a useful platform to display and contrast emerging methods in guideline development approaches and offered a valuable approach for reflecting on learning.
Conclusion
CPG development should be a rigorous, transparent and inclusive process, which is contextualised to the needs of the setting. Approaches for adapting CPGs are not clear globally, and often include a mix of pragmatism and rigour. There is a growing group of experts in poorer countries who are gaining experience in adapting CPGs for local needs. There are lessons to be learnt from approaches used in South Africa.
References
Footnotes
Contributors MM and TK drafted the manuscript. MM, DE, HT and BD contributed to the case studies. Case study 1: HT was part of the guideline panel and the guideline methodologist. Case study 2: BD led the guideline development team and actively participated in the design, content and research for the evidence base. Case study 3: MM was part of the core guideline panel and guideline methodologist. Cast study 4: DE was the principal investigator. All authors reviewed and approved the final version of this manuscript.
Funding Project SAGE is the result of funding provided by the South African Medical Research Council (SAMRC) in terms of the SAMRC’s Flagships Awards Project SAMRC-RFA-IFSP-01-2013/SAGE.
Competing interests None declared.
Ethics approval Ethical approval was not required for this paper or individual case studies in the manuscript. The methods reported are all part of public record as part of the guidelines that were developed.
Provenance and peer review Not commissioned; externally peer reviewed.
Patient consent for publication Not required.