Article Text
PDF
Statistics from Altmetric.com
“The good news is that evidence can matter. The bad news is that it often does not.”1
If imitation is the sincerest form of flattery, the clinicians and clinical epidemiologists who promoted evidence-based medicine in the early 1990s should feel flattered. Evidence-based medicine now has many imitators: from evidence-based nursing,2 dentistry3 and public health4 through to evidence-based social work and social interventions.5 6 To this growing list, we can now add evidence-based global health policy, a ‘movement’ that is gaining increasing prominence.7 8
We are firm supporters of this movement. Indeed, we lead a project called the Evidence to Policy initiative (E2Pi), which aims to help narrow the gap between evidence synthesis and practical policymaking in global health. Research evidence has undoubtedly been crucial in formulating countless global health policies which have saved many millions of lives.9 Improving the flow of evidence between global health researchers and policymakers is an important tool for improving health outcomes and thus supporting low-income countries in reaching the Millennium Development Goals. For example, up to 70% of deaths of young children could be prevented through the better use of existing evidence.10
However, although evidence-based policymaking in global health has scored many successes, at the same time, we believe there are several common fallacies about its ‘real world’ application. We hope this perspective will spark discussion and debate on its payoffs and pitfalls.
The impact of evidence
Evidence-based global health policymaking aims to improve global health outcomes by urging policymakers to base their policies on the best available evidence – rather than on opinion, whim or political popularity (figure 1). Who exactly is a ‘global health policymaker’? The group is enormously diverse and includes bilateral and multilateral donors, development banks, foundations, presidents and prime ministers, ministries of health and finance, and non-governmental organisations. What they all have in common is ‘the authority to make or influence decisions directly’.8
When such global health actors have used sound research evidence to shape policy, the results have often been dramatic. For example, national and international campaigns to aggressively scale-up evidence-based malaria control tools, such as insecticide-treated bed nets (ITNs) and indoor residual spraying with insecticide, have been associated with large declines in the malaria burden in many countries.11 Thailand's national campaign to promote ‘100% condom use’ among sex workers, another intervention based on high-quality evidence from randomised controlled trials (RCTs), has been associated with a large fall in HIV transmission nationally.12 And in one of the great public health success stories of all time, smallpox eradication was achieved through applying evidence-based approaches to ‘surveillance, isolation, case tracing and vaccination’.13
Obstacles facing policymakers
Despite these success stories, there are several obstacles that are getting in the way of global health policymakers being able to use evidence in their daily lives. These exist at three key steps in the ‘linear’ flow between research evidence and practical policymaking.
First, there is too little primary research that specifically examines the value of interventions among poor communities. The effectiveness of many interventions to improve health in poor populations in the developing world, say Buekens and colleagues, “remains untested and therefore unproven.”14 Second, systematic reviews of the evidence are often not relevant to the public health priorities facing policymakers in low-income settings.15 Third, the take-home messages from these reviews are often presented to policymakers in an overly complex, confusing manner; the result, says John Lavis, is that “policymakers hear noise instead of music.”16
The good news is that the evidence-based global health policy movement has taken many important strides towards tackling these obstacles over the last few years. Take, for example, the case of drug treatment for uncomplicated falciparum malaria in sub-Saharan Africa. The WHO recommends artemisinin-based combination therapies (ACTs) as first-line treatment for this disease, based on high-quality evidence.17 African policymakers who wish to support the national adoption and roll-out of ACTs will now find a wealth of RCTs that were conducted locally; over 30 reviews of the evidence, including reviews of quantitative and qualitative studies; and many policy briefs containing punchy messages on ACTs.18 A wide variety of short evidence-based summaries and briefs is now freely available addressing a broad range of global health topics, such as those produced by SUPPORT (http://www.support-collaboration.org), EVIPNET (http://www.who.int/rpc/evipnet) and E2Pi (http://www.e2pi.org).18
Although we must continue to improve this global flow of evidence – from RCTs to policy change – we should also realise that the idealised version of a linear flow is sometimes an inaccurate picture of what happens in the real world.19
The fallacy of linear policymaking
In the idealised version, the global health problem is first clearly defined. The research evidence is then used to provide the correct policy option to solve this problem.20 However, this ideal is rare in the real world. Even when the evidence in support of an intervention is high quality and has been disseminated to the relevant policymakers, uptake of the intervention may still not get prioritised. For example, there is strong evidence for the benefits of co-trimoxazole prophylaxis in people with HIV, and this evidence has been widely disseminated; yet policymakers in low-income countries have not pushed for its scale-up.19
Policymaking is a messy, non-linear process, in which evidence is only one of many inputs. Peter John characterises policymaking as a slippery interaction between ‘four Is’: institutions (structures, and the rules shaping decisions), interests (the individuals or groups who stand to gain or lose from a policy), incidents (external events that can shape policy, eg, the global economic recession) and ideas (the evidence).21 A realistic version of global health policymaking acknowledges that “the other legitimate influences on policy (social, electoral, ethical, cultural, and economic) must be accommodated.”20
Instead of being shocked that policymakers sometimes ignore evidence, we need to better understand the values, beliefs and processes that guide their decisions. Or, as Hutchinson and colleagues put it, instead of feeling frustrated that research results are being ignored, we need to “engage more explicitly with policy analysis approaches.”19
The fallacy that global health outcomes are easily ‘purchased’
A second fallacy, which we believe is widespread, is that health outcomes can be ‘purchased’ in a kind of global health shopping place: donors can simply ‘buy’ a set of specific health outcomes if they pay for a certain set of evidence-based interventions. For example, if a donor spends US$X million to pay for childhood vaccinations and ITNs for a developing country, this should surely translate into preventing Y number of childhood deaths.
We rarely see such an outcome in the real world. Dollars don't automatically buy outcomes as they have little influence on the complex, poorly understood apparatus of implementation, which is largely a product of health systems, politics and economic realities.
An example of how paying for evidence-based interventions did not lead to better health outcomes is Unicef's Accelerated Child Survival and Development programme, rolled out between 2001 and 2005 in 11 African countries.22 The US$27 million programme, mostly funded by the Canadian International Development Agency, aimed to accelerate reductions in childhood deaths by increasing coverage with three different evidence-based packages of interventions. Unfortunately, a retrospective impact evaluation found that the programme “did not accelerate child survival in Benin and Mali focus districts relative to comparison areas.”22
It is unclear what went wrong. The authors of the evaluation offer a variety of suggestions, such as a lack of supportive national policies to promote the programme, shortages of medicines and weaknesses in the programme's ‘community component’.22 However, the origins of these shortcomings remain murky. The evaluation wasn't able, says Stefan Peterson, to penetrate the ‘black box of implementation’.23 Our lack of understanding of this black box is a major barrier to the large-scale implementation of evidence-based global health tools.24
The fallacy that RCTs always reign supreme
The best way to assess a new biological or behavioural health intervention – or a package of such interventions – is through an RCT. Such trials can also be used to assess the health impacts of specific socioeconomic, environmental and educational interventions, as has been shown by research groups such as the Abdul Latif Jameel Poverty Action Lab (J-PAL) at the Massachusetts Institute of Technology. A recent RCT run by J-PAL, for example, found that, although a microfinance scheme in India helped women to expand their businesses, it had no impact on women's health or education outcomes.25
But in global public health, the following question is rarely posed: “Does this specific biological intervention work when studied in a highly controlled environment?” More often, the question is one that may not be amenable to an RCT: “How do we assess very complex and multifaceted, large-scale, real-world changes in global public health governance, institutional arrangements, financing or delivery?” To add yet more complexity, these changes are almost always occurring upon a backdrop of multiple, existing, overlapping public health programmes, funded and rolled out by governments, donors and private voluntary organisations. The result, say Victora and colleagues, is that “traditional research designs, which compare areas with and without a given programme, are no longer relevant at a time when many programmes are being scaled up in virtually every district in the world.”26
So while the RCT is rightly hailed as the ‘pinnacle’ of evidence-based medicine, in the global public health community, there is growing recognition that new research designs are desperately needed to help evaluate ‘real world’ programmes.26 27 Such designs would, we believe, also help to illuminate the implementation ‘black box’ described above.
Conclusion
What will it take for global health policies to become more evidence based, as shown in figure 1? We have argued that improving the evidence flow from RCT to policymaker is necessary but insufficient. We also need to better understand the sociopolitical and cultural influences upon policy development, the complex apparatus of large-scale implementation and the ‘real world’ impact of such implementation. And we are going to need better tools for the job than the RCT alone.
References
Footnotes
-
Competing interests Both authors have completed the Unified Competing Interests form at http://www.icmje.org/coi_disclosure.pdf and declare: (1) No financial support from any commercial entity for the submitted work. (2) RF is adviser to McKinsey and Company and chair of the health policy advisory board at Gilead Sciences; GY declares no financial relationships with commercial entities that might have an interest in the submitted work. (3) RF is married to Neelam Sekhri Feachem, the chief executive officer of Healthcare Redesign Group; GY has no spouse, partner or children with relationships with commercial entities that might have an interest in the submitted work. (4) RF has held senior positions at the World Bank and the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM), and he has advised several bilateral and multilateral development financing organisations. He directs the Global Health Group (GHG) and the Evidence to Policy Initiative, E2Pi; the GHG as well as E2Pi have received financial support from the Bill and Melinda Gates Foundation, and E2Pi has also received support from GFATM and the Clinton Health Access Initiative. He is a member of the scientific oversight group that advises the board of the Institute for Health Metrics and Evaluation. GY is the San Francisco Team Lead of E2Pi in the GHG.