Oliver, K., Innvar, S., Lorenc, T., Woodman, J., & Thomas, J. (2014). A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Services Research, 14:2. doi:10.1186/1472-6963-14-2. http://www.biomedcentral.com/1472-6963/14/2
Abstract
Background: The gap between research and practice or policy is often described as a problem. To identify new barriers of and facilitators to the use of evidence by policymakers, and assess the state of research in this area, we updated a systematic review.
Methods: Systematic review. We searched online databases including Medline, Embase, SocSci Abstracts, CDS, DARE, Psychlit, Cochrane Library, NHSEED, HTA, PAIS, IBSS (Search dates: July 2000 – September 2012). Studies were included if they were primary research or systematic reviews about factors affecting the use of evidence in policy. Studies were coded to extract data on methods, topic, focus, results and population.
Results: 145 new studies were identified, of which over half were published after 2010. Thirteen systematic reviews were included. Compared with the original review, a much wider range of policy topics was found. Although still primarily in the health field, studies were also drawn from criminal justice, traffic policy, drug policy, and partnership working. The most frequently reported barriers to evidence uptake were poor access to good quality relevant research, and lack of timely research output. The most frequently reported facilitators were collaboration between researchers and policymakers, and improved relationships and skills. There is an increasing amount of research into new models of knowledge transfer, and evaluations of interventions such as knowledge brokerage.
Conclusions: Timely access to good quality and relevant research evidence, collaborations with policymakers and relationship- and skills-building with policymakers are reported to be the most important factors in influencing the use of evidence. Although investigations into the use of evidence have spread beyond the health field and into more countries, the main barriers and facilitators remained the same as in the earlier review. Few studies provide clear definitions of policy, evidence or policymaker. Nor are empirical data about policy processes or implementation of policy widely available. It is therefore difficult to describe the role of evidence and other factors influencing policy. Future research and policy priorities should aim to illuminate these concepts and processes, target the factors identified in this review, and consider new methods of overcoming the barriers described.
This is an update to an important article:
Innvaer, S., Vist, G., Trommald, M. & Oxman, A (2002) Health policymakers’ perceptions of their use of evidence: A systematic review. J Health Serv Res Policy, 7, 239-244.
Innvaer (close but not relation to co-author of this paper, Innvar) in 2002 did a systematic review looking at health policy makers’ perceptions of their use of evidence. Many articles that look at the use of evidence in policy making look to the researchers’ side of the equation. This is still cited as a problem in this 2014 article. Both Innvaer 2002 and this 2014 article asked the question form the policy makers’ perspective. This is an important lesson for all knowledge brokers: are you looking at the needs of your research evidence providers as well as your evidence consuming receptors? And the wise among us will note that these two are inextricably linked when working in a co-production paradigm (more on collaborations later).
So it is not sufficient to quote Joni Mitchell and say “I’ve looked at life from both sides now” but it is more accurate to try to understand someone else by walking a mile in their shoes (or at least including them in your research).
Innvaer (2002) included 24 studies in his systematic review, all from health and mainly from OECD countries. Oliver (2014) included 145 studies, mainly from health but included a number of other policy contexts and at least 1/3 were from low and middle income countries. This change is testament to the growing interest in the linkages between research and policy as well as the growing breadth and depth of scholarship in this area.
In 2014 the most frequently cited barriers were:
- Lack of availability to research
- Lack of relevant research
- Lack of time/opportunity to use research
- Lack of policy maker skills
- Cost of using evidence (which may be related to lack of skills and time and what would be interesting to try to understand is the cost of not using evidence to inform research)
In 2014 the most frequently reported facilitators included:
- Access to and improved dissemination of research
- Existence of relevant research (Side note: of course, the top two facilitators are the same as the top two barriers!)
- Collaborations between policy makers and research staff
- Leadership and authority
Important to the use of evidence was clarity, relevance and reliability or research findings.
“Contact, collaboration and relationship are a major facilitator of evidence use, reported in over 2/3 of all studies“. Related to this “timing and opportunity were the most prominent barriers“.
The article suggests that this contact is often serendipitous as some studies discussed the role of informal, unplanned contact. This means that our role as knowledge brokers is to work with policy makers to create the right opportunities for engaging in contact with researchers so that collaborations and relationships are able to form.
If we create the right opportunities will policy makers create the time?
“Emerging as a new stream of research, eleven studies evaluated or described knowledge broker roles or related concepts with dedicated dissemination strategies evaluated in 7 studies and mentioned as a facilitator in 43“. This is an interesting emergence since Innvaer (2002) and suggests an emerging role for knowledge brokering that includes dissemination of research.
One of the conclusions of this systematic review is “research into how to alleviate organizational and resource barriers effectively would be welcomed“. Why is one conclusion of every systematic review the need for more research? I suspect it is because researchers write systematic reviews and researchers always need to justify their next research project. Would it be different if policy makers were the ones to conduct the systematic review? Would policy makers conclude that the evidence is sufficient to inform decisions?
Major bitch here: all the authors are university based. While this systematic review specifically looked at the policy makers’ side of the evidence equation, no policy makers were invited to be co-authors. The paper specifically indicates that “senior academics” were involved in the study design but by implication policy makers weren’t. The authors cite collaboration and relationships as a key enabler of evidence use. Yet this is another example of academic researchers failing to heed their own advice. This journal club has previously reported this with respect to community organizations and social work.
Questions for brokers:
- How do you create an environment where you maximize the chance that serendipity happens? Do you support events such as the Ontario Education Research Symposia that annually brings together education researchers, practitioners, knowledge brokers and policy makers?
- Dissemination strategies are important but is dissemination an end point? We all know that making evidence accessible is necessary but not sufficient to inform behaviour change. How are you using dissemination as a tool to introduce the right researcher to the right policy maker?
- How do you make your researchers available for expert advice? And when they are accessible what’s the difference between researchers doing consulting and knowledge mobilization?
- And on this issue of the major bitch…how are you engaging end users as well as researchers in the design of your knowledge brokering work? How are you ensuring that the stakeholders that you engage are engaged in the design of your work?
ResearchImpact-RéseauImpactRecherche is producing this journal club series as a way to make the evidence and research on knowledge mobilization more accessible to knowledge brokers and to create on line discussion about research on knowledge mobilization. It is designed for knowledge brokers and other knowledge mobilization stakeholders. Read the articles. Then come back to this post and join the journal club by posting your comments.