Lavis, J. N. (2009). How can we support the use of systematic reviews in policymaking? PLoS Med, 6(11), e1000141. doi:10.1371/journal.pmed.1000141 http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.1000141
Abstract
- Policymakers need many types of research evidence-synthesized and packaged for them-and the use of this evidence supported in multiple complementary ways. Stakeholders who seek to influence the policymaking process have the same requirements.
- Policymakers and stakeholders need many types of systematic reviews. For example, reviews of qualitative studies can help to identify alternative framings of the problem, to understand how or why a policy or program option works, and to appreciate stakeholders’ perspectives on particular options.
- Policymakers and stakeholders now have access to many review-derived products: (1) summaries of systematic reviews highlighting decision-relevant information; (2) overviews of systematic reviews providing a “map” of the policy questions addressed by systematic reviews and the insights derived from them; and (3) policy briefs drawing on many systematic reviews to characterize a problem, policy or program options to address the problem, and implementation strategies.
- A range of activities are being undertaken to support the use of reviews and review-derived products in policymaking, all of which warrant rigorous evaluation.
- Future challenges include: (1) examining whether and when any apparent duplication of efforts occurs in the production of review-derived products at the international level; and (2) scaling up activities that are found to be effective in supporting the use of reviews and review-derived products in policymaking.
This paper instructs on how systematic reviews can be used by policy makers and those seeking to influence/inform policy makers (John Lavis calls these “stakeholders”, I prefer the term policy “advocates”). This paper is useful for two reasons: 1) the methods of engagement apply beyond just systematic reviews; and, 2) it brings in the concept of policy stakeholder/advocate – an important role as we seek to inform decision makers.
The paper starts from the premise that we now have enough systematic reviews in many cases. They have become regular tools in the policy making process. “Now it isn’t uncommon for these groups to find dozens of systematic reviews that address the governance, financial, and delivery arrangements within health systems that can determine whether a cost-effective program, service, or drug reaches those who need it.”…”[T]he challenge in using research evidence has shifted from making the best possible use of local studies to: (1) finding systematic reviews that address their many questions related to the policy issue at hand; (2) deriving insights from the reviews for a particular context (which may differ from where the studies included in the review were conducted); and (3) combining these insights with the insights from local data and studies and from local tacit (“how to”) knowledge and other forms of knowledge”.
Right off the top this parallels the PARIHS Framework that says you need to pay attention to three elements to maximize the implementation of research evidence on policy/practice: 1) evidence (in this case find the right systematic review); 2) facilitation (in this case derive insights from the reviews; and, 3) context (in this case “for a particular context” and “combining with local data and studies and from local tacit knowledge“). I often remark on seeing the PARIHS Framework in many seemingly unrelated studies in the broad field of knowledge mobilization.
The original premise for this paper might be correct for health policy but may not hold for social policy. A scan of all subject areas on the Cochrane Collaboration Library produced a total of 10,586 reviews across 33 topics. By comparison searching all five topics (Crime & Justice, Education, International Development, Social Welfare and Methods) in the Campbell Collaboration Library produced only 100 reviews published between their earliest date (2003) and 2014. There are other sources of systematic reviews such as What Works Clearing House which covers education so social policy makers will have a more difficult time hitting Lavis’ #1 opportunity: find the right systematic review. What Works Clearing House has summarized 10,573 studies; however, these are single studies not systematic reviews and their review is merely a single sentence indicating if the research paper meets the criteria for utility by the What Works Clearing House criteria.
One thing I learned from this paper is that there are at least three different types of products derived from systematic reviews:
- summaries of systematic reviews that provide decision related information
- overviews of systematic reviews that provide a map (i.e. the breadth) of what policy questions have been addressed
- policy briefs informed by systematic reviews “which start with a policy issue, not with the reviews that researchers happen to have produced.“
With all this work in producing products from systematic reviews John Lavis cautions about the duplication of effort at the international level if many health policy actors are producing products from reviews and how to ensure those products are updated as the reviews are updated.
John Lavis also questions “whether the briefs are seen as an end in themselves or as an input to one or more “deliberative dialogues.” Such dialogues typically involve convening one to two dozen policymakers and stakeholders to work through a policy issue, drawing on both the policy brief and their own and others’ tacit knowledge about the issue.” For more on deliberative dialogues see the journal club post fromSeptember 2014.
The paper then presents three activities from systematic reviews that can support the use of systematic reviews which, as a knowledge mobilization practitioner, I found the most useful part of this paper.
- Interactions between researchers and policymakers increased (and a lack of interactions decreased) the prospects for using research evidence, particularly when the interactions were based on informal relationships;
- Timeliness increased (and a lack of timeliness decreased) the prospects for research use; and
- Accordance between research evidence and the beliefs, values, interests or political goals, and strategies of policymakers and stakeholders (or when political positions had not yet been taken) increased (and discordance decreased) the prospects for using research evidence.
Commenting on these John Lavis indicates that the “importance of interactions underpins efforts by some organizations to engage both researchers and policymakers in priority-setting and/or production activities [this is a common goal of knowledge brokering] and in deliberative dialogues. The importance of timeliness underpins efforts to create and continuously update databases that provide “one stop shopping” for optimally packaged reviews and review-derived products… The importance of an accordance between research evidence and policymakers’ beliefs, values, interests or political goals, and strategies underpins efforts to identify “windows of opportunity” in policymaking processes” [this again speaks to implementation in context from the PARIHS Framework].
However when compared against the activities presented in table 3 there is only one that I can see maps onto one of these three. From Table 3 “packaging reviews and review derived products” maps onto “timeliness” from the systematic reviews. It’s not like the other activities in Table 3 are not likely to enhance use of systematic reviews just that there is an incongruence between those presented in Table 3 presumably derived from the references provided and John Lavis’ own practice and those derived from systematic reviews.
Questions for Brokers
- How much do you use systematic reviews in your work? While many agree that the gold standard of research evidence is a systematic review does this negate efforts to broker other forms of co-production and research use?
- Reflect on your knowledge brokering processes. Do these map onto the PARIHS Framework: evidence, facilitation, context?
- There’s a lot packed into the first activity: Interactions between researchers and policymakers increased (and a lack of interactions decreased) the prospects for using research evidence, particularly when the interactions were based on informal relationships. What methods are you using to enhance interactions between researchers and policy makers/policy advocates and what constitutes an informal vs. formal relationship?
ResearchImpact-RéseauImpactRecherche is producing this journal club series as a way to make the evidence and research on knowledge mobilization more accessible to knowledge brokers and to create on line discussion about research on knowledge mobilization. It is designed for knowledge brokers and other knowledge mobilization stakeholders. Read the article. Then come back to this post and join the journal club by posting your comments.