Flows of Knowledge, Expertise and Influence: A Method for Assessing Policy and Practice Impacts From Social Science Research

Meagher, L., Lyall, C., & Nutley, S. (2008). Flows of knowledge, expertise and influence: A method for assessing policy and practice impacts from social science research. Research Evaluation, 17(3), 163-173. DOI: 10.3152/095820208X331720
Social science research undoubtedly does impact on public policy and practice but such non-academic impacts are rarely amenable to precise, quantitative metrics. In the interests of accountability, it is however possible to find proxy indicators of connectivity with research users and these may form steps toward impacts. Understanding these connections can lead to a deeper appreciation of the factors that shape the processes leading to research uptake. This study adopted a detailed and largely qualitative approach to identify the flows of knowledge, expertise and influence that take place during the process of knowledge transfer in order to trial a method for assessing policy and practice impacts from social science research. As a corollary to this assessment, the study further identified five factors that can influence and enhance the process of knowledge exchange between researchers and users.
The paper builds from an earlier paper by Meagher and Lyall in 2007. In that paper the authors identified that a « small majority (57%) of award-holders [ie grant recipients from the Economic and Social Research Council] who responded to the survey considered that their research had had an impact beyond the research community, although they found this difficult to document precisely. » Two observations here: 1) It therefore follows that 43% of respondents were unable to articulate non-academic impacts of their publicly funded research; and 2) The authors asked the researchers to articulate their non-academic impact. Impact is more appropriately measured at the level of the user not the producer. If they had asked downstream research users (policy makers, practitioners) to point to impacts derived from the ESRC funded research the results would likely have been lower.
The authors critique the use of the term knowledge transfer but continue to use it without implying linearity of the concept. Knowledge transfer is a general term used in the UK to support the economic impact of research « with ‘economic’ defined broadly enough to include policy, practice and other dimensions of importance to society« . While the official definition of KT includes policy, in reality KT in the UK is synonymous with technology transfer in Canada. More recently knowledge exchange is being used by the Centre for Research in Families and Relationships at University of Edinburgh.
What did they do? The authors surveyed 134 ESRC grant recipients as well as research users and knowledge intermediaries associated with these grants and used a range of qualitative research methods to investigate examples of impacts and processes by which these impacts were achieved. Kudos for interviewing end users and knowledge intermediaries. They were « able to draw some lessons about the processes that seemed to accelerate research uptake. While these lessons are certainly not definitive, [they]identified five factors that could have a particular influence on processes likely to lead to non-academic research impacts. »
These five factors are:

  1. The value placed upon/incentives provided for generation of impact
  2. Supporting two-way interactions between researchers and users
  3. Injections of financial support, dedicated staff, infrastructure
  4. Facilitating role(s) of knowledge intermediaries
  5. Communication/increasing accessibility of research

Key Points for discussion: There are two items for further discussion and reflection:

  1. Review these five factors. As you build your KMb practice understand how these factors can influence the success of your KMb services.
    1. Does your institution value and create incentives for non-academic impact? Most universities tenure & promotion policies are seen as a barrier to this; however, there is a movement in Canada, led by University of Guelph and Campus-Community Partnerships for Health, to examine incentives and rewards for community engaged scholarship – check out www.engagedscholarship.ca.
    2. Do you have systems and services that support the two way (or multidirectional) interaction of researchers and users? Do you practice knowledge exchange and knowledge mobilization instead of unidirectional knowledge transfer?
    3. Do you fund dedicated staff and an infrastructure including tools (such as the ResearchImpact-RéseauImpactRecherche tool kit in English and French or York’s clear language research summary tool kit) to support the interaction of researchers and users?
    4. Do you support dedicated knowledge intermediary staff? While many individuals are involved in producing, connecting and using knowledge, there is a role for professional, dedicated knowledge brokers. Knowledge mobilization should not be practiced « off the side of someone’s desk ».  Make it someone’s dedicated job.
    5. Does your KMb service support the increased accessibility of research? Do you have a program of writing research summaries in accessible language?  Check out York’s ResearchSnapshot clear language research summaries that have been adopted by the Institute for Community Engaged Scholarship at the University of Guelph, Ontario Mental Health and Addictions Knowledge Exchange Network at the Centre for Addictions and Mental Health and the Knowledge Network for Applied Education and Research.
  2. About Figure 1 in the paper, the conceptual framework for research impact assessment. This figure represents researchers and policy makers/practitioners linked by arrows which represent interactions between them. This hearkens back to the old « two communities » approach of Caplan [Caplan, N. (1977). The use of social research knowledge at the national level. In Social Research in Public Policymaking, 183-197] where research and policy/practice are seen to be working in two separate communities. « However, it is our experience that policy and academic communities do collaborate and co-create useful knowledge, working in a shared space alongside others from a variety of settings.  This more optimistic view acknowledges that there are multiple players working to produce, use and channel research.  The concepts of knowledge transfer and bridging the gap between the two communities are less applicable than the language of knowledge exchange or mobilization, implying this more interactive approach » (this excerpt is taken from a paper written with Sarah Morton, CRFR). Recently I have seen Sandra Nutley include co-creation in her work acknowledging how the concepts have evolved since this paper in 2008. Take some time to re-draw Figure 1 showing researchers and policy makers/practitioners sharing some collaborative spaces but also working in interactive processes as depicted in Figure 1. In reality co-creation and knowledge exchange are often practiced in any one setting. This is not an easy model to draw.

Go ahead.  If you get a comprehensive diagram that is also elegant please let me know
One last thing, important for all brokers to know.  Read Sandra Nutley’s work.  Sandra Nutley leads the Research Unit for Research Utilization at the University of Edinburgh. She is a leading scholar in research utilization especially in social services and social policy. Her book, Using Evidence, from 2007 is still a leading resource for our work.  Using Evidence is a must read (really, a MUST read) for all knowledge brokers.
RIR is producing this journal club series as a way to make the evidence on KMb more accessible to knowledge brokers and to create on line discussion about research on knowledge mobilization. It is designed for knowledge brokers and other knowledge mobilization stakeholders. Unfortunately this article isn’t available in an open access format. If you’re a community member seek a colleague at your local university to obtain this article for you. Read the article. Then come back to this post and join the journal club by posting your comments.