A Collaborative Approach to Defining the Usefulness of Impact: Lessons From a Knowledge Exchange Project Involving Academics and Social Work Practitioners

Wilkinson, H., Gallagher, M., & Smith, M. (2012). A collaborative approach to defining the usefulness of impact: Lessons from a knowledge exchange project involving academics and social work practitioners. Evidence & Policy, 8(3), 311-327.
This paper reports on a knowledge exchange project involving academics and practitioners in six local authority social work departments. It contributes to recent debates about the coproduction of knowledge, presenting findings in three key areas: the importance of relationships for knowledge exchange; ‘what works’ for practitioners engaging with academics; and the value of practitioner research for enabling knowledge exchange. It also considers the assessment of ‘impact’ against the shifting landscape of the Research Excellence Framework, arguing that academic and performance management notions of impact and research quality risk eclipsing the perspectives of research users.
This paper presents the experiences of Scottish social work researchers and their social work partners from local authorities (=municipalities for Canadians). The 12 month project focused on six practitioner-led research projects to understand the drivers of collaboration and knowledge exchange and to understand the different concepts of impact as viewed by different participants. The researchers also developed guide books and held training sessions for research and knowledge exchange.
You can clearly read the influence of Sarah Morton, Co-Director, Knowledge Exchange, Centre for Research in Families and Relationships at U. Edinburgh as the authors use contribution analysis as one of their evaluation methods. Sarah has been a champion of Contribution Analysis, recently defending her thesis on the subject. Look to Sarah for some peer reviewed papers forthcoming from her dissertation.
The researchers build from the conclusion that collaboration for co-production is the most robust form of knowledge mobilization. The authors cite Sandra Nutley’s well-read book Using Evidence (2007), « personal contact … seems to be the most important route for research to enter policy and practice. This suggests that research use may above all be a social process, involving interaction among individuals and the joint (re)construction of research evidence through ongoing debate, interplay and exchange« .  However the authors also cite literature back to 1966 that speaks to the importance of tacit (practitioner derived) knowledge and communities of practice indicating that the creation and use of knowledge in social work has always been a social process.
Like much within the knowledge mobilization literature, these are not new ideas.
The paper focuses on the role of partnerships as being « the most important factor determining whether or not research evidence is used by decision-makers… It has been suggested that such co-production ensures that research benefits from the local knowledge of practitioners, generates more practice-relevant research and enables academics to improve their ability to communicate to wider audiences« .  Co-production happens throughout the research cycle from concept to planning to execution to evaluation to dissemination. In our work at York University’s Knowledge Mobilization Unit we practice a variety of methods including producer push, user pull and knowledge exchange but they are all designed to drive towards a co-production method.  For more on York’s services please see a 2011 paper that describes our tools and methods.
Interestingly the authors tried to support the face to face co-productions with on line tools. They soon abandoned the wiki they produced and even abandoned e mails. This is a minor point in the paper but one worth reflecting upon. Collaborative on line tools are becoming increasingly used in knowledge mobilization projects and practices. They work well in some situations and not well in others. This is an area where more research is needed to understand the drivers, barriers and dynamics of on line mediated knowledge mobilization.
The authors present a number of success factors for successful practitioner led research projects:

  • Ensure that project activities are relevant to the parties involved
  • Put a structure in place
  • Be flexible around practitioner needs and interests
  • Provide mentoring from university partners (however, I also suggest that the community/practitioner partners mentor the university partners)
  • Consult with practice partners from the outset
  • Make time for discussion at high quality events

The paper concludes with reflection on impact and how this is interpreted differently in university and practice settings. This is particularly relevant for the UK that is about to undergo a Research Evaluation Framework exercise in 2014. The REF will drive part of a university’s funding. In previous Research Assessment Exercises universities had to produce evidence about the quality of their scholarship. In the REF 2014 they will also have to produce evidence of the impact of their scholarship. « Assessment of impact will be based on expert review of case studies submitted by higher education institutions to the REF sub-panels whose composition will include research ‘users’. This is a key development for research evaluation in the UK. It builds on work piloted by Australia with their Research Quality Framework. Universities have developed units for research impact and there is an entire REF related industry springing up as universities seek to articulate the extra-academic impacts of their research. This effort is one of the drivers of knowledge mobilization and related activities in the UK. If impact is what we are trying to achieve then knowledge mobilization is one method for universities to achieve this impact. Knowledge mobilization (the « how ») enables impact (the « what »).
Key Points for discussion: There are three items for further discussion and reflection:
1.    A key take home message here is that knowledge mobilization is more than dissemination on steroids. So often I see knowledge mobilization strategies focused on end of project dissemination. This is important. It is necessary. But it is not stuffiest to effectively inform decisions about practice or policy. Integrated knowledge mobilization is messier and harder to do than enhanced dissemination. How do you encourage project teams to practice integrated knowledge mobilization throughout the research to research use cycle?
2.    These aren’t new ideas. If the ideas presented in this paper are new ways of validating what we already know then what is new in knowledge mobilization research?
3.    It’s time that we practice what we preach. The authors state that « the participating local authorities were involved in the development and implementation of the project from the earliest stages of planning. » So why aren’t they co-authors of the paper? This is another example of academics producing research that speaks of the need for engagement of their non-academic partners throughout the research process yet they don’t publish with their partners. I made this comment in an earlier Knowledge Mobilization Journal Club entry. It’s time that academics practice what they preach.
4.    One last thing, important for all brokers to know.  Read Sandra Nutley’s work.  Sandra Nutley leads the Research Unit for Research Utilization at the University of Edinburgh. She is a leading scholar in research utilization especially in social services and social policy. Her book, Using Evidence, from 2007 is still a leading resource for our work.  Using Evidence is a must read (really, a MUST read) for all knowledge brokers.
RIR is producing this journal club series as a way to make the evidence on KMb more accessible to knowledge brokers and to create on line discussion about research on knowledge mobilization. It is designed for knowledge brokers and other knowledge mobilization stakeholders. Read the article. If you’re a community member seek a colleague at your local university to obtain this article for you. Then come back to this post and join the journal club by posting your comments.