To Know is Not Enough: Research Knowledge and Its Use

Levin, B. (2013). To know is not enough: Research knowledge and its use. Review of Education, 1(1), 2-31. DOI: 10.1002/rev3.3001 http://onlinelibrary.wiley.com/doi/10.1002/rev3.3001/pdf
Abstract
This paper is about the relationship between research, policy and practice in education. It outlines reasons for the increased interest in research and its impact and describe some of the difficulties in studying this relationship. A conceptualization of the knowledge mobilization process is presented that identifies three overlapping and interacting domains – the production of research, the end use of research, and the intermediary processes that link these two. The paper reviews current research and develops ideas about all three of these aspects, identifying areas of understanding and gaps in current knowledge.
This journal club previously reviewed a paper from Ben Levin and you can read more about the work of his team on the knowledge mobilization blog, Mobilize This! where we covered a study recently released by Amanda Cooper (now Faculty of Education, Queen’s University).
In this article Ben brings together some of the evidence (or lack thereof) about connecting research to policy and practice in education. He also introduces some thinking about the role of the internet and social media as tools for knowledge mobilization. He writes, « a conceptualization of the knowledge mobilization process is presented that identifies three overlapping and interacting domains – the production of research, the end use of research, and the intermediary processes that link these two. » It’s another framework designed to help us think through the elements of successful knowledge mobilization. While I have previously lamented about the proliferation of frameworks and the paucity of empirical research to critically assess these frameworks, Ben agrees with me. In the article he comments about frameworks and « the irony has been noted more than once that the debate over the use of research is itself not well informed by research« . In other words (also from Ben), « where is the evidence that evidence works? »
Ben reviews some of the drivers of an increased interest in knowledge mobilization. These include a growing interest on behalf of the academic to provide research to decision makers. As well, research funders and Foundations are putting increasing emphasis on articulating the impacts of research. The Connections theme is one of three streams of activities funded by the Social Sciences and Humanities Research Council. Every grant application, even a modest $25,000 meeting grant, must have a statement of the expected outcomes of the funded activity (the impact) and how the applicant will achieve that impact (the knowledge mobilization strategy). This focus on impact is also pursued by the UK’s Economic and Social Research Council and the evaluation of research impact in the UK is being driven by the Research Excellence Framework (REF 2014).
However, despite this increased interest in knowledge mobilization there are challenges to researching and practicing it:

• Differing understanding of what is « research » and what is « use »
• Debates around methodological rigour including quantitative vs. qualitative research
• The different ways in which research can be used: instrumental, conceptual, symbolic/political
• Methods of evaluating the impact of research and research use which is predicated upon surveys and self-assessments
• The role of organizations and the difficulty of assessing organizational as opposed to individual factors

Ben recommends « that more research is needed on questions of how to study the impact of ideas on practices. » Of course…that’s the main recommendation of every review ever undertaken. Research is, after all, a self-perpetuating activity. But he is right. As we begin to invest in knowledge mobilization practice we do need more research to inform our practices. We need knowledge mobilization on knowledge mobilization.
This section reads to me like it is predicated on a knowledge transfer/translation paradigm. Likely much of the literature Ben reviews is studying earlier linear and unilateral approaches to knowledge mobilization. Many of these challenges may be minimized in co-production and/or facilitated knowledge mobilization paradigms. When researchers and decision makers collaborate they develop a mutual understanding of acceptable methodology, research and use is a given and the individuals concerned accept their roles in their own organizations.
Ben presents a useful conceptualization of these concepts identifying 3 overlapping contexts. The first is the production of education research-often but certainly not only done in universities. The second is the ‘use’ context-a policy or practice setting such as a school system or government where research findings can be applied. The third is the mediation context-all those individuals and organizations who attempt, in one way or another, to connect research to practice, such as think tanks and lobby groups, the media, professional associations and entrepreneurs.
To recap the three concepts presented by Ben are:

1. Production of education research
2. The « use » context
3. Mediation

This lines up exactly with the three core elements of thePARIHS Framework:

1. Evidence
2. Context
3. Facilitation

Ben’s work in education research lines up exactly with the PARIHS Framework which was developed in health research settings. When two independent lines of inquiry converge there is likely an underlying truth. To maximize the impact of research on policy and practice you need to pay attention to: 1) the production of research evidence; 2) the context in which this evidence is implemented; and, 3) the facilitation/mediation that brokers between these two.
Ben also discusses impacts of research upon policy vs. practice.« Although in one sense, policy can be viewed as a kind of practice, it is important to note that changing policy, while difficult, is in many respects less difficult than changing practice. Typically, a change in policy can be made by a relatively small number of people, but practice in a large system such as education involves changing the behavior of very large numbers of people spread across many organizations. A government may alter a policy in some area of education, but the policy change may or may not affect what thousands of teachers and students actually do. »
Ben then discusses the impact of the internet and social media on knowledge mobilization pointing out that « the impact of the internet and related technologies on research communication cannot easily be overstated…Today, the internet and its various vehicles are by far the dominant way in which research information is shared around the world, and by far the most common way in which people look for information, including information derived from research. Even academic researchers now begin with internet searches not library searches – and library collections are also available online. »  However, Ben’s own research shows that we are not using the internet as a tool to maximize the impact of knowledge mobilization. He shows « that most research pages get few visitors and most research resources get few hits and even fewer downloads…Our conclusion is that the full potential of the internet for sharing research is not well used, and that organizations are placing too much emphasis on websites as places to store research materials for use by interested persons, as opposed to taking active measures to bring research to people who might have an interest in it or, even more, to build ongoing connections among research users. »
The question then becomes not one of merely posting research and evidence on a web site but connecting that research and expertise with research users. Ben provides some discussion of the role of social media to support connecting on line but acknowledges that our formal understanding of these systems is lacking.
Ben then discusses efforts to make these connections and observes that unlike technology transfer and commercialization, their research shows that few academic institutions are making similar efforts in knowledge mobilization saying that researchers are largely left to their own devices. He’s right…with a great big BUT…
The ResearchImpact-RéseauImpactRecherche (RIR) universities are making this investment. His own website analysis research published by Amanda Cooper show that the RIR universities received the highest score for the lowest budget of all 44 knowledge intermediary organizations studied in Canada. Yet no mention of this innovation in institutional knowledge mobilization…rant over…
Amanda and Ben’s research also shows that activities provided by these 44 knowledge intermediary organizations including the RIR universities falls into eight categories

1. facilitating linkages and partnerships across diverse stakeholder groups
2. increasing awareness of research findings
3. increasing the accessibility of research by tailoring formats
4. increasing engagement with research in different modes
5. influencing policy
6. building system capacity to use research
7. implementation support
8. organizational development.

Questions for brokers:

1. Much of our work as knowledge brokers is on a small scale. We seek to influence practice within a single organization. If we move our thinking to systems such as schools or hospitals we run into regulated industries and unions of teachers and nurses etc. How might unions and regulatory/licensing bodies help or hinder evidence informed change?
2. Read what Ben writes about the very important, but poorly reported, potential of graduate students to act as knowledge brokers. In technology transfer we always say we transfer more technology through graduate students being hired by technology companies than we ever will by patenting research. Might the same be true for graduate students in education, social work, nursing, management etc?
3. Check your knowledge mobilization practice against the eight categories of support identified above. Are there any gaps that might need to be addressed? If you were to prioritize these in what order would you place them for your practice?
4. The internet: are you posting content and hoping your audience will find it? How are you pulling your audience to your content? More importantly, how are you placing your content before your audience? How are you using social media to enable these connections?

ResearchImpact-RéseauImpactRecherche (RIR) is producing this journal club series as a way to make evidence on KMb more accessible to knowledge brokers and to create on line discussion about research on knowledge mobilization. It is designed for knowledge brokers and other knowledge mobilization stakeholders. Read this open access article. Then come back to this post and join the journal club by posting your comments.