Morrow, S. (2016). The Impact of Higher Education Leadership, Management and Governance Research: Mining the 2014 Research Excellence Framework Impact Case Studies. Report from Leadership Foundation for Higher Education.
Elizabeth Morrow is an independent researcher who produced this report for the Leadership Foundation for Higher Education. It is an exhaustive review (121 pages) of 1,309 Research Excellence Framework (REF) impact case studies from legal, governance and management (LGM) research. This report is important not for LGM research (well it is…but that’s not why I find it important) but because it is a different look at the REF. The REF is the only system wide (every UK university participated) research impact assessment process in the world so we can learn a lot about the opportunities and challenges to research impact assessment by studying the REF.
Why should knowledge mobilizers care about the REF? We care because impact is what we want to achieve and knowledge mobilization is one way to maximize impact. When evaluating our work impact is the dependent variable and knowledge mobilization is the independent variable (see more on this here). So we should care about the REF if we want to measure the effects of our knowledge mobilization efforts.
This is such a big report it is impossible for me to pull out all the good and interesting sections so I will use the limited space of this journal club to present the Adaptive Systems Framework for Advancing Research (AS-FAR) model. AS-FAR appears on page 86 of the report and is summarized on this HEFCE blog post. Click that link now and take a look at the model.
I wrote to Elizabeth Morrow. I thanked her and observed the report and hence AS-FAR are derived from retrospective cases and I struggled to try to think about applying AS-FAR in a prospective manner despite the report claiming this was possible. My world is mostly about helping researchers and partners collaborate to co-produce research that has both academic merit and is useful for the partners who create the impact. I struggled to work through AS-FAR in this prospective, co-production fashion. I found AS-FAR was more easily understood retrospectively collecting evidence and creating the story of impact using transfer and exchange mechanisms.
I frequently critique academic frameworks since they describe a gold standard process or model yet I work in a world that never hits a gold standard. I find many frameworks describe what should be and I work in messier realities of what is. I struggle with thinking how I would implement AS-FAR sitting across from a research partnership helping them develop their impact strategy for their grant application.
Elizabeth responded acknowledging the complexity and indicated her current project is developing a Research Leader’s Impact Toolkit focusing more on what research leaders can do to support researchers and future impact. We need this tool kit because AS-FAR as a model is challenging. We need tools to cut through complexity, working with it but not letting it become a barrier. As practitioners we accept imperfection and work with it to get to impact. I am not certain AS-FAR helps but the tools Elizabeth is developing might.
Stay tuned!
Questions for brokers:
1. You’re probably deep in the trenches helping to create impact. How do you assess it? Do you have a particular framework that guides you (and don’t say the Knowledge To Action Framework for all the reasons here).
2. Would REF work in Canada? We better figure this out because my bet is Canadian universities will be held accountable for impact at some point.
3. Do we have enough frameworks? Don’t we need more practical guidance on their use (and some tools would be nice)?
ResearchImpact-RéseauImpactRecherche is producing this journal club series as a way to make the evidence and research on knowledge mobilization more accessible to knowledge brokers and to create on line discussion about research on knowledge mobilization. It is designed for knowledge brokers and other knowledge mobilization stakeholders. Read the article. Then come back to this post and join the journal club by posting your comments.