Watching Impact in the REF and How It Informs the Canadian Context / Le REF en observation : comment l’impact s’y manifeste, et son influence sur la situation canadienne

The Research Excellence Framework is a system wide research assessment exercise that includes assessment of the various non-academic impacts of research. As the UK prepares for REF 2021 Research Impact Canada is piloting impact assessment in Canada. Not because of any reporting requirement but because we should understand and communicate the impacts we are making. It’s the right thing to do.

Au Royaume-Uni, le Research Excellence Framework est un exercice d’évaluation de la recherche appliqué à l’ensemble du système d’enseignement supérieur, qui prévoit l’évaluation des nombreux impacts de la recherche en dehors de l’université. Tandis que ce pays prépare son REF de 2021, au Canada, le Réseau Impact Recherche réalise son propre projet pilote d’évaluation de l’impact. Non pas parce qu’une autorité quelconque nous l’impose, mais parce que comprendre et communiquer les effets que nous provoquons… c’est ce qu’il faut faire, tout simplement.

For 10 years Research Impact Canada (RIC) has been leading the development of institutional knowledge mobilization practices that create the conditions to maximize the social, economic and/or environmental impacts of university research. Our vision statement is:

We will maximize the impact of university research for the social, cultural, economic, environmental, and health benefits across local and global communities.

If we say we will maximize impact we needed to figure out a way to assess the impact of the research we were helping to mobilize. We looked to the UK Research Excellence Framework for inspiration (for more on REF and why it is important see this recent journal club entry). The REF required all UK universities to articulate the impacts of research using guidelines (page 26 here) and completing an impact case study template. There is much (not all) good about the REF. But there is much in the work of RIC that is not captured in the narrowly construed REF definitions of research and of impact. There is also a decoupling of the efforts made by institutions (as reported in the environment data) to support impact and the impact cases themselves.

The Evaluation Committee of Research Impact Canada did a deep dive into the REF and developed our own adaptation of the REF impact assessment guidelines and case study template. The major changes are summarized in the table below:

Changes between REF and RIC impact assessment table

View this table as a PDF

That’s what we have done. What are we doing?

We are piloting the RIC research impact assessment guidelines and impact case study template on one example from York’s Knowledge Mobilization Unit. We have also delivered our first impact assessment workshop to scientists and knowledge brokers at the research-policy interface at Eawag, the Swiss water research institute. We received good feedback from them and will be incorporating this into successive iterations of the guidelines and template. Ultimately we will roll this out through RIC member universities and beyond to provide a tool for researchers and institutions to collect the evidence of impact and inform different means of disseminating stories of impact so our various stakeholders (funders, partners, governments, and the public) can see the difference that universities make on society, the economy and/or the environment.

Our work in Canada is timely as HEFCE reports it has just finished consultations on REF 2021 and are about to review and analyze over 370 responses. We can continue to learn from each other. The Canadian and UK contexts are different. The main difference is the driver. We don’t have a REF in Canada so we have greater leeway to construct research impact assessment tools that work in our contexts. But our contexts are also not really that different. UK and Canadian funders require grant applicants to express the potential impacts of their research and the plans (and budgets) for creating those impacts. Now Canada also has a mechanism to facilitate the collection and reporting on the evidence of impact that was inspired by the REF but adapted to meet the needs of the Research Impact Canada network.

Give us a call, HEFCE. We’re happy to share as you pour through those 370 responses!

3 Responses to “Watching Impact in the REF and How It Informs the Canadian Context / Le REF en observation : comment l’impact s’y manifeste, et son influence sur la situation canadienne”

written by Wade Kelly On 17 April 2017 Reply

Hopefully RIC has an eye on what’s happening in Australia as well. ERA (Excellence in Research for Australia) 2018 will incorporate an engagement and impact assessment. That assessment is currently being piloted. http://www.arc.gov.au/engagement-and-impact-assessment

written by David Phipps On 20 April 2017 Reply

Thanks for the comment. Tamike Heiden (@KTAustralia) visited us recently and made a presentation on impact in Australia including the ERA pilot. Very interesting. I think the REF was a good start but its not a system that should just be copied and pasted onto another jurisdiction. I look forward to having more time to dig into ERA and will be watching the pilot closely.

written by Wade Kelly On 20 April 2017 Reply

I think we all have our eyes closely monitoring the situation. It’ll be interesting to see what comes of the pilot.

Leave a Comment