In June 2015 I attended the Research Impact Assessment intensive course hosted by Alberta Innovates Health Solutions. Being a knowledge mobilization professional with little formal experience in evaluation I expected to feel like a fish out of water. Instead it was like discovering a home I never knew I had. Even down to the same furniture, except that same furniture was arranged slightly differently.
En juin 2015, j’ai assisté au cours intensif sur l’évaluation de l’impact de la recherche proposé par l’organisme de solutions en santé Alberta Innovates Health Solutions. Professionnel de la mobilisation des connaissances, mais peu rodé à l’évaluation, je m’attendais à patauger un peu. En fait, je me suis senti chez moi, comme si j’avais vécu là toute ma vie. Même les meubles étaient familiers, quoique disposés un peu différemment.
Banff, Alberta. Such a beautiful setting to relax in before taking a four day deep dive into the minutiae of health research impact assessment (RIA). Because I have little formal experience in evaluation this intensive course was perfect, albeit a little daunting (ok…a lot daunting). After spending the weekend in Banff with my husband I sent him home and logged onto the course site to review materials. Included in the materials was a very detailed RIA planning tool. The tool walked you through purpose, goals, stakeholders, activities, indicators, communications, budget….all the things you need to do in order to plan your RIA. But wait…. purpose, goals, stakeholders, activities, indicators, communications, budget are all the things you need to also develop your KT plan. I have frequently used Melanie Barwick’s KT Planning Tool to support researchers and their partners planning for KT. More recently Anneliese Poetz (Manager, KT Core for NeuroDevNet) and I have adapted this tool and combined it with project management activities to create a hybrid KT planning and project management tool. So when I saw the similarities between RIA planning and KT planning I knew I would have a very interesting four days.
Throughout those four days I was struck by the similarities between KT planning and planning for RIA.
- In the words of Kathryn Graham, Executive Director, Performance Management & Evaluation, AIHS, “the independent variable is KT, while the dependent variable is impact”. I actually had no idea this means that if research impact is “what” (or the effect) we are trying to accomplish then KT is “how” we will accomplish this, something I wrote in 2009.
- Stakeholder Engagement: engaging stakeholders at the beginning and throughout is key to KT and RIA planning
- Both RIA and KT planning should be guided by a theory of change / logic model / framework
- That theory of change / logic model / framework will inform the indicators to be collected at each stage of the plan
- Communicating impact ≠ KT. For more on this see a paper I published with Melanie Barwick
- Need to balance the gold standard of what is possible given unlimited resources with what is possible to actually accomplish within the time and budget allowed.
However, here’s the thing that I never appreciated. Jonathan Grant, Professor, The Policy Institute at King’s College London, stated that evaluation needs to move more to the demand side, prospectively planning for impact. This got me thinking about the relationship between KT/KMb and RIA. KT planning happens at the beginning of a research project. RIA tends to happen at the end (ex post evaluation) but evaluators prefer to be brought in at the beginning (ex ante evaluation) so they can help plan for purpose, goals, stakeholders, activities, indicators, communications, budget.
Since KT planning happens at the front end and we consider purpose, goals, stakeholders, activities, indicators, communications, budget it occurs to me that KT planning is ex ante research impact assessment.
This is a very interesting hypothesis and likely requires unpacking and some debate before we make this conclusion; however, the similarity of planning tools, the similarity of aspects of each plan and the ideal of ex ante evaluation creates a compelling hypothesis. One thing I do know is that the more I work with Kathryn Graham the more I realize that we have complementary expertise to bring to a project. We look at a project through a similar lens but with different perspectives. If we were decorating the same room we would use the same furniture but arrange it slightly differently.