With thanks to Karen Ritchie, Head of Knowledge and Information, Health Improvement Scotland, who first coined this phrase. This post examines the plethora of impact frameworks and their – usually inappropriate – use.
Merci à Karen Ritchie, chef du service des connaissances et de l’information de l’organisme écossais Health Improvement, qui a forgé cette métaphore. Ce blogue s’intéresse à la pléthore de cadres, structures et méthodes d’évaluation de l’impact et à l’usage – généralement inadéquat – qui en est fait.
“Impact frameworks are like toothbrushes. Everyone has one and no one wants to use anyone else’s”.
Co-Produced Pathway to Impact
“Impact frameworks are like toothbrushes. Everyone has one and no one wants to use anyone else’s”.
Knowledge to Action Cycle, Canadian Academy of Health Sciences Impact Assessment Framework, Payback method, Co-produced pathway to impact (CPPI), SPIRIT Action Framework, etc., etc., etc.
See a recent review of the strengths and weaknesses of some of these models here.
In Canada, the KTA Cycle dominates. Many networks, programs and projects cite the KTA Cycle as their framework without knowing that the KTA authors themselves never expected It to be used in whole by any single organization. In a review of papers citing KTA, only 10/146 actually implemented a portion of it and only one employed KT methods to move from one stage to the next.
No pathway is perfect which is why everyone creates a new pathway or new modification to a pathway to solve the one thing that doesn’t work for them despite the many things that do work.
STOP TRYING TO MAKE THE PERFECT PATHWAY (it doesn’t and won’t exist) AND START USING THE EXISTING PATHWAYS.
But with a plethora of pathways – a veritable plentiful profusion of pathways – how does one go about choosing a pathway that’s right for your research to impact project? NIHR asked me this in 2016 and I came up with the following five criteria for impact pathway assessment (as published in this blog on May 5, 2016).. Does the pathway:
1. Accommodate and enable collection of evidence for patient benefit?
2. Support engagement of end users (incl. patients, policy, service providers) throughout?
3. Work at the level of the project, the program, the organization, the system?
4. Enable planning by providing general logic informing specific adaptation?
5. Drive uptake/adoption?
In the May 5, 2016 post, I reviewed three pathways: KTA, Payback and the CPPI. Acknowledging bias as I am the author of CPPI (yes, even I made another damn framework!), the CPPI came out on top on these five criteria.
But here’s the thing about any pathway. It is at best generic. No framework can be specific to every project implementing the framework. The CPPI can be used to monitor the progress from biomarker identification to successful clinical microarray test as it can be used to monitor the progress from understanding needs of at risk youth to successful implementation of a life skills training program. Clearly these two pathways will be very different. At York in 2016 we had supported 121 large scale grant applications of which 42 (35%) had been successful attracting $47M in external research income. Each one had a different pathway to impact.
Of the 6,679 impact case studies in the UK Research Excellence Framework there were 3,709 unique pathways to impact (see here).
With this diversity, clearly even the best impact frameworks can only be generic. The best advice any funder can give is generic (for example the guidance on knowledge mobilization strategies from SSHRC). It is up to the researchers, partners and the research impact practitioners who support them to use planning tools to develop a specific (or bespoke as @JulieEBayley likes to say) impact pathway for every research to impact project.
Since almost all grant applications require some form of impact pathway seek out your local research impact practitioner to help secure your next research grant.