Donnelly, C., Letts, L., Klinger, D. & Shulha, L. (2014). Knowledge translation through
evaluation: Evaluator as knowledge broker. Canadian Journal of Program Evaluation, 29(1), 36–61 doi: 10.3138/cjpe.29.1.36
http://www.evaluationcanada.ca/secure/29-1-036.pdf
Abstract
The evaluation literature has focused on the evaluation of knowledge translation activities, but to date there is little, if any, record of attempts to use evaluation in support of knowledge translation. This study sought to answer the question: How can an evaluation be designed to facilitate knowledge translation? A single prospective case study design was employed. An evaluation of a memory clinic within a primary care setting in Ontario, Canada, served as the case. Three data sources were used: an evaluation log, interviews, and weekly e-newsletters. Three broad themes emerged around the importance of context, efforts supporting knowledge translation, and the building of KT capacity.
I usually post a summary of an article I think makes a valuable contribution to the knowledge mobilization literature and hence the practice of knowledge mobilization. Not so in this case. This article creates false dichotomies between evaluators/evaluation and knowledge brokers/knowledge translation. This article might be news to evaluators but there is nothing new for knowledge brokers. Nonetheless, this article begs the question why is this news to evaluators and what can we do to let them realize they are already an important part of the various worlds of knowledge mobilization.
Here’s a quick summary of the article.
Evaluators want to evaluate knowledge translation.
Evaluators implement knowledge translation activities in a memory clinic.
Evaluators assess knowledge translation success.
Conclusion: evaluators can function as knowledge brokers, a new role for evaluators.
Duh. This is not a “drop the mic” moment for knowledge brokers.
If you ask an evaluator to undertake knowledge translation roles then they are knowledge brokers. If you asked a plumber to undertake knowledge translation s/he would be a knowledge broker.
I don’t see what this adds to the literature.
But this allows us to examine the intrinsically interlinked roles of knowledge translation and evaluation.
Every good knowledge broker establishes an evaluation of their knowledge translation intervention to assess if our work made a difference. Knowledge brokers are evaluators so it should come as no surprise that evaluators working in a knowledge translation study are knowledge brokers.
When planning a knowledge translation intervention we know that the impact of the intervention is the dependent variable (that thing we measure) and knowledge translation is the independent variable (that thing we change to observe an effect on the dependent variable). The two roles of knowledge broker (affecting the independent variable) and evaluator (assessing the dependent variable) are intrinsically linked.
Much evaluation happens along the way (measuring process indicators) and at the end (ex post measuring outcome indicators). But since knowledge brokers plan the knowledge translation (including how to evaluate it) at the beginning of the process then knowledge translation planning is ex ante research impact assessment.
See a blog I wrote about this September 2015.
When you connect the dots like this then knowledge mobilization/translation embraces evaluation and knowledge brokers can be evaluators and vice versa. The paper adds nothing new to this but it lets us step back and realize how knowledge translation and evaluation are intimately linked.
Questions for brokers:
1- Was this a moment of “doh, of course I knew that” or was this a revelation to you? Why and how different do you see your role now?
2- If knowledge brokers and evaluators both work on a spectrum from planning for impact to assessing impact why do we present them as an artificial duality with artificially distinct roles?
3- What can the disciplines of knowledge mobilization/translation and evaluation/research impact assessment do to create a shared space of “research impact practitioners” (thank you @JulieEBayley)?
ResearchImpact-RéseauImpactRecherche is producing this journal club series as a way to make the evidence and research on knowledge mobilization more accessible to knowledge brokers and to create on line discussion about research on knowledge mobilization. It is designed for knowledge brokers and other knowledge mobilization stakeholders. Read the article. Then come back to this post and join the journal club by posting your comments.