Michael’s ‘Aha Moment’!

I am flattered to know someone asked for a blog based on a Tweet I contributed on the ResearchImpact Twitter feed.  First, a few observations and disclaimers.  I am glad blog posts do not mirror dissertations in rigour or length.  Next, I do not claim to be an authority on ‘outcomes’ or ‘impacts’ although my work is heavily invested in both terms/processes.  Lastly, I admit I carried around strong assumptions that the logic model for impact followed a sequential (and not very quick moving) flow from activity to outcome to impact. 

January 11 and 12, I had the pleasure of attending a Scientist Knowledge Translation Training event which was hosted by The Hospital for Sick Children and was led by Drs. Melanie Barwick and Donna Lockett .  Over two days, Melanie and Donna shared practical tools for developing Knowledge Translation (KT) plans, led discussions toward a more clear understanding of KT and provided valuable exercises to improve attendees capacity to understand the ‘user context’ for successful linkage and exchange, which is a foundation for successful KT.  The 25 attendees present were predominantly health practitioners who had KT responsibilities embedded into their job descriptions although there were some health researchers and policy professionals in attendance as well.

However, back to the notion of impact.  Never one to be terribly shy, I asked about the relationship between outcomes and impact, stating my feeling it was not possible to measure impact so closely to any KT transaction because impact was a by-product of outcomes.  What triggered this question was a slide that identified short-term, intermediate and long-term outcomes.  Moreover, while this makes sense, some confusion arose for me with regard to ‘measuring impact’, which in my experience is a challenge in policy and practice-relevant research.  So when one of the facilitators commented that she would be seeking impact measures based on short-term behavioural or practice changes amongst the participants of the session, I was skeptical.  What followed was a brief discussion between us two about the relationship of outcome and impact and that it is possible to identify impact measure very closely after a KT transaction.

The ensuing discussion did not necessarily change my beliefs around impact in relation to outcomes.  Reflecting back, I would say they have expanded my beliefs.  Impact is no longer solely a longitudinal process which one must wait (pick your timeframe – 6 months, one year, five years, and so on) to identify behaviour or practice changes.

I look forward to further discussion on this topic, and the inevitable reading that I will embark upon to challenge and reinforce my expanded belief system on impact.  Given the significance of this topic for publicly funded researchers and practitioners, it is a conversation which we should all be engaged with, and a topic we should give voice to.  Hey, that could be a second ‘aha moment’!

4 Responses to “Michael’s ‘Aha Moment’!”

written by JoVE On 19 January 2010 Reply

Thanks. This does raise interesting issues.

Would short term impacts be things like recognition that there are useful concepts? or engagement in serious discussion of the research?

I’m wondering if this looks different if you are looking for conceptual uses — getting people in a policy context to think differently about problems, for example — yet your model of impact measurement is about instrumental uses.

Keep wondering out loud. I think that’s one of the good things about blogging as a medium. It can get some half-formed ideas out there where we can collaboratively form them into something.

written by Michael Johnny On 19 January 2010 Reply

Hi Jo,
I feel context is a critical component. Your examples, such as recognition of useful concepts demonstrate a shift in thought or behaviour so yes, I suspect that would be an impact. In our work, to-date, we have looked to examine impact in areas of public policy and/or professional practice. Speaking for myself, I know I may not have been looking closely enough at aspects such as changes in behaviour (for example, have decision-makers expanded where they go get information to inform decisions, or are practitioners using social media to learn alternative modes of program delivery). These subtle (or not so subtle) shifts are likely the precursors to the larger, and more visible aspects of change (or impact) which I have been seeking. Again, I speak for myself here and not my colleagues, after all, it was my preconceived notions which have been challenged!

I will continue to poke and prod on this issue, as it fascinates me. Terms which are used so often, and are very significant in the context of accountability, require a more clear and common understanding.

Thanks for the reply,

written by Sarah Morton On 3 February 2010 Reply

I am currently doing some research on impact, and its assessment combined with my job as a KE professional so these issues are very much on my mind.
I wanted to share some interesting reading on this for those of you looking.
A literature review on the impact literature
and Capturing Research Impacts: A review of international practice

very happy to enter into more discussion on this

written by Michael Johnny On 3 February 2010 Reply

Thanks for the information, Sarah
This is so very helpful, to me and others following this blog. I would welcome the chance to learn more about your research, especially given your professional interests in KE. (mjohnny@yorku.ca)

I will review the resources you have enclosed and be sure to reply again with any comments or questions. It is great to see the interest in this topic on the rise.

Cheers, Michael

Leave a Comment