{"id":12506,"date":"2010-05-28T14:23:08","date_gmt":"2010-05-28T18:23:08","guid":{"rendered":"http:\/\/researchimpact.ydsinc.ca\/york-responds-to-the-knowledge-mobilization-evaluation\/"},"modified":"2021-11-23T15:03:55","modified_gmt":"2021-11-23T20:03:55","slug":"york-responds-to-the-knowledge-mobilization-evaluation","status":"publish","type":"post","link":"https:\/\/researchimpact.ca\/fr\/archive\/york-responds-to-the-knowledge-mobilization-evaluation\/","title":{"rendered":"York responds to the knowledge mobilization evaluation"},"content":{"rendered":"

\"\"<\/a>
\nLike most service units York\u2019s Knowledge Mobilization (KMb) Unit counts lots of stuff. We count every presentation we make according to the audience, we count numbers of requests for service, we count students, web hits, tweets, students engaged with non-academic research stakeholders and, of course, we count money.
\nWhat has this shown? We\u2019ve brokered over 155 collaborations between researchers and non-academic research stakeholders. We\u2019ve worked with over 200 York faculty. We\u2019ve had over 2M web hits on our web site. We have helped faculty raise over $15M in funding for engaged research, we have placed 29 graduate student KMb Interns with partners (4 of whom were subsequently hired by their internship partner organization) and we have attracted over $700K for funding from KMb partners. But the question remains:
\nSo What?<\/strong>
\n\"\"<\/a>We have been busy but what impact has this activity had on faculty, students and partners? Most of what we count are inputs into a system of institutional KMb support. What \"\"<\/a>are the outputs, outcomes or impacts? We wish to better articulate the downstream impacts of our efforts so in May 2009 we began a formal evaluation of the first 2.5 years of KMb. York\u2019s KMb Unit contracted the Program Evaluation Unit of the York Institute for Health Research<\/a> to develop and undertake an evaluation of the KMb Units at York, at UVic and at their collaboration called ResearchImpact. Under the supervision of Michaela Hynie, Director of the Program Evaluation Unit, the evaluation ran from June \u2013 September 2009 and a final report which will be released at Congress 2010<\/a> was tabled in November 2009.
\nOn February 19, 2010 the report was reviewed by a committee of stakeholders representing local and national perspectives on KMb. Chaired by Stan Shapson (Vice-President Research & Innovation, York University), two York faculty met with external stakeholders (see below for Committee membership) and were asked three questions:
\n1.\tHow should we continue to develop institutional support services at York University?
\n2.\tHow should we develop ResearchImpact (Canada\u2019s national knowledge mobilization network)?
\n3.\tHow can we apply the principles of KMb to industry liaison?
\nThe review committee was positive about York\u2019s institutional KMb capacity and recognized that KMb has certainly created benefits for York, for researchers, for graduate students, for research partners. However, they also provided helpful critical suggestions for the future. The reviewers felt the evaluation fell short of identifying the social, economic, cultural or environmental impacts of research. One reviewer commented that the report did not capture the intangible benefits of KMb at the community level. In a presentation<\/a> on measuring the outcomes and benefits of university research at the Canadian Association of University Research Administrators<\/a> (May 9-12, 2010 ) David Phipps recognized that York invested a lot of time to get to a point where we were less than satisfied with our ability to demonstrate the impact of KMb. This is not a criticism of the evaluation report or of the methodology but an observation about the state of evaluation of the non-academic impacts of research, especially in the social sciences and humanities. We also recognize that 2.5 years is insufficient time for many impacts to be realized. Outputs (such as graduate student interns trained) and outcomes (such as graduate students hired by their intern host organizations) were identified but impacts and a social return on investment were generally lacking.
\nNonetheless we are pleased that the evaluation reported many successes and achievements and the Committee left the KMb Unit with a number of areas on which we can focus for continued growth. Since the Evaluation Committee meeting on February 19, 2010, Stan Shapson has met three times with David Dewitt (Associate VP Research, Social Sciences & Humanities), David Phipps (Director, Research Services & Knowledge Exchange) and Michael Johnny (Manager, Knowledge Mobilization) to consider the outcomes of the evaluation report and the Committee\u2019s input to inform decisions about future KMb developments at York and ResearchImpact. See the table below for recommendations and action items arising from York\u2019s response to the review committee\u2019s comments on the evaluation report. See also the following documents for more information on York\u2019s evaluation of the KMb Unit and ResearchImpact.
\n\u2022\tResearchSnapshot summary of evaluation report\u00a0Hynie Evaluation of KMb ResearchSnapshot<\/a>
\n\u2022\tPresentation made by Michael Hynie to Evaluation Committee\u00a0Evaluation Report Feb 19, 2010<\/a>
\n\u2022\tFinal Evaluation report\u00a0KMb Evaluation Final Report<\/a>
\nIf you have any feedback please comment on this article using the Leave a Comment feature above.
\nWe wish to thank Michaela Hynie and her team for undertaking the evaluation. We also wish to thank the members of the Evaluation Review Committee:
\nLisa Drouillard, Director NSERC Liaison,
\nScience and Innovation Sector, Industry Canada
\nPierre-Gerlier Forest, President
\nTrudeau Foundation
\nJanet Halliwell
\nFormer Executive VP, Social Sciences and Humanities Research Council
\nBill Hogarth
\nFormer Director, York Region District School Board
\nValerie Preston
\nProfessor, Department of Geography, York University
\nDirector, Centre of Excellence for Research on Immigration and Settlement (CERIS)
\nStephen Gaetz,
\nAssociate Professor, Faculty of Education
\nDirector, Canadian Homelessness Research Network<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Recommendation<\/strong><\/td>\nAction Items<\/strong><\/td>\nNext Steps <\/strong><\/td>\n<\/tr>\n
1. Capacity Building<\/td>\nInclude KMb tools in KM in AM schedule<\/td>\n\n
    \n
  • Develop KMb tool kit (already developing tools for brokers and P2P<\/li>\n
  • Alternate meet and greet with KMb tools<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
KM in PM<\/td>\n\n
    \n
  • Themes will emerge from KM in AM<\/li>\n
  • Currently exploring youth engagement<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
KM in AM<\/td>\n\n
    \n
  • Diabetes, aging workforce May 2010<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
Roll out KMb Tool Kit<\/td>\n\n
    \n
  • Clear language research summary completed in draft<\/li>\n
  • Schedule of new tools developed for release<\/li>\n
  • Roll out: P2P, national knowledge brokers teleconference, KM in AM<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
2. Faculty Engagement<\/td>\nLink to White Paper<\/td>\n\n
    \n
  • MJ and PR developing document<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
Articulate Value Proposition for faculty<\/td>\n\n
    \n
  • Identify faculty already working in York Region (completed)<\/li>\n
  • Highlight faculty working in York Region on blog and in newsletters<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
Governance<\/td>\n\n
    \n
  • Focus with faculty working in York Region \u2013 what can they contribute to a governance structure<\/li>\n
  • Launch new Joint Advisory Committee<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
3. Student Engagement<\/td>\n2010 Interns (UWYR-York)<\/td>\n\n
    \n
  • One page vision including budget (completed)<\/li>\n
  • SS speak with DZ (completed)<\/li>\n
  • Materials for competition prepared (completed)<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
4. Strategic vs Portfolio Approach to knowledge brokering<\/p>\n
    \n
  • Consensus for a blended approach<\/li>\n<\/ul>\n<\/td>\n
UW Priorities<\/p>\n
    \n
  • helping our youth grow up strong<\/li>\n
  • enabling individuals and families to achieve economic independence<\/li>\n
  • improving the wellbeing of individuals and community<\/li>\n<\/ul>\n

    York Capacity<\/p>\n

      \n
    • Possible York capacity<\/li>\n
    • Mental health<\/li>\n
    • Immigration & settlement<\/li>\n
    • Poverty & Homelessness<\/li>\n
    • Climate Change<\/li>\n<\/ul>\n<\/td>\n
\n
    \n
  • Confirm York capacity<\/li>\n
  • Map unique York capacity onto UW goals<\/li>\n
  • Determine where there is \u201cbench strength\u201d and capacity to work in public policy<\/li>\n
  • Meeting with York Region to explore YR priorities<\/li>\n
  • Thematic meetings to explore collaborations; 1) reactivate poverty working group; 2) explore CERIS to lead immigration; 3) successful SSHRC Public Outreach grant for climate change; 4) mental health<\/li>\n
  • Need to identify funding opportunities<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
5. Convergence Centre<\/td>\nMove Community Collaboration Stations to CC<\/td>\n\n
    \n
  • Open convergence centre Fall 2010<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
Hold events at CC<\/td>\n\n
    \n
  • Every other KM in AM<\/li>\n
  • KMb Expo 2011<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
KMb Unit presence at CC<\/td>\n\n
    \n
  • MJ spend 1 day\/week at CC<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
Community Awareness<\/td>\n\n
    \n
  • Identify key community partners and develop communication strategy for dissemination to encourage use of CC for community uses<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
6. Evaluation & Impacts<\/td>\nEvaluation Report: what are the most important\/critical findings<\/p>\n
    \n
  • Evaluation didn\u2019t capture some of the impacts experienced in York Region<\/li>\n<\/ul>\n<\/td>\n
\n
    \n
  • Develop clear language research summary<\/li>\n
  • MJ and DP review report and presentation<\/li>\n
  • Provide points to VPRI\/Community for comments<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
Continue to track all activities<\/td>\n<\/td>\n<\/tr>\n
Case Study Method<\/p>\n
    \n
  • possibly with UWYR as case study partner<\/li>\n<\/ul>\n<\/td>\n
\n
    \n
  • identify faculty with case study experience<\/li>\n
  • discuss with Janet Halliwell<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
Break through evidence based evaluation<\/p>\n
    \n
  • <\/li>\n<\/ul>\n<\/td>\n
\n
    \n
  • Contact Janet Halliwell for guidance on methods<\/li>\n
  • Review materials from SSHRC Evaluation meeting at Congress 2010<\/li>\n
  • Big thinking: CIFAR, John Halliwell (UBC), others?<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n
7. ResearchImpact<\/td>\nFunctions:<\/p>\n
    \n
  • Community of Practice<\/li>\n
  • Source of research for receptors<\/li>\n<\/ul>\n

    Role for Industry Liaison?<\/td>\n

\n