Thinking About Impact in New Zealand / De Nouvelle-Zélande, une réflexion sur l’impact

I went to New Zealand prepared to speak about research impacts. I found out (just in time) they had already prepared the audiences for me.
Je suis parti pour la Nouvelle-Zélande avec une conférence toute prête sur l’impact de la recherche. J’ai découvert (à la dernière seconde) que mon auditoire était déjà très renseigné!

I was privileged to be invited to be part of the Expert Advisory Committee by the New Zealand Ministry of Business, Innovation and Employment (MBIE) to help MBIE, the Ministry of Health and the NZ Health Research Council implement their NZ Health Research Strategy. We had our first meeting the week of February 26 and I also made a presentation on research impact to a diverse government audience. I was up early on February 28 cruising through twitter when I saw the following tweet from @JulieEBayley


An impact consultation in New Zealand???? You don’t say! So of course, I click the link and see a report from MBIE released at the end of 2017.

Imagine that. I am 2.5 hours away from presenting to MBIE and others about impact only to find that they have already published their own consultation on impact. Only having time to read the executive summary I see they made conclusions that align well with my messages.

MBIE Report cover From the report’s executive summary:

• “Many submissions supported the use of the generic results chain model but often with significant qualifications, in particular, that its linear approach was too simplistic and did not portray the networked nature of science.

• There was a strong call to continue to measure intermediate results, and reflect more on the interim stages of impact. This was partly because of the difficulties of measuring the long-term impacts. Many submitters supported valuing and measuring academic impact and development of human capital.

• Many submissions emphasised that end-users and other stakeholders had an important role in determining the impact of science and that accountabilities ought to reflect the various roles.”

Let me summarize the summary: respondents to the consultation want a non-linear logic model that tracks interim as well as later stage impacts and captures the contributions of non-academic partners.

Well MBIE…have I got a pathway for you! We published the co-produced pathway to impact in 2016 and continue to develop it with Kids Brain Health Network. This pathway is a logic model that has non-linear feedback loops, identifies categories of indicators along the pathway and features co-production partners, receptors/audiences and stakeholders playing key roles.

Co-produced pathway to impact 2018 versionAs mentioned, York University’s Knowledge Mobilization Unit and Kids Brain Health Network have continued to refine the 2016 pathway to the pathway in the figure above. We use this pathway to help plan for impact which includes ex ante research impact assessment.

I am happy to loan the co-produced pathway to impact to MBIE and other interested organizations including the Tertiary Education Commission which manages the NZ Performance Based Research Fund. The PBRF offers the opportunity for university researchers to provide evidence of academic and non-academic research impacts.

MBIE…you’re welcome!

4 thoughts on “Thinking About Impact in New Zealand / De Nouvelle-Zélande, une réflexion sur l’impact

  1. Thanks David Phipps. We have a PBRF quality evaluation round this year, broadly equivalent to the UK’s REF but without any obligation to provide evidence of impact. It’s likely that the Tertiary Education Commission will then review PBRF but whether MBIE limits the introduction of the impact agenda to grant funding or extends it to the whole PBRF process is anyone’s guess at this stage.

    1. Thanks for your comment, Lesley. There are a bunch of us around the world interested in impacts of research – check out the Network for Advancing and Evaluating the Societal Impact of Science (AESIS Network): http://aesisnet.com/. PBRF is on the radar and it will be interesting to see the interplay between TEC/PBRF and MBIE as this agenda builds momentum in NZ.

  2. Great insights David, I have been working with researchers in NZ on their impact pathways in grants and for the PBRF for a few years now with great success. They are really embracing impact at the granting end which will facilitate any plans for impact assessment in the future. Next time let me know and I can inform you of the NZ updates if needed!

  3. excellent article David. I am working in this area and also undertaking a PhD looking at how evaluation practices affect law academics. It is very exciting times ahead!

Leave a Reply

Your email address will not be published. Required fields are marked *