Ha TC, McNamara M, Melo L, Frost EK, Moore GM. Filling the gap between evidence, policy and practice: are 45 and Up Study researchers planning for impact? Public Health Res Pract. 2022; Online early publication. Published 18 May 2022. https://doi.org/10.17061/phrp32122207
Abstract
Aim: To improve health outcomes, policy and practice decisions should be guided by relevant and timely evidence. High-quality, large-scale population data could play an essential role in supporting evidence-based decision making. The 45 and Up Study is a long-term, large-scale cohort study with more 250 000 participants aged 45 years and over from New South Wales (NSW), Australia. Data collected by the Study is accessible to researchers, government and non-governmental bodies. The study aimed to identify the proportion of researchers using data from the Study who intended to have an impact and achieved impact; the types of impact they intended and achieved; and the pathways through which they achieved it.
Methods: Using data extracted from the application, progress and final report documents for 25 projects using 45 and Up Study data from January 2011 until December 2017, we a) determined the proportion of projects that intended to have policy or practice impact and b) described the type of policy and practice impact achieved.
Results: We found that 88% (n = 22) of projects intended to have a policy or practice impact. Of those, 68% (n = 15) planned to influence or inform a policy or program, and 41% (n = 9) planned to share findings at conferences or in journals. Almost half of projects with intended impact (45%, n = 10) did not state how they planned to achieve impact. Approximately 16% of all projects (n = 4) reported achieving an impact on policy or services. The type of impact achieved by all four of these projects was influencing, informing or changing a policy or program. One of these four projects also achieved a change to legislation or regulation.
Conclusions: Further strategies to promote a targeted approach to impact planning in research projects using datasets such as the 45 and Up Study would help guide researchers in achieving impact.
Let’s start where the abstract ends: “Further strategies to promote a targeted approach to impact planning in research projects…would help guide researchers in achieving impact.” The majority of projects using this data set intended to make impact on policy and yet half of the projects intending to create an impact did not state how they planned to achieve impact. That’s like saying “I’m going to drive to XYZ but I don’t have a map so I’m just going to drive somewhere and hope I get to XYZ”.
And then it isn’t a surprise that only 4 projects reported achieving impact on policies or services.
There is a good analysis of the causes of the gaps between research and policy
- Lack of timely, relevant research
- Limited applicability of research to policy contexts
- Insufficient access to journals
- Institutional priorities favouring academic metrics and not relationships with policy makers
- Not enough time, training or incentives to pursue policy impact
- Research evidence is only one of many inputs into policy decisions
The authors acknowledge a limitation that the final project reports are delivered at the end of the project where policy impact is likely not yet to have been made and that the reports are made by researcher self reporting so might not capture policy impact since they’re not making policy.
Hmmm….
And a couple of conclusions
The lack of a targeted approach to knowledge mobilisation is one of the most common barriers identified in the literature to achieving real-world impacts.
These findings suggest that 45 and Up Study researchers, and potentially researchers more broadly, may benefit from a more targeted approach to impact planning.
There is a discordance between ‘intention to have impact’ and ‘achieving impact’ and further studies are required to elucidate why this is the case.
Ummm…yeah.
Questions for brokers:
- If the study has the limitations mentioned above can any conclusions really be drawn if the study uses researcher self reporting at a time where impact is not likely to have occurred and those making the impact are not part of the reporting? Is this study fatally flawed?
- Despite these flaws how might you use this paper to encourage researchers to plan for impact in their proposals?
- Should those providing access to the data set require potential users to have a plan for impact?
Research Impact Canada is producing this journal club series to make evidence on knowledge mobilization more accessible to knowledge brokers and to create online discussion about research on knowledge mobilization. It is designed for knowledge brokers and other knowledge mobilization stakeholders. Read this open access article. Then come back to this post and join the journal club by posting your comments.