Zardo, P., Barnett, A.G., Suzor, N. and Cahill, T. (2018) Does engagement predict research user? An analysis of The Conversation Annual Survey 2016. PLoS ONE 13(2): e0192290. https://doi.org/10.1371/journal.pone.0192290
Abstract
The impact of research on the world beyond academia has increasingly become an area of focus in research performance assessments internationally. Impact assessment is expected to incentivize researchers to increase engagement with industry, government and the public more broadly. Increased engagement is in turn expected to increase translation of research so decision-makers can use research to inform development of policies, programs, practices, processes, products, and other mechanisms, through which impact can be realized. However, research has shown that various factors affect research use, and evidence on ‘what works’ to increase decision-makers’ use of research is limited. The Conversation is an open access research communication platform, published under Creative Commons licence, which translates research into news articles to engage a general audience, aiming to improve understanding of current issues and complex social problems. To identify factors that predict use of academic research and expertise reported in The Conversation, regression analyses were performed using The Conversation Australia 2016 Annual Survey data. A broad range of factors predicted use, with engagement actions being the most common. Interestingly, different types of engagement actions predicted different types of use. This suggests that to achieve impact through increased engagement, a deeper understanding of how and why different engagement actions elicit different types of use is needed. Findings also indicate The Conversation is overcoming some of the most commonly identified barriers to the use of research: access, relevance, actionable outcomes, and timeliness. As such, The Conversation offers an effective model for providing access to and communicating research in a way that enables use, a necessary precursor to achieving research impact.
There is a lot of literature in this journal club about research impact and knowledge mobilization as that represents the bulk of literature. There is relatively less literature, and hence fewer journal club posts, on research engagement. This article uses the data form a readers’ survey of The Conversation Australia. “The Conversation is an online research communication platform that provides access to academic research evidence that has been ‘translated’ and communicated in a form more accessible to general audiences than traditional academic journal articles. The Conversation articles include papers that report on or review research findings as well as articles that offer academic expert opinion and analysis on current news issues.” Importantly the survey asks readers not only (essentially) “what did you do with the research” but it also asked the survey respondent to disclose the sector they worked in and their professional role.
The back drop for the research is the Australian Engagement & Impact Assessment exercise. While (I think) the engagement elements ask for case studies it pulls data from four existing data sets. That is good because it is easier to administer and harder to game. However, it pulls data from industry contracts, commercialization income and funding from non-traditional sources (charities, government ministries etc). It’s all about funding which is an important metric but is a very blunt metric for the many subtleties of engagement. The Australian Research Council (ARC) has heard this from me before…moving on…
The ARC definition of impact is “The contribution that research makes to the economy, society, environment and culture beyond the contribution to academic research”. This definition is fine, but the authors’ interpretation is slightly flawed. “This aligns with the academic definitions and types of use outlined above. It suggests that research must be used in some way that contributes to the world beyond academia. What if the users of your research are within academia? Not other researchers but if your work seeks to inform academic policy and practice? My scholarship is used by universities to inform knowledge mobilization practice, policy and systems within academia. Rant over…moving on…
Read the paper for all the good stuff. Bottom line, only 15% of respondents reported using The Conversation articles for policy and strategy but this increases if your role is in policy, in government or in management. Makes sense. If you’re not making policy you’re not likely to use it for policy. But there was a 71% probability that articles in The Conversation would be used for work related discussions and debate.
This article establishes that engagement with research facilitates use of research. Makes sense since you can’t use something you haven’t obtained; nonetheless, it’s good to see qualitative data support this.
But here’s the thing…this helps on the engagement side, but does it help on the impact side? The article doesn’t present anything about readers ability (or lack thereof) to feedback to the researcher authors of articles in The Conversation. It’s great that people are using the articles for various reasons and some of these will no doubt be used to inform public policy or professional practice but how are we ever going to know? And if we (in the university) don’t get that information then we can’t craft the impact case studies that underpin impact assessment exercises. Like a tree falling in the forest, if impact happens but no one is there to tell the story then do we know if anything actually happened?
Questions for brokers:
- How would you close the loop between a reader of The Conversation using a story and the researcher producing the research output and writing the article for The Conversation?
- If your research can be used by academic administrators for academic policy and practice do you feel you fit definitions of impact and engagement?
- There is also The Conversation Canada. Check it out.
Research Impact Canada is producing this journal club series to make evidence on KMb more accessible to knowledge brokers and to create online discussion about research on knowledge mobilization. It is designed for knowledge brokers and other knowledge mobilization stakeholders. Read this open access article. Then come back to this post and join the journal club by posting your comments