Belcher, B., Halliwell, J. Conceptualizing the elements of research impact: towards semantic standards. Humanities and Social Science Communications. 8, 183 (2021). https://doi.org/10.1057/s41599-021-00854-2.
Abstract
Any effort to understand, evaluate, and improve the impact of research must begin with clear concepts and definitions. Currently, key terms to describe research results are used ambiguously, and the most common definitions for these terms are fundamentally flawed. This hinders research design, evaluation, learning, and accountability. Specifically, the terms outcome and impact are often defined and distinguished from one another using relative characteristics, such as the degree, directness, scale, or duration of change. It is proposed instead to define these terms by the kind of change rather than by the degree or temporal nature of change. Research contributions to a change process are modeled as a series of causally inter-related steps in a results chain or results web with three main kinds of results: (i) the direct products of research, referred to as outputs; (ii) changes in the agency and actions of system actors when they are informed/influenced by research outputs, referred to as outcomes; and (iii) tangible changes in the social, economic, environmental, or other physical condition, referred to as realized benefits. Complete definitions for these terms are provided, along with examples. This classification aims to help focus research evaluation appropriately and enhance appreciation of the multiple pathways and mechanisms by which scholarship contributes to change.
I was expecting to be pissed off by this paper authored by two people who I know and respect. I have publicly stated that I am not personally invested in definitions. I acknowledge their importance but I am not a researcher. I am an impact practitioner, and I am busy enough “doing” impact to worry about what everyone calls it. I resist getting stuck in definitional dystopia. I leave debates about definitions to my academic colleagues. I also don’t know anyone that suffers from a “debilitating lack of clarity” in definitions. Hyperbole aside, I was expecting this paper to be heavy on definitions…and it was. But the paper had what I have been waiting to see published for a long time – Brian Belcher’s diagram of spheres of control, influence, and interest.
Figure 1 presents a generic theory of change for research and research impact.
- Sphere of control is where actions are within control of the researcher.
- Sphere of influence happens when research results leave the academic space and are taken up by users who hopefully see a change in knowledge, attitudes, skills and/or relationships (KASR) and those subsequently affect a change in behaviour. The researcher, through partnerships with users, can influence these changes.
- Sphere of interest is where broader changes in society, economy or environment are occurring. The researcher is interested but can neither control nor influence them.
But then they embed another logic model in the theory of change. Within the sphere of control the sequence of research actions run vertically down from partnerships to new knowledge and innovations.
That’s cool and something I hadn’t seen in an earlier incarnation.
But this is what I see to be possible. Apart from forming partnerships (which is drawn as something researchers do because it is in the sphere of control), Figure one is all from the perspective of the researcher who is controlling, influencing or being interested. As the research “outputs” move into the sphere of influence they leave the research space and move into a user space.
But the user then has their own sphere of control that overlaps with the researcher’s sphere of influence since the user is in control of how they use the research. Then when the research has been used to change KASR and changes happen for policy and practice then there is a new sphere of control for those organizations/practitioners who are implementing a new service or policy. As the theory of change progresses different actors enter into their own sphere of control.
It should be possible to reflect progress from research to impact in a series of nested theories of change for researchers, research users and end beneficiaries.
Questions for brokers
- How do you avoid getting stuck in definitional dystopia?
- Definitions are important but to whom?
- How debilitated are you by lack of definitions?
Research Impact Canada is producing this journal club series to make evidence on KMb more accessible to knowledge brokers and to create online discussion about research on knowledge mobilization. It is designed for knowledge brokers and other knowledge mobilization stakeholders. Read this open access article. Then come back to this post and join the journal club by posting your comments.