{"id":11929,"date":"2018-04-02T15:35:17","date_gmt":"2018-04-02T19:35:17","guid":{"rendered":"http:\/\/researchimpact.ydsinc.ca\/the-leiden-manifesto-for-research-metrics\/"},"modified":"2021-11-23T09:34:16","modified_gmt":"2021-11-23T14:34:16","slug":"the-leiden-manifesto-for-research-metrics","status":"publish","type":"post","link":"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/","title":{"rendered":"The Leiden Manifesto for Research Metrics"},"content":{"rendered":"<p>Hicks, D. &#038; Wouters, P. (2015). The Leiden Manifesto for research metrics. <em>Nature, 520<\/em>(7548), 429-431.<br \/>\n<a href=\"https:\/\/www.nature.com\/news\/bibliometrics-the-leiden-manifesto-for-research-metrics-1.17351\">https:\/\/www.nature.com\/news\/bibliometrics-the-leiden-manifesto-for-research-metrics-1.17351<\/a><br \/>\nNo Abstract: \u201c<em>Use these ten principles to guide research evaluation, urge Diana Hicks, Paul Wouters and colleagues<\/em>\u201d<br \/>\nThis article lays out 10 principles of research evaluation. In a world increasingly obsessed with research metrics (quality \u2013 or excellence \u2013 as well as impact) we need some principles by which we can either create research evaluation systems or assess those systems by which we are evaluated. This manifesto comes from the group that administers the <a href=\"http:\/\/www.leidenranking.com\/\">Leiden Rankings<\/a>, one of the global university ranking systems. At first this seems like a potential conflict of interest but as you will see, those who are running a ranking system based on bibliometrics are critical of making conclusions based solely on numbers.<br \/>\nTheir 10 principles are listed below with commentary on some that can be extrapolated to thinking about impact assessment systems:<br \/>\n<strong>1. Quantitative evaluation should support qualitative, expert assessment<\/strong><br \/>\nNo numbers without stories and no stories without numbers. Research on the UK Research Excellence Framework (REF) <a href=\"https:\/\/www.kcl.ac.uk\/sspp\/policy-institute\/publications\/Analysis-of-REF-impact.pdf\">concludes<\/a> that the narrative or case study is the correct tool for impact assessment but the stories need to have some numbers to illustrate the claims of impact.<br \/>\n<strong>2. Measure performance against the research missions of the institution, group or researcher<\/strong><br \/>\n<strong>3. Protect excellence in locally relevant research<\/strong><br \/>\nKey for impact assessment. Many impacts, at least to start, are local as relationships tend to be local. These should not be discounted against national or global impacts, especially since those latter impacts are usually distal from the original research. It depends on why you are doing your impact assessment. If you\u2019re assessing the actions of a researcher or the institution to engage for impact then local is critical. If you\u2019re assessing national or international networks then scale becomes relevant.<br \/>\n<strong>4. Keep data collection and analytical processes open, transparent and simple<\/strong><br \/>\nOpen and transparent yes; however, I have never seen a \u201csimple\u201d impact assessment method although use of tools and skilled personnel can reduce transaction costs<br \/>\n<strong>5. Allow those evaluated to verify data and analysis<\/strong><br \/>\n<strong>6. Account for variation by field in publication and citation practices<\/strong><br \/>\nSimilar for impact assessment. Impacts of the humanities should not be compared against impact of engineering. Academic and non-academic experts should identify what counts as impact in their fields.<br \/>\n<strong>7. Base assessment of individual researchers on a qualitative judgement of their portfolio<\/strong><br \/>\nImpact assessment should not be on a single publication but on a body of published work and the experience of the researcher(s)<br \/>\n<strong>8. Avoid misplaced concreteness and false precision<\/strong><br \/>\n<strong>9. Recognize the systemic effects of assessment and indicators<\/strong><br \/>\nAny impact assessment system will by nature affect the system it is measuring. Institutions and individuals will change behaviour to maximize performance. This may or may not be a good thing.<br \/>\n<strong>10. Scrutinize indicators regularly and update them<\/strong><br \/>\nWhat is implicit but not explicitly mentioned is that if you are going to go beyond collecting the evidence of impact and develop an assessment system then someone is going to be assessed against the performance of someone else. The indicators and methods you choose must be public, verifiable and comparable across those being assessed.<br \/>\n<strong>Questions for brokers:<\/strong><br \/>\n1. Are you collecting the evidence of impact from your research? How do your methods and indicators stack up against these 10 principles?<br \/>\n2. Look at research evaluation systems that have impact as a feature (<a href=\"http:\/\/www.ref.ac.uk\/\">UK REF<\/a>, <a href=\"http:\/\/www.tec.govt.nz\/funding\/funding-and-performance\/funding\/fund-finder\/performance-based-research-fund\/\">NZ PBRF<\/a>, <a href=\"http:\/\/www.arc.gov.au\/engagement-and-impact-assessment\">Australia Engagement and Impact Assessment<\/a>, <a href=\"https:\/\/www.knaw.nl\/nl\/actueel\/publicaties\/standard-evaluation-protocol-2015-2021\">NL Standard Evaluation Protocol<\/a>, others?). How do they measure up against these principles?<br \/>\n3. Where is impact assessment in your jurisdiction? (Hint: if you don\u2019t think there is one it is likely someone in a policy capacity is talking about it)<br \/>\n<em>Research Impact Canada is producing this journal club series as a way to make evidence on Knowledge Mobilization more accessible to knowledge brokers and to create on line discussion about research on knowledge mobilization. It is designed for knowledge brokers and other knowledge mobilization stakeholders. Read this open access article. Then come back to this post and join the journal club by posting your comments. <\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hicks, D. &#038; Wouters, P. (2015). The Leiden Manifesto for research metrics. Nature, 520(7548), 429-431. https:\/\/www.nature.com\/news\/bibliometrics-the-leiden-manifesto-for-research-metrics-1.17351 No Abstract: \u201cUse these ten principles to guide research evaluation, urge Diana Hicks, Paul Wouters and colleagues\u201d This article lays out 10 principles of research evaluation. In a world increasingly obsessed with research metrics (quality \u2013 or excellence \u2013 [&#8230;]<\/p>\n<p><a class=\"btn btn-secondary understrap-read-more-link\" href=\"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/\">Lire la suite\u2026<span class=\"screen-reader-text\"> from The Leiden Manifesto for Research Metrics<\/span><\/a><\/p>\n","protected":false},"author":22,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[2386],"tags":[1454],"class_list":["post-11929","post","type-post","status-publish","format-standard","hentry","category-archive","tag-research-impact-fr-2"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>The Leiden Manifesto for Research Metrics - Research Impact Canada<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/\" \/>\n<meta property=\"og:locale\" content=\"fr_FR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The Leiden Manifesto for Research Metrics - Research Impact Canada\" \/>\n<meta property=\"og:description\" content=\"Hicks, D. &#038; Wouters, P. (2015). The Leiden Manifesto for research metrics. Nature, 520(7548), 429-431. https:\/\/www.nature.com\/news\/bibliometrics-the-leiden-manifesto-for-research-metrics-1.17351 No Abstract: \u201cUse these ten principles to guide research evaluation, urge Diana Hicks, Paul Wouters and colleagues\u201d This article lays out 10 principles of research evaluation. In a world increasingly obsessed with research metrics (quality \u2013 or excellence \u2013 [...]Lire la suite\u2026 from The Leiden Manifesto for Research Metrics\" \/>\n<meta property=\"og:url\" content=\"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/\" \/>\n<meta property=\"og:site_name\" content=\"Research Impact Canada\" \/>\n<meta property=\"article:published_time\" content=\"2018-04-02T19:35:17+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-11-23T14:34:16+00:00\" \/>\n<meta name=\"author\" content=\"Connie Tang\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u00c9crit par\" \/>\n\t<meta name=\"twitter:data1\" content=\"Connie Tang\" \/>\n\t<meta name=\"twitter:label2\" content=\"Dur\u00e9e de lecture estim\u00e9e\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/archive\\\/the-leiden-manifesto-for-research-metrics\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/archive\\\/the-leiden-manifesto-for-research-metrics\\\/\"},\"author\":{\"name\":\"Connie Tang\",\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/#\\\/schema\\\/person\\\/394e3b399bbe436d094e028d9c00962d\"},\"headline\":\"The Leiden Manifesto for Research Metrics\",\"datePublished\":\"2018-04-02T19:35:17+00:00\",\"dateModified\":\"2021-11-23T14:34:16+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/archive\\\/the-leiden-manifesto-for-research-metrics\\\/\"},\"wordCount\":682,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/#organization\"},\"keywords\":[\"Research impact\"],\"articleSection\":[\"Archiv\u00e9\"],\"inLanguage\":\"fr-FR\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/archive\\\/the-leiden-manifesto-for-research-metrics\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/archive\\\/the-leiden-manifesto-for-research-metrics\\\/\",\"url\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/archive\\\/the-leiden-manifesto-for-research-metrics\\\/\",\"name\":\"The Leiden Manifesto for Research Metrics - Research Impact Canada\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/#website\"},\"datePublished\":\"2018-04-02T19:35:17+00:00\",\"dateModified\":\"2021-11-23T14:34:16+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/archive\\\/the-leiden-manifesto-for-research-metrics\\\/#breadcrumb\"},\"inLanguage\":\"fr-FR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/archive\\\/the-leiden-manifesto-for-research-metrics\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/archive\\\/the-leiden-manifesto-for-research-metrics\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/accueil\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The Leiden Manifesto for Research Metrics\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/#website\",\"url\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/\",\"name\":\"Research Impact Canada\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"fr-FR\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/#organization\",\"name\":\"Research Impact Canada\",\"url\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/researchimpact.ca\\\/wp-content\\\/uploads\\\/RIC-FR-CMYK.jpg\",\"contentUrl\":\"https:\\\/\\\/researchimpact.ca\\\/wp-content\\\/uploads\\\/RIC-FR-CMYK.jpg\",\"width\":286,\"height\":66,\"caption\":\"Research Impact Canada\"},\"image\":{\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/#\\\/schema\\\/person\\\/394e3b399bbe436d094e028d9c00962d\",\"name\":\"Connie Tang\",\"url\":\"https:\\\/\\\/researchimpact.ca\\\/fr\\\/author\\\/connie\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The Leiden Manifesto for Research Metrics - Research Impact Canada","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/","og_locale":"fr_FR","og_type":"article","og_title":"The Leiden Manifesto for Research Metrics - Research Impact Canada","og_description":"Hicks, D. &#038; Wouters, P. (2015). The Leiden Manifesto for research metrics. Nature, 520(7548), 429-431. https:\/\/www.nature.com\/news\/bibliometrics-the-leiden-manifesto-for-research-metrics-1.17351 No Abstract: \u201cUse these ten principles to guide research evaluation, urge Diana Hicks, Paul Wouters and colleagues\u201d This article lays out 10 principles of research evaluation. In a world increasingly obsessed with research metrics (quality \u2013 or excellence \u2013 [...]Lire la suite\u2026 from The Leiden Manifesto for Research Metrics","og_url":"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/","og_site_name":"Research Impact Canada","article_published_time":"2018-04-02T19:35:17+00:00","article_modified_time":"2021-11-23T14:34:16+00:00","author":"Connie Tang","twitter_card":"summary_large_image","twitter_misc":{"\u00c9crit par":"Connie Tang","Dur\u00e9e de lecture estim\u00e9e":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/#article","isPartOf":{"@id":"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/"},"author":{"name":"Connie Tang","@id":"https:\/\/researchimpact.ca\/fr\/#\/schema\/person\/394e3b399bbe436d094e028d9c00962d"},"headline":"The Leiden Manifesto for Research Metrics","datePublished":"2018-04-02T19:35:17+00:00","dateModified":"2021-11-23T14:34:16+00:00","mainEntityOfPage":{"@id":"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/"},"wordCount":682,"commentCount":0,"publisher":{"@id":"https:\/\/researchimpact.ca\/fr\/#organization"},"keywords":["Research impact"],"articleSection":["Archiv\u00e9"],"inLanguage":"fr-FR","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/","url":"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/","name":"The Leiden Manifesto for Research Metrics - Research Impact Canada","isPartOf":{"@id":"https:\/\/researchimpact.ca\/fr\/#website"},"datePublished":"2018-04-02T19:35:17+00:00","dateModified":"2021-11-23T14:34:16+00:00","breadcrumb":{"@id":"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/#breadcrumb"},"inLanguage":"fr-FR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/researchimpact.ca\/fr\/archive\/the-leiden-manifesto-for-research-metrics\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/researchimpact.ca\/fr\/accueil\/"},{"@type":"ListItem","position":2,"name":"The Leiden Manifesto for Research Metrics"}]},{"@type":"WebSite","@id":"https:\/\/researchimpact.ca\/fr\/#website","url":"https:\/\/researchimpact.ca\/fr\/","name":"Research Impact Canada","description":"","publisher":{"@id":"https:\/\/researchimpact.ca\/fr\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/researchimpact.ca\/fr\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"fr-FR"},{"@type":"Organization","@id":"https:\/\/researchimpact.ca\/fr\/#organization","name":"Research Impact Canada","url":"https:\/\/researchimpact.ca\/fr\/","logo":{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/researchimpact.ca\/fr\/#\/schema\/logo\/image\/","url":"https:\/\/researchimpact.ca\/wp-content\/uploads\/RIC-FR-CMYK.jpg","contentUrl":"https:\/\/researchimpact.ca\/wp-content\/uploads\/RIC-FR-CMYK.jpg","width":286,"height":66,"caption":"Research Impact Canada"},"image":{"@id":"https:\/\/researchimpact.ca\/fr\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/researchimpact.ca\/fr\/#\/schema\/person\/394e3b399bbe436d094e028d9c00962d","name":"Connie Tang","url":"https:\/\/researchimpact.ca\/fr\/author\/connie\/"}]}},"_links":{"self":[{"href":"https:\/\/researchimpact.ca\/fr\/wp-json\/wp\/v2\/posts\/11929","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/researchimpact.ca\/fr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/researchimpact.ca\/fr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/researchimpact.ca\/fr\/wp-json\/wp\/v2\/users\/22"}],"replies":[{"embeddable":true,"href":"https:\/\/researchimpact.ca\/fr\/wp-json\/wp\/v2\/comments?post=11929"}],"version-history":[{"count":0,"href":"https:\/\/researchimpact.ca\/fr\/wp-json\/wp\/v2\/posts\/11929\/revisions"}],"wp:attachment":[{"href":"https:\/\/researchimpact.ca\/fr\/wp-json\/wp\/v2\/media?parent=11929"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/researchimpact.ca\/fr\/wp-json\/wp\/v2\/categories?post=11929"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/researchimpact.ca\/fr\/wp-json\/wp\/v2\/tags?post=11929"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}