Measuring research impact responsibly
Researchers are often asked to demonstrate research impact for grants, awards, promotion and probation. It’s important to understand the various measures that may be used, recognise their limitations, and learn how to use them appropriately.
If you are using traditional or alternative metrics, consider these as the support act for your narrative you want to tell about yourself and your research, rather than metrics being centred as the main attraction.
Not everything can or should be measured with metrics. For things which are measured, you need to consider the following:
- Relevance - metrics suitable for your discipline and context
- Diversity - use a range of measures, qualitative and quantitative, rather than a single metric
- Transparency - acknowledge where and when you got the data
- Currency - ensure the data is up to date
- Equity - discipline citation practices vary, career length and career interruptions can impact your publication record, it’s important to compare like with like
- Visibility - not all outputs are measured and tools have geographical, subject, age, gender and language bias
- Complexity - key metrics can be simplistic, citation count is a single-value measure that gives no context to the citation, and publication count gives no context to the level of contribution
- Distortion - citation averages such as citations per paper, field-weighted citation impact etc can be misleading as they can hide the fact that 1 or 2 outliers can skew the average, especially for smaller datasets
- Integrity - some practices such as ‘citation clubs’ or excessive self-citation can over-inflate key measures
- Validity - journal metrics are not a proxy for article impact, a paper may be published in a high-impact journal however it doesn’t mean the article has been cited or even read.
Want to know more?
