This information includes data about the number of times a scholarly publication has been cited in other scholarly publications – called citation counts. Eleven of REF 2021’s 34 expert panels have said they plan to use citation data to inform the peer review process during the assessment phase of REF 2021.
A team at ISI will match publication records, which higher education institutions (HEIs) will submit to REF 2021, to an online subscription-based scientific citation indexing service called the Web of Science.
They will collaborate with REF 2021’s expert panels to work out which additional information will help them make their decisions and make sure the citation counts they provide can be reviewed in a meaningful way.
Panels will use the principles set out in The Metric Tide in their use of the data. The Metric Tide, published in July 2015, looked in detail at the potential uses and limitations of research metrics and indicators, exploring the use of metrics within institutions and across disciplines. The team at Research England that runs the REF on behalf of the HE funding bodies will also support the panels to make sure the metrics are used responsibly.
REF Director, Dr Kim Hackett, said:
"We are pleased to be working with Clarivate Analytics on the provision of citation information for REF 2021. The use of citation data in this exercise presents a key opportunity to build on the principles of the responsible use of metrics, following the Metric Tide report, and we look forward to working with the ISI team and the panels on this task."
In the interests of transparency, institutions submitting during REF 2021 will be able to view the citation counts for items they plan to submit to the REF in the relevant units of assessment and confirm that a correct match has been obtained.
ISI will build an Application Programming Interface (API) tool for UK universities to provide their assessment data easily and efficiently. To ensure openness and transparency, ISI will also create a web portal based on the API tool that will enable users to review citation data outside of the REF submissions system.
Further information about the provision of citation data will be provided to HEIs in 2019.
Director of ISI, Dr Jonathan Adams, said:
"Clarivate Analytics welcomes the opportunity to work closely with the REF management team and with researchers and research managers across the UK to ensure that data and indicators that support output assessment in the REF are not only of the highest quality and relevance but are also fully understood so they can be used informatively and responsibly."
The award to Clarivate Analytics follows a formal tender process.
Notes
1. Clarivate Analytics has extensive background in bibliometrics (the analysis of publication and citation data) and in UK and global research assessment. ISI was the originator of the Science Citation Index and many of the derived metrics and indicators that underpin modern research assessment. Its staff were involved in background work for RAEs 1992 and 1996, the validation of RAE 2001 and the pilot metrics project run parallel to RAE 2008. They have also been involved in national agency research assessments in other countries.
2. See 'The Metric Tide: Report of the independent review of the role of metrics in research assessment and management’, available at https://responsiblemetrics.org/the-metric-tide/
3. The REF is undertaken by the four UK higher education funding bodies: the Department for the Economy, Northern Ireland(DfE), the Higher Education Funding Council for Wales (HEFCW), Research England, and the Scottish Funding Council (SFC).