16 September 2020
So, how do you know when the size of a unit’s output pool has been disproportionately impacted? This is a question which the REF Equality & Diversity Advisory Panel (EDAP) tussled with when we started to assess unit rationale statements earlier this year.
One of the measures to support equality and diversity in REF 2021 was to allow institutions to request a reduction in the number of outputs that had to be submitted in one or more of its submitting units, where it was felt that equality-related circumstances had a disproportionate impact on the size of the available output pool. Given the flexibility offered by the decoupling of staff and outputs, however, it was expected that institutions would not routinely need to submit unit reduction requests. Individual institutions were asked to set out processes for determining whether any of their units had been disproportionately affected.
During the development of the REF guidance, EDAP thought long and hard about whether one or more specific criteria should be set for determining disproportionality. For example, we considered whether we should stipulate the minimum proportion of staff in the unit with declared applicable circumstances. However, we were mindful of the very different size and make-up of institutions across the sector, and the individual units therein, and felt that setting a minimum proportion would not sufficiently account for the variability in the nature of circumstances experienced and their effects on different units. Hence, the Guidance on Submissions did not include specific criteria that had to be met. It is not surprising, therefore, that institutions applied different strategies in determining how many requests to make, and on what basis. These varying strategies resulted in a wide disparity in the number of unit requests made by any one institution, which ranged from zero to 100% of UOAs in which an institution was planning to submit. Given the nature of the guidance, EDAP felt it was important to judge each case on the basis of the evidence provided, irrespective of the particular strategy adopted by the institution.
Where a unit reduction request was made, institutions had to provide a statement outlining the context of the unit, how the declared circumstances affected the unit’s output pool, and how the HEI determined this in accordance with their REF Code of Practice. In making its judgements on unit rationale statements, EDAP considered the size of the unit (both in terms of headcount and FTE), the proportion of staff who declared, the potential impact on other staff members, the anticipated reduction to the size of the available output pool, and any relevant factors linked to academic discipline or subject field.
Through the process of calibration, to develop a common understanding of disproportionality, EDAP found it helpful to explicate some broad rules of thumb. We agreed that these would primarily be used for sense checking assessments we had made, rather than for driving initial decision making. Broadly speaking, the panel agreed that where 15% or more of staff members in a unit had declared eligible circumstances, and clear information was provided in the rationale statement, we would accept that a case for disproportionality had been made. In contrast, where the staff declaration rate was 10% or lower, EDAP felt that the impact could be accommodated with the flexibility provided by the decoupling of staff and outputs, unless very exceptional factors had been raised. For cases where declaration rates fell between these levels, the panel’s judgements were primarily influenced by factors outlined in the rationale statement, such as the size of the unit, the proportion of staff who were working part-time (particularly at low FTEs), and the publication profile of the discipline.
Of the 461 rationale statements assessed by EDAP, 88% were judged to have made a clear case for the cumulative effect of circumstances having disproportionately impacted on the size of the output pool. For the remaining 12%, it was agreed that the impact of the circumstances could be accommodated within the flexibility provided by decoupling. In the latter case, it was expected that institutions would continue to provide support for those staff members with equality-related circumstances. So, does a 88% acceptance rate sound about right? It probably does, given our strong commitment to promoting and supporting equality and diversity in REF 2021, whilst recognising the increased flexibility provided by the decoupling of staff and outputs in the current exercise.
Alongside the publication of this blog, and EDAP’s more extensive Interim Report to the sector, which can be found here, institutions will receive feedback on why any of their unit reduction requests were not accepted by the panel. This should enable them to amend their submissions ahead of the revised submission deadline at the end of March 2021. More generally, the broader feedback provided in this blog and, particularly, the Interim Report should help all institutions to prepare their final submissions.