Measuring the impartiality of UK parliamentary research services using automatic annotation and corpus linguistics

Sun, Xinmei (2025) Measuring the impartiality of UK parliamentary research services using automatic annotation and corpus linguistics. In: Corpus Linguistics 2025, 2025-06-30 - 2025-07-03.

Abstract

The UK House of Commons Library aims to provide impartial information services for MPs and MPs’ staff regardless of their political affiliation, thereby helping them “scrutinise legislation, prepare for debates, develop policies” (Commons Library, n.d.). Their briefing reports are described as the “first port of call” and an aid to decision-making by MPs (Kenny et al., 2017). Given the crucial part that the Commons Library plays in legal and political decision-making, it is critical to assess whether they are indeed impartial as mandated. This project develops and tests a way of measuring impartiality in texts using automatic annotation and corpus tools. It then applies this method to determine the impartiality of a corpus of briefing reports on the topic of immigration published between 1999-2024. Impartiality is defined as the presentation of information without privileging a particular party or expressing personal opinions. Drawing from cognitive linguistics, it is approached through framing and epistemic positioning. This poster presents the first stage in this project and assesses semantic frame annotation tools trained using FrameNet (Ruppenhofer et al., 2016). I consider: 1) the reliability of the automatic annotation of semantic frames and frame elements when compared against manual annotation in a sample of sentences from the briefing reports; 2) how the tools’ performance can be enhanced with limited manual annotation resources; and 3) the potential benefits of using semantic frames as the analytical unit (as opposed to words). Using frequency measure and concordancing, I consider: 4) the distribution of competing frames on a policy issue; 5) the treatment of sources with different political affiliations; 6) the use of overt evaluations. This poster will discuss the implications of the findings for impartiality and how corpus linguistics can contribute to the development of a paradigm for monitoring impartiality.