Auditing AI Systems : A Metadata Approach

Adams, Carl and Eslamnejad, Mohsen and Khadka, Anita and M'manga, Andrew and Shaw, Heather and Zhao, Yuchen (2023) Auditing AI Systems : A Metadata Approach. In: Artificial Intelligence XL. SGAI 2023 : 43rd SGAI International Conference on Artificial Intelligence, AI 2023, Cambridge, UK, December 12–14, 2023, Proceedings. Lecture Notes in Computer Science . Springer, Cham, pp. 241-246. ISBN 9783031479939

Abstract

The EU AI regulatory framework and corresponding AI Act, call for stronger'product safety regime'for AI development and set out requirements for more testing, transparency, and impact evaluation in AI based systems, along with significant penalties for corporations that do not follow these requirements. Similar rhetoric is emerging from the UK and USA governments. There is an immediate emerging theme within AI looking at how to test and how to audit compliance within these evolving requirements. This paper presents a metadata model to support auditing compliance and capturing key attributes of bounding of applicability of AI elements to support compliant reuse within AI systems development. The metadata model builds on the IEEE Learning Object Metadata (LOM) model standard to develop the AI-LOM, which provides a base for compliance within the ISTQB'AI Development Framework'covering testing of AI.