The State-of-the-Art in Lifelog Retrieval : A Review of Progress at the ACM Lifelog Search Challenge Workshop 2022-24

Tran, Allie and Bailer, Werner and Dang-Nguyen, Duc-Tien and Healy, Graham and Hodges, Steve and Jónsson, Björn Þór and Rossetto, Luca and Schoeffmann, Klaus and Tran, Minh-Triet and Vadicamo, Lucia and Gurrin, Cathal (2025) The State-of-the-Art in Lifelog Retrieval : A Review of Progress at the ACM Lifelog Search Challenge Workshop 2022-24. IEEE Access, 13. pp. 216340-216363. ISSN 2169-3536

Abstract

The ACM Lifelog Search Challenge (LSC) is the only long-running benchmark that evaluates lifelog retrieval systems through real-time, human-in-the-loop interaction. This paper presents a longitudinal analysis of technical and interaction design progress at LSC from 2022 to 2024, conducted under fixed datasets, tasks, and evaluation protocols. Unlike prior surveys of lifelogging research, our study leverages competitive, synchronous evaluations to examine how retrieval architectures, interface design choices, and interaction strategies directly influence task performance. Through a comparative analysis of known-item search, question answering, and ad-hoc retrieval tasks, we identify a clear paradigm shift from concept-based pipelines toward embedding-driven and LLM-supported retrieval systems. We show how these shifts affect retrieval effectiveness, response time, and user behaviour, as well as provide empirical insight into how retrieval and interface design choices manifest in performance during interactive search. Beyond documenting emerging techniques, we expose evaluation artefacts such as performance variability across system instances and user familiarity effects, raising important considerations for future interactive benchmarks.