Lancaster EPrints

Performance metrics for activity recognition

Ward, Jamie and Lukowicz, Paul and Gellersen, Hans (2011) Performance metrics for activity recognition. ACM Transactions on Intelligent Systems and Technology (TIST), 2 (1). ISSN 2157-6904

Full text not available from this repository.

Abstract

In this article, we introduce and evaluate a comprehensive set of performance metrics and visualisations for continuous activity recognition (AR). We demonstrate how standard evaluation methods, often borrowed from related pattern recognition problems, fail to capture common artefacts found in continuous AR—specifically event fragmentation, event merging and timing offsets. We support our assertion with an analysis on a set of recently published AR papers. Building on an earlier initial work on the topic, we develop a frame-based visualisation and corresponding set of class-skew invariant metrics for the one class versus all evaluation. These are complemented by a new complete set of event-based metrics that allow a quick graphical representation of system performance—showing events that are correct, inserted, deleted, fragmented, merged and those which are both fragmented and merged. We evaluate the utility of our approach through comparison with standard metrics on data from three different published experiments. This shows that where event- and frame-based precision and recall lead to an ambiguous interpretation of results in some cases, the proposed metrics provide a consistently unambiguous explanation.

Item Type: Article
Journal or Publication Title: ACM Transactions on Intelligent Systems and Technology (TIST)
Uncontrolled Keywords: Activity recognition ; Metrics ; Performance evaluation
Subjects:
Departments: Faculty of Science and Technology > School of Computing & Communications
ID Code: 53444
Deposited By: ep_importer_pure
Deposited On: 12 Apr 2012 11:22
Refereed?: Yes
Published?: Published
Last Modified: 09 Apr 2014 23:17
Identification Number:
URI: http://eprints.lancs.ac.uk/id/eprint/53444

Actions (login required)

View Item