Human Action Recognition from Various Data Modalities:A Review

Sun, Zehua and Ke, Qiuhong and Rahmani, Hossein and Bennamoun, Mohammed and Wang, Gang and Liu, Jun (2022) Human Action Recognition from Various Data Modalities:A Review. IEEE Transactions on Pattern Analysis and Machine Intelligence. ISSN 0162-8828

Text (2012.11866)
2012.11866.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.

Download (3MB)


Human Action Recognition (HAR), aiming to understand human behaviors and then assign category labels, has a wide range of applications, and thus has been attracting increasing attention in the field of computer vision. Generally, human actions can be represented using various data modalities, such as RGB, skeleton, depth, infrared sequence, point cloud, event stream, audio, acceleration, radar, and WiFi, etc., which encode different sources of useful yet distinct information and have various advantages and application scenarios. Consequently, lots of existing works have attempted to investigate different types of approaches for HAR using various modalities. In this paper, we give a comprehensive survey for HAR from the perspective of the input data modalities. Specifically, we review both the hand-crafted feature-based and deep learning-based methods for single data modalities, and also review the methods based on multiple modalities, including the fusion-based frameworks and the co-learning-based approaches. The current benchmark datasets for HAR are also introduced. Finally, we discuss some potentially important research directions in this area.

Item Type:
Journal Article
Journal or Publication Title:
IEEE Transactions on Pattern Analysis and Machine Intelligence
Additional Information:
©2022 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Uncontrolled Keywords:
ID Code:
Deposited By:
Deposited On:
22 Jun 2022 10:05
Last Modified:
22 Nov 2022 11:34