REMOTE : Reinforced Motion Transformation Network for Semi-supervised 2D Pose Estimation in Videos

Ma, Xianzheng and Rahmani, Hossein and Fan, Zhipeng and Yang, Bin and Cheng, Jun and Liu, Jun (2022) REMOTE : Reinforced Motion Transformation Network for Semi-supervised 2D Pose Estimation in Videos. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence :. AAAI press, Palo Alto, Calif., pp. 1944-1952. ISBN 9781577358763

Text (AAAI_22_Pose_Estimation)
AAAI_22_Pose_Estimation.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.
Download (5MB)

Abstract

Existing approaches for 2D pose estimation in videos often require a large number of dense annotations, which are costly and labor intensive to acquire. In this paper, we propose a semi-supervised REinforced MOtion Transformation nEtwork (REMOTE) to leverage a few labeled frames and temporal pose variations in videos, which enables effective learning of 2D pose estimation in sparsely annotated videos. Specifically, we introduce a Motion Transformer (MT) module to perform cross frame reconstruction, aiming to learn motion dynamic knowledge in videos. Besides, a novel reinforcement learning-based Frame Selection Agent (FSA) is designed within our framework, which is able to harness informative frame pairs on the fly to enhance the pose estimator under our cross reconstruction mechanism. We conduct extensive experiments that show the efficacy of our proposed REMOTE framework.