Skeleton-prompt : A cross-dataset transfer learning approach for skeleton action recognition

Lu, M. and Lu, X. and Liu, J. (2026) Skeleton-prompt : A cross-dataset transfer learning approach for skeleton action recognition. Pattern Recognition, 169: 111885. ISSN 0031-3203

Text (SKELETON_PROMPT)
SKELETON_PROMPT.pdf - Accepted Version
Available under License Creative Commons Attribution.
Download (2MB)

Abstract

This paper presents Skeleton-Prompt, a novel tuning method designed to tackle cross-dataset transfer issues in skeleton action recognition models. Given the scarcity of large-scale 3D skeleton datasets and the variability in keypoint structures across datasets, existing methods often rely on training models from scratch, necessitating extensive labeled data and exhibiting high sensitivity to occlusion. Our approach aims to fine-tune pre-trained models to adapt to limited real-world skeleton data. We use 2D skeletons as inputs and leverage a large human motion dataset for 2D to 3D pose estimation to learn generalizable motion features. A lightweight prompt generator produces instance-level prompts, and we employ dynamic queries with cross-attention to refine the semantic information of the input data. Additionally, we introduce a joint-enhanced multi-stream fusion mechanism based on self-attention to improve robustness against incomplete skeletons. Skeleton-Prompt represents a significant advancement in efficient fine-tuning for skeleton action recognition, effectively addressing cross-dataset generalization challenges in a data-efficient and parameter-efficient manner.