Real-time facial expression recognition based on iterative transfer learning and efficient attention network

Kong, Y. and Zhang, S. and Zhang, K. and Ni, Q. and Han, J. (2022) Real-time facial expression recognition based on iterative transfer learning and efficient attention network. IET Image Processing, 16 (6). pp. 1694-1708. ISSN 1751-9659

Abstract

Real-time facial expression recognition is the basis for computers to understand human emotions and detect abnormalities in time. To effectively solve the problems of server overload and privacy information leakage, a real-time facial expression recognition method based on iterative transfer learning and efficient attention network (EAN) for edge resource-constrained scenes is proposed in this paper. Firstly, an EAN is designed with its parameter number and computation amount strictly limited by depth separable convolution and local channel attention mechanism. Then, the soft labels of facial expression data were obtained by EAN based on the idea of knowledge distillation, so as to provide more supervision information for the training process. Finally, an iterative transfer learning method of teacher-student (T-S) network was proposed; it refines the soft labels of the teacher network and further improves the recognition accuracy of the student network. The tests on the public datasets, FER2013 and RAF-DB, show that this method can significantly reduce the model complexity and achieve high recognition accuracy. Compared with other advanced methods, the proposed method strikes a good balance between complexity and accuracy, and well meets the real-time deployment requirements of facial expression recognition technology for edge resource-constrained scenes. © 2022 The Authors. IET Image Processing published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology.