Personalized emotion recognition by personality-aware high-order learning of physiological signals

Zhao, S. and Gholaminejad, A. and Ding, G. and Gao, Y. and Han, J. and Keutzer, K. (2019) Personalized emotion recognition by personality-aware high-order learning of physiological signals. ACM Transactions on Multimedia Computing, Communications, and Applications, 15 (1S): 14. ISSN 1551-6857

[thumbnail of PersonalityTOMM]
Preview
PDF (PersonalityTOMM)
PersonalityTOMM.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.

Download (1MB)

Abstract

Due to the subjective responses of different subjects to physical stimuli, emotion recognition methodologies from physiological signals are increasingly becoming personalized. Existing works mainly focused on modeling the involved physiological corpus of each subject, without considering the psychological factors, such as interest and personality. The latent correlation among different subjects has also been rarely examined. In this article, we propose to investigate the influence of personality on emotional behavior in a hypergraph learning framework. Assuming that each vertex is a compound tuple (subject, stimuli), multi-modal hyper-graphs can be constructed based on the personality correlation among different subjects and on the physiological correlation among corresponding stimuli. To reveal the different importance of vertices, hyperedges, and modalities, we learn the weights for each of them. As the hypergraphs connect different subjects on the compound vertices, the emotions of multiple subjects can be simultaneously recognized. In this way, the constructed hypergraphs are vertex-weighted multi-modal multi-task ones. The estimated factors, referred to as emotion relevance, are employed for emotion recognition. We carry out extensive experiments on the ASCERTAIN dataset and the results demonstrate the superiority of the proposed method, as compared to the state-of-the-art emotion recognition approaches.

Item Type:
Journal Article
Journal or Publication Title:
ACM Transactions on Multimedia Computing, Communications, and Applications
Additional Information:
© ACM, 2018. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) - Special Section on Deep Learning for Intelligent Multimedia Analytics and Special Section on Multi-Modal Understanding of Social, Affective and Subjective Attributes of Data, 15, 1s, 2019 http://doi.acm.org/10.1145/3233184
Uncontrolled Keywords:
/dk/atira/pure/subjectarea/asjc/1700/1708
Subjects:
?? hypergraph learningmulti-modal fusionpersonality-sensitive learningpersonalized emotion recognitionphysiological signal analysisbiomedical signal processingphysiological modelsphysiologyspeech recognitionhypergraphphysiological signalsgraph theoryhardware ??
ID Code:
131551
Deposited By:
Deposited On:
08 Mar 2019 16:25
Refereed?:
Yes
Published?:
Published
Last Modified:
16 Dec 2023 01:30