Applying Delaunay triangulation augmentation for deep learning facial expression generation and recognition

Valev, Hristo and Gallucci, Alessio and Leufkens, Tim and Westerink, Joyce and Sas, Corina (2021) Applying Delaunay triangulation augmentation for deep learning facial expression generation and recognition. In: Pattern Recognition. ICPR International Workshops and Challenges. Lecture Notes in Computer Science . Springer, Cham, pp. 730-740. ISBN 9783030687953

Abstract

Generating and recognizing facial expressions has numerous applications, however, those are limited by the scarcity of datasets containing labeled nuanced expressions. In this paper, we describe the use of Delaunay triangulation combined with simple morphing techniques to blend images of faces, which allows us to create and automatically label facial expressions portraying controllable intensities of emotion. We have applied this approach on the RafD dataset consisting of 67 participants and 8 categorical emotions and evaluated the augmentation in a facial expression generation and recognition tasks using deep learning models. For the generation task, we used a deconvolution neural network which learns to encode the input images in a high-dimensional feature space and generate realistic expressions at varying intensities. The augmentation significantly improves the quality of images compared to previous comparable experiments and it allows to create images with a higher resolution. For the recognition task, we evaluated pre-trained Densenet121 and Resnet50 networks with either the original or augmented dataset. Our results indicate that the augmentation alone has a similar or better performance compared to the original. Implications of this method and its role in improving existing facial expression generation and recognition approaches are discussed.