Generating Multiple 4D Expression Transitions by Learning Face Landmark Trajectories - École nationale supérieure Mines-Télécom Lille Douai Accéder directement au contenu
Article Dans Une Revue IEEE Transactions on Affective Computing Année : 2023

Generating Multiple 4D Expression Transitions by Learning Face Landmark Trajectories

Résumé

In this work, we address the problem of 4D facial expressions generation. This is usually addressed by animating a neutral 3D face to reach an expression peak, and then get back to the neutral state. In the real world though, people show more complex expressions, and switch from one expression to another. We thus propose a new model that generates transitions between different expressions, and synthesizes long and composed 4D expressions. This involves three sub-problems: (i) modeling the temporal dynamics of expressions, (ii) learning transitions between them, and (iii) deforming a generic mesh. We propose to encode the temporal evolution of expressions using the motion of a set of 3D landmarks, that we learn to generate by training a manifold-valued GAN (Motion3DGAN). To allow the generation of composed expressions, this model accepts two labels encoding the starting and the ending expressions. The final sequence of meshes is generated by a Sparse2Dense mesh Decoder (S2D-Dec) that maps the landmark displacements to a dense, per-vertex displacement of a known mesh topology. By explicitly working with motion trajectories, the model is totally independent from the identity. Extensive experiments on five public datasets show that our proposed approach brings significant improvements with respect to previous solutions, while retaining good generalization to unseen data.
Fichier principal
Vignette du fichier
4D_FacialExpressionGeneration_AffectiveComputing-2.pdf (3.91 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04100664 , version 1 (18-05-2023)

Identifiants

  • HAL Id : hal-04100664 , version 1

Citer

Naima Otberdout, Claudio Ferrari, Mohamed Daoudi, Stefano Berretti, Alberto Del Bimbo. Generating Multiple 4D Expression Transitions by Learning Face Landmark Trajectories. IEEE Transactions on Affective Computing, In press. ⟨hal-04100664⟩
34 Consultations
101 Téléchargements

Partager

Gmail Facebook X LinkedIn More