Recent advances in Generative Adversarial Networks (GANs) have shown impressive results for the task of facial expression synthesis. The most successful approaches are conditioned on discrete expressions or action units, which limits the variety of generated facial animations. In this paper, we introduce a novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining human expressions. Our approach allows controlling the magnitude of activation of each AU and combining several of them. We also propose a fully unsupervised strategy that only requires images annotated with their activated AUs and exploit attention mechanisms that make our network robust to changing backgrounds and lighting.

Facial expression generation has been a long-standing problem in computer vision and graphics. Previous GAN-based methods like StarGAN treat facial expressions as discrete categories (e.g., happy, sad, angry), which cannot capture the richness and subtlety of human expressions. In contrast, the Facial Action Coding System (FACS) describes expressions through combinations of Action Units (AUs) — elementary facial muscle movements that can vary in intensity.

We propose GANimation, a GAN architecture that can generate anatomically-aware facial animations from a single image. Our key contributions are: (1) a conditioning scheme based on continuous AU activations rather than discrete labels; (2) an attention mechanism that focuses changes on face regions while preserving the background; and (3) a fully unsupervised training procedure that does not require paired data of the same person with different expressions.

Conditional image generation has advanced rapidly with models like Pix2Pix, CycleGAN, and StarGAN. For facial expression manipulation, most methods require paired training data or are limited to discrete expression categories. The Facial Action Coding System (FACS), developed by Ekman and Friesen, defines 30+ Action Units that can describe virtually any anatomically possible facial expression. AU detection has matured as a field, providing reliable AU annotations that we leverage as conditioning signals.

The generator G takes as input an image I and a target AU vector y_t, and produces the output image G(I | y_t). Critically, the generator produces two outputs: a color mask C and an attention mask A. The final image is computed as I_out = A * C + (1 - A) * I, where the attention mask determines which regions should change. This attention mechanism preserves the background and focuses changes on the face, resulting in more realistic and artifact-free results.

The training objective combines four losses: (1) an adversarial loss with a conditional discriminator D(I, y) that evaluates both realism and AU consistency; (2) a cycle consistency loss that ensures applying the inverse AU transformation recovers the original image; (3) a self-reconstruction loss that enforces identity preservation when conditioning on the same AUs; and (4) an attention regularization loss that prevents the attention mask from becoming trivially all ones.

We evaluate GANimation on the EmotioNet dataset containing over one million in-the-wild face images with AU annotations. Qualitative results demonstrate that our method generates high-quality facial animations with smooth transitions between different AU activations. Compared to StarGAN, our method produces fewer visual artifacts, better preserves identity, and enables continuous expression control rather than being limited to discrete categories.

Quantitative evaluation using Amazon Mechanical Turk user studies shows that our method achieves 68.2% preference rate over StarGAN in terms of expression accuracy and 73.1% preference rate in terms of image quality. The attention mechanism ablation confirms its critical role: removing it increases background artifacts by 47% and reduces identity preservation scores significantly.

We have introduced GANimation, a novel approach for anatomically-aware facial animation that operates in a continuous AU space. Our attention-based generator enables fine-grained control over facial expressions while preserving identity and background. The fully unsupervised training makes our method practical for real-world applications. Future work includes extending the framework to video sequences and full-body animation.