We describe a new training methodology for generative adversarial networks. The key idea is to grow both the generator and discriminator progressively: starting from a low resolution, we add new layers that model increasingly fine details as training progresses. This both speeds up the training and greatly stabilizes it, allowing us to produce images of unprecedented quality, e.g., CelebA images at 1024 x 1024. We also propose a simple way to increase the variation in generated images, and achieve a record inception score of 8.80 in unsupervised CIFAR10. Additionally, we describe several implementation details that are important for discouraging unhealthy competition between the generator and discriminator.

Generative adversarial networks produce sharp images but are notoriously difficult to train. The generated image quality and training stability are typically inversely correlated with the target resolution — higher resolutions require larger networks, which amplify the instabilities inherent in GAN training. We observe that learning large-scale structure is fundamentally easier than fine details. By progressively growing the networks from low to high resolution, we allow the networks to first discover the broad spatial structure of the image distribution and then shift attention to increasingly fine-scale detail. This is in contrast to the typical approach of training all layers simultaneously.

Since the introduction of GANs, considerable effort has gone into improving training stability. Wasserstein GAN (WGAN) and its gradient penalty variant WGAN-GP address mode collapse and training divergence through modified loss functions. Spectral normalization constrains discriminator weights for stability. On the generation side, LAPGAN uses a Laplacian pyramid framework to generate images in a coarse-to-fine manner, but requires separate models at each scale. Our approach differs in that we train a single growing network end-to-end, where new layers are smoothly faded in, avoiding the discontinuities of multi-stage training.

Both networks start at 4 x 4 resolution and are progressively grown by adding new layers during training. When a new layer is added, we use a smooth fade-in mechanism: the new layer's contribution is gradually increased from 0 to 1 using a linearly interpolated residual connection. This prevents sudden shocks to the already well-trained lower layers. The generator and discriminator are mirror images of each other, using replicated 3-layer blocks at each resolution. This progressive approach has a key advantage: the networks are trained on simpler data distributions initially (low-resolution images exhibit less variation), which stabilizes training. The approach also provides a 2-6x speed improvement depending on target resolution.

We introduce several techniques to improve training. Minibatch standard deviation: instead of the complex minibatch discrimination approach, we append a simple statistic — the average standard deviation across all features and spatial locations — as an additional feature map to the discriminator. This encourages the generator to produce varied outputs without learnable parameters. Equalized learning rate: we scale the weights at runtime using the He initializer constant, ensuring that the learning speed is the same for all parameters regardless of their shape. Pixelwise feature normalization: we normalize each feature vector in the generator to unit length after each convolutional layer, preventing signal magnitudes from escalating.

We evaluate on CelebA-HQ (1024 x 1024), LSUN categories (256 x 256), and CIFAR10. For CelebA-HQ, we created a high-quality version of CelebA with 30,000 images at 1024 x 1024 resolution through a pipeline of JPEG artifact removal, 4x super-resolution, and facial landmark-based cropping. As a new evaluation metric, we propose Sliced Wasserstein Distance (SWD), which measures patch-level statistical similarity between generated and real images using a Laplacian pyramid decomposition. On CIFAR10, we achieve an inception score of 8.80, the highest reported for unsupervised methods. The generated CelebA-HQ faces at 1024 x 1024 show unprecedented quality with fine details such as hair strands, skin pores, and background elements.

We have described progressive growing of GANs, a training methodology where both networks are grown incrementally from low to high resolution. This has several benefits: faster and more stable training, higher quality results, and the ability to produce megapixel-scale images. The accompanying techniques — minibatch standard deviation, equalized learning rate, and pixelwise normalization — further improve the training dynamics. We have also introduced the CelebA-HQ dataset and the Sliced Wasserstein Distance metric to facilitate future research in high-resolution image synthesis. We believe that progressive training is a broadly applicable principle that can benefit many generative model architectures.