Keuper Labs
← All publications

AAAI · 2021

Spectral distribution aware image generation

Steffen Jung, Margret Keuper

Generative ModelsGANsFrequency Analysis

Why this publication matters

A generated image can fool the eye yet still have an unnatural balance of coarse and fine detail. This method adds a lightweight check for that balance during training. It helps generators reproduce more of the statistical structure of real photographs, beyond what a conventional visual check captures.

Abstract

Recent advances in deep generative models for photo-realistic images have led to high quality visual results. Such models learn to generate data from a given training distribution such that generated images can not be easily distinguished from real images by the human eye. Yet, recent work on the detection of such fake images pointed out that they are actually easily distinguishable by artifacts in their frequency spectra. In this paper, we propose to generate images according to the frequency distribution of the real data by employing a spectral discriminator. The proposed discriminator is lightweight, modular and works stably with different commonly used GAN losses. We show that the resulting models can better generate images with realistic frequency spectra, which are thus harder to detect by this cue.

Abstract source ↗

Figures

Generated portraits and comparisons of their frequency spectra.
Figure 1. (top) The frequency profiles from the FFHQ images are diverse, suggesting that their distribution is not unimodal. (bottom) Average spectral profiles of the real data (FFHQ) after clustering with k-means (k=2). In the highest frequencies, the profiles can be well separated in two clusters. View in source ↗
Average spectral profiles of real data (FFHQ) and data generated by StyleGAN2 (Karras et al.
Figure 2. Average spectral profiles of real data (FFHQ) and data generated by StyleGAN2 (Karras et al. 2019). High frequency components in the generated images indicate grid artifacts or noise, which was not removed by the discriminator. View in source ↗

Cite this paper

Download .bib 
@inproceedings{jung2021spectraldistributionaware8,
  title = {{Spectral distribution aware image generation}},
  author = {Steffen Jung and Margret Keuper},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  url = {https://ojs.aaai.org/index.php/AAAI/article/view/16267/16074}
}

Figures and abstract are reproduced from the linked research sources. Credit remains with the authors and publishers.