48. Conditional GANs

Learning objectives

  • Introduce Conditional Generative Adversarial Networks

Conditional GANs

cGANs

“We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations.”

example

cGAN in ecology

Activity: Pix2Pix Image-to-Image

image-to-image

Activity: Pix2Pix Instruct

pix2pix

Stacked GANs

Stacked GANs

  • “Stage-I GAN sketches the primitive shape and colors of the object based on the given text description, yielding Stage-I low-resolution images”
  • ” Stage-II GAN takes Stage-I results and text descriptions as inputs, and generates high-resolution images with photo-realistic details”

Zhang, et al., 2016

Architecture

Zhang, et al., 2016

Scaling

Zhang, et al., 2016

Cycle GANs

cGANs

  • “… learning to translate an image from a source domain \(X\) to a target domain \(Y\) in the absence of paired examples …” — Jun-Yan Zhu, et al., 2017

cycle GAN possibilities

Architecture

cGAN Objective

In a cyclic GAN, the generator and discriminator converge toward a Nash equilibrium.

cycle GAN architecture