48. Conditional GANs
Learning objectives
- Introduce Conditional Generative Adversarial Networks
Conditional GANs
cGANs
“We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This makes it possible to apply the same generic approach to problems that traditionally would require very different loss formulations.”
example
cGAN in ecology
- image source: Hayatbini, et al.
Activity: Pix2Pix Image-to-Image
image-to-image
- app (link) by Christopher Hesse
- GitHub repo
Activity: Pix2Pix Instruct
pix2pix
- app (link) by Tim Brooks
- GitHub repo
Stacked GANs
Stacked GANs
- “Stage-I GAN sketches the primitive shape and colors of the object based on the given text description, yielding Stage-I low-resolution images”
- ” Stage-II GAN takes Stage-I results and text descriptions as inputs, and generates high-resolution images with photo-realistic details”
Zhang, et al., 2016
Architecture
Zhang, et al., 2016
Scaling
Zhang, et al., 2016
Cycle GANs
cGANs
- “… learning to translate an image from a source domain X to a target domain Y in the absence of paired examples …” — Jun-Yan Zhu, et al., 2017
cycle GAN possibilities
Architecture
cGAN Objective
In a cyclic GAN, the generator and discriminator converge toward a Nash equilibrium.
cycle GAN architecture
- images source: Bansal and Rathore
48. Conditional GANs