Generative adversarial networks (GANs)

What is GAN?

A generative adversarial network (GAN) is a type of construct in neural network technology that offers a lot of potential in the world of artificial intelligence. A generative adversarial network is composed of two neural networks: a generative network and a discriminative network, pitting one against the other (thus the “adversarial”).

The basic idea behind GANs is actually very simple. At its core, a GAN includes two agents with competing objectives that work through opposing goals. This relatively simple setup results in both of the agent's coming up with increasingly complex ways to deceive each other. This kind of situation can be modelled in Game Theory as a minimax game.

What does GAN do?

The main focus for GAN (Generative Adversarial Networks) is to generate data from scratch, mostly images but other domains including music has been done. But the scope of application is far bigger than this. Just like the example below, it generates a zebra from a horse. In reinforcement learning, it helps a robot to learn much faster.

So how is it done technically? 

The discriminator looks at real images (training samples) and generated images separately. It distinguishes whether the input image to the discriminator is real or generated. The output D(X) is the probability that the input x is real, i.e. P(class of input = real image).



We train the discriminator just like a deep network classifier. If the input is real, we want D(x)=1. If it is generated, it should be zero. Through this process, the discriminator identifies features that contribute to real images.

On the other hand, we want the generator to create images with D(x) = 1. So we can train the generator by backpropagation this target value all the way back to the generator, i.e. we train the generator to create images that towards what the discriminator thinks it is real.



We train both networks in alternating steps and lock them into a fierce competition to improve themselves. Eventually, the discriminator identifies the tiny difference between the real and the generated, and the generator creates images that the discriminator cannot tell the difference. The GAN model eventually converges and produces natural look images.

This discriminator concept can be applied to many existing deep learning applications also. The discriminator in GAN acts as a critic. We can plug the discriminator into existing deep learning solutions to provide feedbacks to make it better.

In this code

In the following code, we use MNIST

Github Repo: https://github.com/singhsidhukuldeep/Generative-Adversarial-Network-GAN 

The Paper: https://arxiv.org/pdf/1406.2661.pdf

References

https://en.wikipedia.org/wiki/Generative_adversarial_network

https://blog.paperspace.com/implementing-gans-in-tensorflow/

https://github.com/wiseodd/generative-models

https://wiseodd.github.io/techblog/2016/09/17/gan-tensorflow/

https://skymind.ai/wiki/generative-adversarial-network-gan