Blog: test cycleGAN

paper: CycleGAN
I want to apply cycleGAN on some custom dataset. How much sample is needed? How to handle resolution? What is expected result? What’s the hardware requirement? Is cycleGAN suitable for this task?
Play with cycleGAN’s dataset and have a taste first.
sadly I only have a poor RTX 2070, which only could fit 2 batches.

horse2zebras

Tried pyTorch CycleGAN (Python 3 + pyTorch 1.4) to run horse2zebras, superisingly good :D Expecially horse2zebras.

zebras2horse is relatively bad comparing to horse2zebras, probably because generator try to keep zebras detail? If continus training, the generator of zebras2horse will try to encode the striped coats into somewhere imperceptible to human (and discriminator based on the loss function)?
There is a paper already discussing this issue: CycleGAN, a Master of Steganography
-31/03/2020

apple2orange

horse2zebras have almost same shape, what if dataset have different shape? try apple2orange and see if how well cycleGAN could modifiy the shape…

real	fake	rec	idt

trained 200 epochs, only change color and texture, fail to change shape. :(
Btw, there is some weird apple in the dataset…

Failure cases

According to cycleGAN#failure-cases

On translation tasks that involve color and texture changes, like many of those reported above, the method often succeeds. We have also explored tasks that require geometric changes, with little success.

Other test cases

GANで犬を猫にできるか~cycleGAN編(1)~ tok_msi produced cats ↔ dogs CycleGAN results with a local+global discriminator and a smaller cycle loss.

Possible improvements

Aim: GAN learning shape from unpaired dataset, drop unnessary info

Possible improvement methods/ direction based on above obversations, reviews and other papers.

1. reduce the cycle consistency loss
2. For cycle consistency loss, use perceptual loss such as VGG or discriminator of source domain instead of L1
3. global + local discriminator
4. Attention
5. train small image instead of cropped patch, then increase resolution via pix2pixHD or Progressive GAN

– 04/04/2020