Video Synthesis¶
vid2vid¶
Video-to-Video Synthesis (NeurIPS 2018) |
Video-to-Video Synthesis |
github - by Nvidia (Includes works of pix2pixHD and SPADE)
conditional GAN + optical flow given paired data.
Deep Video-Based Performance Cloning¶
Deep Video-Based Performance Cloning (2018/ Eurographics 2019) - Kfir Aberman
conditional GAN + optical flow, concurrent with vid2vid. Used in Learning Character-Agnostic Motion for Motion Retargeting in 2D (Same author)
Everybody Dance Now¶
Few-shot vid2vid¶
Few-shot Video-to-Video Synthesis (NeurIPS 2019) - Nvidia | PyTorch 1.2
Above auto-encoder models (vid2vid, EDN) do not consider generalization to unseen domains.
For example, for pose transfer, above models train with dataset of single person, and the person is remembered by the decoder.
To synthesize a new person, one needs to collect a dataset of the new person and uses it to train a new vid2vid model.
Few-Shot vid2vid can synthesize videos of new persons without re-training by leveraging few example images as input provided during inference.
conditional GAN + attention
Note: it is kinda based on optical flow, not fully generative. The sample output a bit looks like moving and stretching the body part to target location, just like optical flow based frame interpolation.