Inpaint

Context encoder

Context Encoders: Feature Learning by Inpainting (CVPR 2016)
Project | torch code Semantic Image Inpainting with Deep Generative Models
GAN with encoding manifold, weighted context loss

Globally and Locally Consistent Image Completion

Globally and Locally Consistent Image Completion (SIGGRAPH 2017)
Project

• global discriminator + local(patch) discriminator

Partial Convolutions

Image Inpainting for Irregular Holes Using Partial Convolutions (ECCV 2018) - Nvidia
PyTorch | Playground
Existing deep learning based image inpainting methods use convolutional filter responses conditioned on both valid pixels as well as the substitute values in the masked holes . This often leads to artifacts such as color discrepancy and blurriness. Partial convolutions is masked and renormalized to be conditioned on only valid pixels.
Could also used for general CNN padding Partial Convolution based Padding (2018)

Image synthesis

Stereo Magnification

Stereo Magnification: Learning View Synthesis using Multiplane Images (SIGGRAPH 2018)
Project | google/stereo-magnification tensorflow | Dataset: RealEstate10K

• input: frames triplet with camera trajectories

3D Photo Inpant

3D Photography using Context-aware Layered Depth Inpainting (CVPR 2020)
Project Website | PyTorch | Google Colab

• input: RGB-D iamge

Dynamic Scenes View

Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera (CVPR 2020)
Project - Nvidia | Youtube

• monocular view synthesis of a dynamic scene