Super-resolution

some cross-project repo: Martin Krasser/Keras-super-resolution: SRGAN, EDSR, WDSR
open-mmlab/MMSR: SRResNet, SRGAN, ESRGAN, EDVR, etc.

SRCNN

Image Super-Resolution Using Deep Convolutional Networks (TPAMI 2014) by CUHK
Project - include Matlab and Caffe code
first upscale it to the desired size using bicubic interpolation

FSRCNN

Accelerating the Super-Resolution Convolutional Neural Network (ECCV 2016) by CUHK
use deconvolution to upscale instead of pre-processing

VDSR

Accurate Image Super-Resolution Using Very Deep Convolutional Networks (CVPR 2016)
start to use residual, one long skip-connection
../../_images/VDSR.png
Same as SRCNN, it take upscaled LR image as imput

SRGAN

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (CVPR 2017) by Twitter
Tensorflow 2.0
SRResNet + long skip-connection, diverge from MSE novel perceptual loss using high-level feature maps of VGG network

EDSR

Enhanced Deep Residual Networks for Single Image Super-Resolution (CVPR 2017) remove batch normalization of SRResNet→ save resource spend resource on convolution instead

MDSR

multiscale architecture that reconstructs various scales of high-resolution images in a single model

WDSR

Wide Activation for Efficient and Accurate Image Super-Resolution (CVPR Workshop 2018)
Official pyTorch (based on EDSR) | tensorflow ../../_images/WDSR_block.png

  • increase channel before ReLU
  • linear low-rank convolution stack and large channel

SFTGAN

Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform (CVPR 2018) - SenseTime + CUHK
pyTorch + Lua | STF Layer

  • Spatial Feature Transform: learns a mapping function \(M\) that outputs a modulation parameter pair (γ, β) based on some prior condition Ψ (Segmentation probability maps)
  • generate affine transformation parameters for spatial-wise (pixel-wise) feature modulation
    ../../_images/SFTGAN_architecture.png ../../_images/SFTGAN.png

RDN

Residual Dense Network for Image Super-Resolution (CVPR 2018)

  • Residual + Dense block
  • local feature fusion + global feature fusion -> contiguous memory mechanism used in video debluring/BIN

ESRGAN

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks (ECCV 2018) - CUHK

  • RRDB (Residual in Residual Dense Blcok)
  • enhance adversarial loss and perceptual loss
  • relativistic GAN
  • improve the perceptual loss by using the features before activation