Motion Retargeting

LCM

Learning Character-Agnostic Motion for Motion Retargeting in 2D (SIGGRAPH 2019) - Kfir Aberman
Project | pyTorch 0.4
Train a deep neural network to decompose temporal sequences of 2D poses into three components: motion, skeleton, and camera view-angle. Having extracted such a representation, we are able to re-combine motion with novel skeletons and camera views, and decode a retargeted temporal sequence. Pose transfer/ performance cloning is not focus of this paper. It apply Deep Video-Based Performance Cloning directly with skeleton retarget instead of global scaling. img
dataset: synthetic paired data

EDN

Everybody dance now (ICCV 2019)
Project | [not available for public] | reproduce in pyTorch
img
concurrent with vid2vid and LCM

Pose Encoding and Normalization

Encoding body poses + Global pose normalization

Pose to Video Translation

Frame-by-frame synthesis + Temporal smoothing + Face GAN

Dataset

  • target(person): five long single-dancer videos that can be used to train and evaluate our (personalized) model
  • source(motion): large collection of short YouTube videos that can be used for transfer and fake detection.

TransMoMo

TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting (CVPR 2020)
Project | pyTorch
img

  1. Skeleton Extraction: OpenPose or DensePose
  2. Motion Retargeting Network
  3. Skeleton-to-Video Rendering: Everybody Dance Now

comparing to LCM

  • LCM use synthetic paired data, TransMoMo is pure unlabeled web data