Basic models

Restricted Boltzann Machine, RBM

shallow, 2 layer neural networks. First is visible layer, second is hidden layer with sigmoid.
probabilistic graphical model models the distribution of data p(x) (not using backpropagation) Get embedding from hidden layer

Deep Belief Network, DBN (2009)

Stacking unsupervised network (RBMs/autoencoders) Add classifier (semi-supervisied, layer-wise pre-training)
The initial weighting is better-> solve local minimum application: classification, collaborative filtering, feature learning (better improve activation function, rarely use now)

Multi. layers perceptron(MLP)

CNN (convolutional neural network)

CNN

RNN

RNN

Generative Models

Generative Models

Siamese network (IJPRAI 1993)

Signature Verification using a “Siamese” Time Delay Neural Network 孿生網路
usually used for comparison

e.g. facenet (with triplet loss, i.e. 3 inputs), DaSIAMPRN

3D convolution

3D Convolutional Neural Networks for Human Action Recognition change color into frame