NLP

embedding

word2vec (NIPS 2013)

Distributed Representations of Words and Phrases and their Compositionality Efficient estimation of word representations in vector space google code archive

word2vec Parameter Learning Explained

The Illustrated Word2vec

ELMo (NAACL 2018)

Deep contextualized word representations
code
LSTM

Transformer (NIPS 2017)

Attention is all you need

The Illustrated Transformer

BERT (2018)

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Google AI Blog: Open Sourcing BERT: State-of-the-Art Pre-training for Natural Language Processing

[R] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)

Tasks:

1. Sentence Pair Classification
2. Single Sentence Classification
3. Question Answering
4. Single Sentence Tagging