A curated list of awesome embedding models tutorials, projects and communities.
Switch branches/tags
Nothing to show
Clone or download
Hironsan Merge pull request #3 from guillaume-chevalier/master
Add 'GloVe as an optimized TensorFlow GPU Layer using chakin'.
Latest commit f44230b Oct 13, 2018



A curated list of awesome embedding models tutorials, projects and communities. Please feel free to pull requests to add links.

Table of Contents


Word Embeddings

Word2vec, GloVe, FastText

  • Efficient Estimation of Word Representations in Vector Space (2013), T. Mikolov et al. [pdf]
  • Distributed Representations of Words and Phrases and their Compositionality (2013), T. Mikolov et al. [pdf]
  • word2vec Parameter Learning Explained (2014), Xin Rong [pdf]
  • word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method (2014), Yoav Goldberg, Omer Levy [pdf]
  • GloVe: Global Vectors for Word Representation (2014), J. Pennington et al. [pdf]
  • Improving Word Representations via Global Context and Multiple Word Prototypes (2012), EH Huang et al. [pdf]
  • Enriching Word Vectors with Subword Information (2016), P. Bojanowski et al. [pdf]
  • Bag of Tricks for Efficient Text Classification (2016), A. Joulin et al. [pdf]

Language Model

  • Semi-supervised sequence tagging with bidirectional language models (2017), Peters, Matthew E., et al. [pdf]
  • Deep contextualized word representations (2018), Peters, Matthew E., et al. [pdf]
  • Contextual String Embeddings for Sequence Labeling (2018). Akbik, Alan, Duncan Blythe, and Roland Vollgraf. [pdf]

Embedding Enhancement

  • Retrofitting Word Vectors to Semantic Lexicons (2014), M. Faruqui et al. [pdf]
  • Better Word Representations with Recursive Neural Networks for Morphology (2013), T.Luong et al. [pdf]
  • Dependency-Based Word Embeddings (2014), Omer Levy, Yoav Goldberg [pdf]
  • Not All Neural Embeddings are Born Equal (2014), F. Hill et al. [pdf]
  • Two/Too Simple Adaptations of Word2Vec for Syntax Problems (2015), W. Ling[pdf]

Comparing count-based vs predict-based method

  • Linguistic Regularities in Sparse and Explicit Word Representations (2014), Omer Levy, Yoav Goldberg[pdf]
  • Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors (2014), M. Baroni [pdf]
  • Improving Distributional Similarity with Lessons Learned from Word Embeddings (2015), Omer Levy [pdf]

Evaluation, Analysis

  • Evaluation methods for unsupervised word embeddings (2015), T. Schnabel [pdf]
  • Intrinsic Evaluation of Word Vectors Fails to Predict Extrinsic Performance (2016), B. Chiu [pdf]
  • Problems With Evaluation of Word Embeddings Using Word Similarity Tasks (2016), M. Faruqui [pdf]
  • Improving Reliability of Word Similarity Evaluation by Redesigning Annotation Task and Performance Measure (2016), Oded Avraham, Yoav Goldberg [pdf]
  • Evaluating Word Embeddings Using a Representative Suite of Practical Tasks (2016), N. Nayak [pdf]

Phrase, Sentence and Document Embeddings



Sense Embeddings

Neural Language Models


Courses and Lectures




Pre-Trained Language Models

Below is pre-trained ELMo models. Adding ELMo to existing NLP systems significantly improves the state-of-the-art for every considered task.

Pre-Trained Word Vectors

Convenient downloader for pre-trained word vectors:

Links for pre-trained word vectors:

Implementations and Tools