A Brief Summary of My Experience in Understanding and Practicing Deep Learning for Natural Language Processing

[JoyceXFZ]

1. Basic knowledge of vector representations

• Foundation work of Mikolov on vector representations of words and documents
  distributed representations of sentences and documents.pdf
  efficient estimation of word representations in vector space.pdf

• This paper explains the parameter learning processes of word embedding models in details, including the continuous bag-of-word (CBOW) and skip-gram (SG) models, and optimization techniques like the hierarchical softmax and negative sampling, which is a good reference for understanding the model through detailed derivations.
  word2vec parameter learning explained.pdf

• This lecture discusses the basic concept of representing words as vectors and approaches to design word vectors.
  https://youtu.be/ERibwqs9p38

2. Practical training

• Doc2vec tutorial on the lee dataset using Gensim library
  gensim/docs/notebooks/doc2vec-lee.ipynb
• Reuters topic classifier using Keras
  Keras dataset includes 11,228 newswires from Reuters, labeled over 46 topics. Following figures show the results of model training and validation accuracy and loss of the Reuters topic classifier.
  

3. More resources

• Keras Documentations, Gensim Github

comments: NLP is a cool and interesting topic if you get more understanding of it.
ps: I would like to thank Gautier M for giving me some instructions, which is greatly helpful.