Deep Learning NLP Tutorial from Scratch

Feedback: sungjin7127@gmail.com

To-Do

##Tensorflow Tutorial (1.4 or Later)

• Conversation-Tensorflow
• Hvass Tensorflow Tutorial

Text Classificaiton

• English

  • Sequence Classification with LSTM Recurrent Neural Networks in Python with Keras - Check

  • Text Classificaiton Using sentence level Attentional RNN - Check

  • Text Classification Using Hierarchical attention network

  • Text Classificaion Using CNN with Keras - Complete

  • Text Classification Using CNN with Tensorflow - Complete

    • Paper: CNN for Text Classification, Yoon Kim: Complete - Complete
    • Paper: Character-Aware Neural Language Models, Yoon Kim

• Korean

  • Domain / Intend Classification Using Deep Learning & Ensemble Model (95+% Accuracy) - On_Progress

Sequence-to-Sequence

• Paper
  • Convolutional_Seq2Seq
  • Google NMT

Text Summarization

• English
  • Sequence Classification with LSTM Recurrent Neural Networks in Python with Keras - TODO

Machine Comprehension

• Dataset

  • NewsQA: A Machine Comprehension Dataset - Maluuba Research (Aquired by MS in 2017)

  • The Stanford Question Answering Dataset - SQuAD - - Check

  • MS MACRO Dataset -Dataset for Deep Learning NLP - - Check

• Source Code - English

  • Bi-directional Attention Flow for Machine Comprehension - Minjoon Seo - Check
  • BM25
  • Machine Learning Knowledge Exchange
  • Machine Comprehension - IBM
  • Teaching Machines to Read and Comprehend (Theano) - Deep Mind
  • Teaching Machines to Read and Comprehend (Theano/Lasagne)
  • Attentive Reader(Tensorflow) - carpedm20
  • SeqMacthSeq(Torch) - shuohangwang
  • Machine Comprehension in Keras(Teaching Machines to Read and Comprehend)

Keras Tutorial

• Keras Documentation - Check
• Keras Github - Check
• Keras Tutorial - Karlijn Willems
• Keras Cheat Sheet - Karlijn Willems: Complete
• Deep Learning in Python(Keras) - Dan Becker: Complete

NLP Tutorial

• Deep Learning for NLP - Oxford (Deep Mind) - On_Progress
• Deep Learning for NLP 224d - Stanford - On_Progress

Word Embedding

• Dataset
  • Fasttext (Eng, Kor, etc) - Facebook - Check
  • Glove: Pre-trained Model - Stanford - Check
  • Word2Vec: Pre-trained model from Google News (100 billion words) - Check

Song Lyrics Generator (Inspired by Siraj Raval)

• Generate Rap Lyrics - Siraj Raval

  • Sample Source Code

• DeepBeat - Sample Website

• Paper

  • Unsupervised Rhyme Scheme Identification in Hip Hop Lyrics Using HMM - HongKong Univ.
  • DopeLearning: A Computational Approach to Rap Lyrics Generation

##Pytorch

• Practial-Pytorch (NLP related)
• the-incredible-pytorch

##Python

##Scala

• Tutorial
  • Programming in scala Summary
  • Scala School

---

Reference

Data Preprocessing

• Chris ALBon: Python/R

Machine Learning / Deep Learning

• Deep Learning Zero To All: Sung Kim
• Deep Learning Tutorial for 10 weeks- sjchoi86
• Step-by-Step Guide: ML Python
• Awesome Code
• Xavier initialization

Natural Language Procesing

• Datasets for NLP: QA/Dialog System/Goal-Oriented Dialogue Systems
• Topic Modeling for Python (Korean & English)
• ParlAI: Dialog Research Open source from Facebook
• AwesomeNLP: 자연어처리 관련 자료 모음

MNIST Training Model by TF-KR

• how-far-can-we-go-with-MNIST