A curated list of resources about natural language processing (NLP).
Maintained by Seonghan Ryu.
Inspired by the awsome lists.
Video Lectures
- Natural Language Processing with Deep Learning, Chris Manning and Richard Socher, Stanford University, 2017
- Deep Natural Language Processing, Phil Blunsom, Oxford University, 2017
- Deep Learning for NLP, Nils Reimers, 2015
- Natural Language Processing, Dan Jurafsky & Chris Manning, Coursera, 2012
Books
- Natural Language Understanding with Distributed Representation, Kyunghyun Cho, arXiv, 2016
- A Primer on Neural Network Models for Natural Language Processing, Yoav Goldberg, arXiv, 2015
- Spoken Language Understanding, Gokhan Tur and Renato De Mori, Wiley, 2012
- Speech and Language Processing, Daniel Jurafsky and James H. Martin, Prentice Hall, 2008
- Foundations of Statistical Natural Language Processing, Chris Manning and Hinrich Schütze, MIT Press, 1999
Related Conferences & Workshops
- Annual Meeting of the Association for Computational Linguistics (ACL)
- Conference on Empirical Methods in Natural Language Processing (EMNLP)
- ACM International Conference on Research and Development in Information Retrieval (SIGIR)
- International World Wide Web Conference (WWW)
- European Chapter of the Association for Computational Linguistics (EACL)
- Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)
- International Conference on Computational Linguistics (COLING)
- International Joint Conference on Natural Language Processing (IJCNLP)
- Interspeech
- IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)
- IEEE Workshop on Spoken Language Technology (SLT)
- Annual SIGdial Meeting on Discourse and Dialogue (SIGdial)
- International Conference on Language Resources and Evaluation (LREC)
- International Workshop on Spoken Dialog Systems (IWSDS)
Shared Tasks
- The SIGNLL Conference on Computational Natural Language Learning (CoNLL)
- Semantic Evaluation (SemEval)
- Text REtrieval Conference (TREC)
- Conference and Labs of the Evaluation Forum (CLEF)
- NII Testbeds and Community for Information access Research (NTCIR)
- Dialog State Tracking Challenge (DSTC)
Reference
Reviews & Tutorials
- Recent Trends in Deep Learning Based Natural Language Processing, T. Young et al., arXiv, 2017
- Generating Sentences from a Continuous Space, CONLL, 2016 #VAE
- Text Classification, Part 3 - Hierarchical attention network, Richard, Personal Blog, 2016
- Text Classification, Part 2 - sentence level Attentional RNN, Richard, Personal Blog, 2016
- Practical seq2seq, Suriyadeepan Ramamoorthy, Personal Blog, 2016
- Recurrent Neural Networks Tutorial [Part 1] [Part 2] [Part 3] [Part 4], Denny Britz, WildML, 2015 [Code]
- Implementing a CNN for Text Classification in TensorFlow, Denny Britz, WildML, 2015 [Code]
- Understanding Convolutional Neural Networks for NLP, Denny Britz, WildML, 2015
Reviews & Tutorials
- Representations for Language: From Word Embeddings to Sentence Meanings, Christopher Manning, Representation Learning Workshop, 2017
- word2vec 관련 이론 정리, Beomsu Kim, Personal Blog, 2016
- On Word Embedding [Part 1] [Part 2] [Part 3], Sebastian Ruder, Personal Blog, 2015
Research Papers
- A Simple But Tough-To-Beat Baseline For Sentence Embeddings, Sanjeev Arora et al., ICLR, 2017
- Swivel: Improving Embeddings by Noticing What's Missing, Noam Shazeer et al., arxiv, 2016 [Code] #Google
- Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks, Kai Sheng Tai et al., ACL, 2015
- A Hierarchical Neural Autoencoder for Paragraphs and Documents, Jiwei Li et al., ACL, 2015 [Code]
- Improving Distributional Similarity with Lessons Learned from Word Embeddings, Omer Levy et al., TACL, 2015.
- From Word Embeddings To Document Distances, Matt J. Kusner et al., ICML, 2015
- Semi-supervised Sequence Learning, Andrew M. Dai et al., NIPS, 2015
- Distributed Representations of Sentences and Document (Paragraph Vector), Quoc V. Le et al., ICML, 2014
- Glove: Global Vectors for Word Representation, Jeffrey Pennington et al., EMNLP, 2014 [GloVe]
- Learning Deep Structured Semantic Models for Web Search using Clickthrough Data, Po-Sen Huang et al., CIKM, 2013
- Learning word embeddings efficiently with noise-contrastive estimation, Andriy Mnih et al., NIPS, 2013
- Linguistic Regularities in Continuous Space Word Representations, Tomas Mikolov et al., NAACL-HLT, 2013
- Distributed Representations of Words and Phrases and their Compositionality (Negative Sampling), Tomas Mikolov et al., NIPS, 2013
- Efficient Estimation of Word Representations in Vector Space (word2vec, CBOW, and Skip-gram), Tomas Mikolov et al., ICLR, 2013 [Code]
- Software Framework for Topic Modelling with Large Corpora (gensim), Radim Řehůřek et al., LREC, 2010 [Code]
Research Papers
- Investigation of Recurrent-Neural-Network Architectures and Learning Methods for Spoken Language Understanding, Grégoire Mesnil et al., Interspeech, 2013 [Code] [Tutorial] #NER #ATIS
Datasets
- Airline Travel Information Systems (ATIS) #NER
- Facebook Dialog bAbI tasks
- Facebook Movie Dialog dataset
- Maluuba Frames Dataset
- An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking, Puyang Xu, Qi Hu, 2018
- Global-Locally Self-Attentive Dialogue State Tracker, Victor Zhong, Caiming Xiong, Richard Socher, ACL, 2018
- Dialog state tracking, a machine reading approach using Memory Network, Julien Perez et al., ACL, 2017
- Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders, Tiancheng Zhao et al., ACL, 2017
- Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning, Tiancheng Zhao et al., SIGDIAL, 2016
- Word-Based Dialog State Tracking with Recurrent Neural Networks, Matthew Henderson et al., SIGDIAL, 2014
- The E2E Dataset: New Challenges For End-to-End Generation, Jekaterina Novikova et al., SIGDIAL, 2017 #NLG
- Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking, Tsung-Hsien Wen et al., SIGDIAL, 2015
- End-to-end joint learning of natural language understanding and dialogue manager, Xuesong Yang et al., ICASSP, 2017
- Learning End-to-end Goal-oriented Dialog, Antoine Bordes et al., ICLR, 2017
- A Network-based End-to-End Trainable Task-oriented Dialogue System, Tsung-Hsien Wen et al., arXiv, 2016
Reviews & Tutorials
- Learning to Reason by Reading Text and Answering Questions, Minjoon Seo, NAVER D2, 2017
Research Papers
- Question-Answering with Grammatically-Interpretable Representations, Hamid Palangi et al, arXiv, 2017
- Learning to Compose Neural Networks for Question Answering, Jacob Andreas et al., NAACL-HLT, 2016
- Question Answering over Freebase with Multi-Column Convolutional Neural Networks, Li Dong et al., ACL-IJCNLP, 2015
- Large-scale Simple Question Answering with Memory Networks, Antoine Bordes et al., arXiv, 2015
- End-to-end Memory Networks (MemN2N), Sainbayar Sukhbaatar et al., NIPS, 2015 [Code]
- Question Answering with Subgraph Embeddings, Antoine Bordes et al., EMNLP, 2014
- Memory Networks (MemNN), Jason Weston et al., arXiv, 2014
- Semantic Parsing for Single-Relation Question Answering, Wen-tau Yih et al., ACL, 2014 [Poster]
- Reasoning With Neural Tensor Networks for Knowledge Base Completion, Richard Socher et al., NIPS, 2013.
- Semantic Parsing on Freebase from Question-Answer Pairs (SEMPRE), Jonathan Berant, EMNLP, 2013 [Code]
Datasets
- Facebook QA bAbI tasks
- Facebook Children's Book test
- Facebook SimpleQuestions dataset
- WebQuestions
- DeepMind Q&A Dataset
- The Stanford Question Answering Dataset (SQuAD)
- CMU Question-Answer Dataset
- Maluuba NewsQA dataset
- Quora Question Pairs
- Generative Encoder-Decoder Models for Task-Oriented Spoken Dialog Systems with Chatting Capability, Tiancheng Zhao et al., SIGDIAL, 2017
Reviews & Tutorials
- Neural Machine Translation and Sequence-to-sequence Models: A Tutorial, Graham Neubig, arXiv, 2017
- Deep Natural Language Understanding, Kyunghyun Cho, Deep Learning Summer School, 2016
- Neural Machine Translation, Kyunghyun Cho, Microsoft Research Invited Talk, 2016
- Introduction to Neural Machine Translation with GPUs [Part 1] [Part 2] [Part 3], Kyunghyun Cho, NVIDIA Blog, 2015
Research Papers
- Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation, Melvin Johnson et al., arXiv, 2016
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation, Yonghui Wu et al., arXiv, 2016
- A Character-level Decoder without Explicit Segmentation for Neural Machine Translation, Junyoung Chung et al., ACL, 2016
- Neural machine translation by jointly learning to align and translate, Dzmitry Bahdanau et al., ICLR, 2015 #Attention
- Sequence to Sequence Learning with Neural Networks (seq2seq), Ilya Sutskever et al., NIPS, 2014 #Google #seq2seq #LSTM
- Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation, Kyunghyun Cho et al., EMNLP, 2014
Reviews & Tutorials
- Developing Korean Chatbot 101, Jaemin Cho, TensorFlow Korea, 2017 [Slide]
- Chatbots with Seq2Seq [Part 1] [Part 2], Suriyadeepan Ram, Personal Blog, 2016
- Building AI Chat bot using Python 3 & TensorFlow, Jeongkyu Shin, PyCon, 2016
- Deep Learning for Chatbots, Denny Britz, WildML, 2016
Research Papers
- Adversarial Learning for Neural Dialogue Generation, Jiwei Li et al., arXiv, 2017 [Code] #GAN
- A Persona-Based Neural Conversation Model, Jiwei Li et al., ACL, 2016
- Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models, Iulian V. Serban et al., AAAI, 2016
- A Neural Conversational Model, Oriol Vinyals et al., ICML, 2015 #Google #LSTM #seq2seq
- A Neural Network Approach to Context-Sensitive Generation of Conversational Responses, Alessandro Sordoni et al., NAACL-HLT, 2015
- Neural Responding Machine for Short-Text Conversation, Lifeng Shang et al., ACL, 2015
Datasets
Reviews & Tutorials
- LSTM Networks for Sentiment Analysis, Pierre Luc Carrier, deeplearning.net, 2012
Research Papers
- A Convolutional Neural Network for Modelling Sentences, Nal Kalchbrenner et al., ACL, 2014
- Convolutional Neural Networks for Sentence Classification, Yoon Kim, EMNLP, 2014
- Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank, Richard Socher et al., EMNLP, 2013
Datasets
Research Papers
- InstructGPT: Training language models tofollow instructions with human feedback, Long Ouyang et al. (OpenAI), 2022
- Token Dropping for Efficient BERT Pretraining, Google, 2022
- Chinchillia: Training Compute-Optimal Large Language Models, DeepMind, 2022 : 모델 크기는 줄이고 데이터를 늘려도 좋은 성능이 나온다
- GPT-3: Language Models are Few-Shot Learners, Tom B. Brown et al. (OpenAI), 2020
- SpanBERT: Improving Pre-training by Representing and Predicting Spans, Mandar Joshi et al., 2020
- Recurrent Highway Networks, Julian Georg Zilly et al., arXiv, 2017 [Code]
- Recurrent Neural Network based Language Model, Tomas Mikolov et al., Interspeech, 2010 [Code]
- A Scalable Hierarchical Distributed Language Model (Hierarchical Softmax), Andriy Mnih, NIPS, 2008
- A Neural Probabilistic Language Model, Yoshua Bengio et al., NIPS, 2001
Datasets
- Neural Architectures for Named Entity Recognition, Guillaume Lample et al., NAACL-HLT, 2016
- Modeling Mention, Context and Entity with Neural Networks for Entity Disambiguation, Yaming Sun et al., IJCAI, 2015
- Improving Efficiency and Accuracy in Multilingual Entity Extraction (DBpedia Spotlight), Joachim Daibe et al., I-Semantics, 2013
- Robust Disambiguation of Named Entities in Text (AIDA), Johannes Hoffart et al., EMNLP, 2011
- Local and Global Algorithms for Disambiguation to Wikipedia, Lev Ratinov et al. ACL, 2011
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention, Kelvin Xu, ICML, 2015
- Show and Tell: A Neural Image Caption Generator, Oriol Vinyals et al., CVPR, 2015
- Deep Visual-Semantic Alignments for Generating Image Descriptions, Andrej Karpathy et al., CVPR, 2015 [Code] [Demo]
- Parsing Natural Scenes and Natural Language with Recursive Neural Networks, Richard Socher et al., ICML, 2011
- Joint Concept Learning and Semantic Parsing from Natural Language Explanations, Shashank Srivastava et al., EMNLP, 2017
- Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detection, Alexander Podolskiy et al., AAAI 2021
- A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks, Kimin Lee, Kibok Lee, Honglak Lee, Jinwoo Shin, NIPS 2018
- Rethinking Calibration of Deep Neural Network: Do Not Be Afraid of Overconfidence, Deng-Bao Wang et al., NIPS 2021