Skip to content
Switch branches/tags


Failed to load latest commit information.
Latest commit message
Commit time
May 29, 2020
Sep 8, 2021

CC6205 - Natural Language Processing

This is a course on natural language processing.


The neural network-related topics of the course are taken from the book of Yoav Goldberg: Neural Network Methods for Natural Language Processing. The non-neural network topics (e.g., grammars, HMMS) are taken from the course of Michael Collins.


  1. Introduction to Natural Language Processing | (tex source file), video 1, video 2
  2. Vector Space Model and Information Retrieval | (tex source file), video 1, video 2
  3. Language Models (slides by Michael Collins), notes, video 1, video 2, video 3, video 4
  4. Text Classification and Naive Bayes (slides by Dan Jurafsky), notes, video 1, video 2, video 3
  5. Linear Models | (tex source file), video 1, video 2, video 3, video 4
  6. Neural Networks | (tex source file), video 1, video 2, video 3, video 4
  7. Word Vectors | (tex source file) video 1, video 2, video 3
  8. Tagging, and Hidden Markov Models (slides by Michael Collins), notes, video 1, video 2, video 3, video 4
  9. MEMMs and CRFs | (tex source file), notes 1, notes 2, video 1, video 2, video 3
  10. Convolutional Neural Networks | (tex source file), video
  11. Recurrent Neural Networks | (tex source file), video 1, video 2, video 3
  12. Sequence to Sequence Models, Attention, and the Transformer | (tex source file), video 1, video 2, video 3
  13. Contextual Words Representations (slides by Chris Manning) video 1, video 2, video 3, notes
  14. Constituency Parsing slides 1, slides 2, slides 3, slides 4 (slides by Michael Collins), notes 1, notes 2, videos 1, videos 2, videos 3, videos 4
  15. Recursive Networks and Paragraph Vectors | (tex source file)

NLP Libraries

  1. NLTK: Natural Language Toolkit
  2. Gensim
  3. spaCy: Industrial-strength NLP
  4. Torchtext
  5. AllenNLP: Open source project for designing deep leaning-based NLP models
  6. Transformers: a library of state-of-the-art pre-trained models for Natural Language Processing (NLP)
  7. Stanza - A Python NLP Library for Many Human Languages
  8. FlairNLP: A very simple framework for state-of-the-art Natural Language Processing (NLP)
  9. WEFE: The Word Embeddings Fairness Evaluation Framework
  10. WhatLies: A library that tries help you to understand. "What lies in word embeddings?"
  11. LASER:a library to calculate and use multilingual sentence embeddings
  12. Sentence Transformers: Multilingual Sentence Embeddings using BERT / RoBERTa / XLM-RoBERTa & Co. with PyTorch
  13. Datasets: a lightweight library with one-line dataloaders for many public datasets in NLP

Notes and Books

  1. Speech and Language Processing (3rd ed. draft) by Dan Jurafsky and James H. Martin.
  2. Michael Collins' NLP notes.
  3. A Primer on Neural Network Models for Natural Language Processing by Joav Goldberg.
  4. Natural Language Understanding with Distributed Representation by Kyunghyun Cho
  5. Natural Language Processing Book by Jacob Eisenstein
  6. NLTK book
  7. Embeddings in Natural Language Processing by Mohammad Taher Pilehvar and Jose Camacho-Collados
  8. Dive into Deep Learning Book

Other NLP Courses

  1. CS224n: Natural Language Processing with Deep Learning, Stanford course
  2. Deep Learning in NLP: slides by Horacio Rodríguez
  3. David Bamman NLP Slides @Berkley
  4. CS 521: Statistical Natural Language Processing by Natalie Parde, University of Illinois
  5. 10 Free Top Notch Natural Language Processing Courses


  1. Natural Language Processing MOOC videos by Dan Jurafsky and Chris Manning, 2012
  2. Natural Language Processing MOOC videos by Michael Collins, 2013
  3. Natural Language Processing with Deep Learning by Chris Manning and Richard Socher, 2017
  4. CS224N: Natural Language Processing with Deep Learning | Winter 2019
  5. Computational Linguistics I by Jordan Boyd-Graber University of Maryland
  6. Visualizing and Understanding Recurrent Networks
  7. BERT Research Series by Chris McCormick
  8. Successes and Challenges in Neural Models for Speech and Language - Michael Collins
  9. More on Transforemers: BERT and Friends by Jorge Pérez

Other Resources

  1. ACL Portal
  2. NLP-progress: Repository to track the progress in Natural Language Processing (NLP)
  3. NLP News By Sebastian Ruder
  4. Corpora Mailing List
  5. Real World NLP Book: AllenNLP tutorials
  6. Attention is all you need explained
  7. The Illustrated Transformer: a very illustrative blog post about the Transformer
  8. ELMO explained
  9. BERT exaplained
  10. Better Language Models and Their Implications OpenAI Blog
  11. RNN effectiveness
  12. SuperGLUE: an benchmark of Natural Language Understanding Tasks
  13. decaNLP The Natural Language Decathlon: a benchmark for studying general NLP models that can perform a variety of complex, natural language tasks.
  14. Chatbot and Related Research Paper Notes with Images
  15. XLNet Explained
  16. Ben Trevett's torchtext tutorials
  17. PLMpapers: a collection of papers about Pre-Trained Language Models
  18. The Illustrated GPT-2 (Visualizing Transformer Language Models)
  19. Linguistics, NLP, and Interdisciplinarity Or: Look at Your Data, by Emily M. Bender
  20. The State of NLP Literature: Part I, by Saif Mohammad
  21. From Word to Sense Embeddings:A Survey on Vector Representations of Meaning
  22. 10 ML & NLP Research Highlights of 2019 by Sebastian Ruder
  23. Towards a Conversational Agent that Can Chat About…Anything
  24. The Super Duper NLP Repo: a collection of Colab notebooks covering a wide array of NLP task implementations
  25. The Big Bad NLP Database, a collection of nearly 300 well-organized, sortable, and searchable natural language processing datasets
  26. A Primer in BERTology: What we know about how BERT works
  27. How Self-Attention with Relative Position Representations works
  28. Deep Learning Based Text Classification: A Comprehensive Review
  29. Teaching NLP is quite depressing, and I don't know how to do it well by Yoav Goldberg
  30. The NLP index