Natural Language Processing

Professor Sable's website: http://faculty.cooper.edu/sable2/courses/spring2021/ece467/

Textbook: Speech and Language Processing, 3rd Edition by Daniel Jurafsky and James H. Martin

This course focuses on computational applications involving the processing of written or spoken human languages. Content may vary from year to year. Theoretical subtopics will likely include word statistics, formal and natural language grammars, computational linguistics, hidden Markov models, and various machine learning methods. Applications covered will likely include information retrieval, information extraction, text categorization, question answering, summarization, machine translation and speech recognition. Course work includes programming projects and problem sets.

Projects

Each in their own respective folder

1. Tag Categorization

Given a training corpus of labeled documents, learn to predict these labels for a new corpus

2. Sentence Parser

Parses sentences using grammar rules in Chomsky normal form

3. Deep Learning Sentiment Analysis

Use a deep learning framework to complete any NLP related project. I chose to perform sentiment analysis on tweets.

Topics:

1. Words, Morphology, & Tokenization

• Stemming
• Lemmatization
• Chomsky Hierarchy
• Regular Expressions
• Morphology

2. N-grams

• N-grams
• Maximum Likelyhood Estimation
• Language Models

4. Part-of-Speech Tagging

• Parts of Speech
• Tagsets
• Ambiguity
• Hidden Markov Models
• Viterbi

4. Vector Space Models, Information Retrievel, & Text Categorization

• Information Retrieval
• Similarity
• Inverted Indices
• Weighting schemes
• Text Categorization
  • Rocchio
  • K-Nearest Neighbors
  • Naive Bayes
  • Evaluation Metrics

5. Grammar

• Syntax
• Context-Free Grammars
• Parse Trees

6. Natural Languages and Psycholinguistics

• Differentiating Between Languages
• Descriptive vs. Prescriptive Grammars

7. Parsing

• Constituency Parsing
• Chomsky Normal Form
• CKY Algorithm
• Probabilistic Context-Free Grammars

8. First order Logic & Semantics

• Meaning representations
• Reference, Conference, & Coherence Resolution
• Winograd Schemas

9. Feedforward Neural Networks

• Nerons
• Activations
• Perceptrons
• Neural Networks
• Loss Functions
• Optimization
• Neural Networks as Computational Graphs

10. Word Embeddings, Neural Language Models, & Word2Vec

• Word Embeddings
• Neural Language Models
• Word2Vec
• Skip-Gram with Negative Sampling
• Differences between Embeddings

11. Recurrent Neural Networks, LSTMs, & GRUs

• RNN Structure & Training
• Stacked RNNs
• Bi-directional RNNs
• LSTMs
• GRUs

12. Encoder-Decoder Models, Attention, & Machine Translation

• Seq2Seq
• Autoregressive Generation
• Different Approaches to Machine Translation
• Encoder-Decoder Models
• Beam Search
• Attention
• Why Attention Helps - Improving Context Vectors

13. Advanced Topics

• Character & Subword Embeddings
• Question Answering Systems
• Transformers
• Contextual Word Embeddings
• Ethical NLP