DSO 560 NLP & Text Analytics (Fall 2022)

Slack Workspace

We will be using Slack workspaces for most of our course communications, including submitting classwork, homework, and grading. Your final grades will be posted on Blackboard, but feedback for assignments will be delivered via Slack. Prior to Week 1's session, please join the Slack workspace. Note that this is a different Slack workspace than the auto-created workspaces by USC Marshall.

Lecture Recordings

• Week 7 Optional Lecture
• Week 6 Part I, Week 6 Part II
• Week 5 Part I, Week 5 Part II
• Week 4 Part I, Week 4 Part II
• Week 3 Part I, Week 3 Part II
• Week 2 Part I, Week 2 Part II
• Week 1 Part I, Week 1 Part II

Async Lectures

• Training Your Own Word Embeddings Using Keras
• RNNs and LSTMs Deep Dive

Optional Lecture (Saturday December 3rd)

Links

• Blog Article About Spark Streaming ML for Push Notification/Recommendations

Notebooks

• Databricks Spark Simple Word Count Example
• Deep Learning to Train Word Embeddings
• LSTMs and Self-Attention for NLP
• Fine-Tuning Hugging Face Model

Week 6 (Thursday, December 1st)

Notebooks

• In-Class Examples Google Sheet
• Other Uses Cases for Skipgram Model
• word2vec Part I
• word2vec Part II
• Huggingface Demo
• BERT
• Example of Attention-Based Image Classification MNIST Handwritten Digits
• Example of Attention-Based Image Classification Faces
• Transformers for Natural Language Processing Github - this is an excellent resource for code snippets that you can leverage in your own experiments and learnings
• Basic Self Attention Walkthrough (from Rothman book)
• Understanding Positional Encodings
• LSTMSs and Self-Attention for NLP
• Using RNNs and LSTMs

Week 5 (Thursday, November 17th)

Notebooks

• In-Class Examples Google Sheet
• Probabilities and N-Gram Language Models
• word2vec Part I
• word2vec Part II
• Deep Learning to Train Word Embeddings

Week 4 (Thursday, November 10th)

Notebooks

• Cosine Similarity, PMI, Collocation
• Probabilities and N-Gram Language Models
• Using Textacy for Text Preprocessing
• Tips and Tricks
• In-Class Examples Google Sheet
• Topic Modelling with Unigrams
• Topic Modelling

Week 3 (Thursday, November 3rd)

Notebooks

• Cosine Similarity, PMI, Collocation
• Probabilities and N-Gram Language Models
• Using Textacy for Text Preprocessing
• Naive Bayes
• Tips and Tricks
• In-Class Examples Google Sheet

Week 2 (Thursday, October 27th)

Notebooks

• Homework 1
• Classification and Model Evaluation
• Term Frequency Inverse Document Frequency

Week 1 (Thursday, October 20th)

Notebooks

• Additional Practice (Week 1)
• Regex Exercises
• Regex Intermediate Exercises
• Basic Python Operations for Working with Text
• Text Preprocessing Techniques

Before Class

• Make sure you have the following items installed on your computer:

1. Python 3.*
2. Jupyter notebooks
3. pandas and scikit-learn libraries

• Join the Slack workspace (check email sent via Blackboard).
• (Optionally) - Clone and fork this repository onto your computer so you can follow along.

Related Resources

Working With Git

• Cloning/forking Github repositories. This video walkthrough takes you through how to clone/fork the course repository. Many students fork the course repository so that they can make local changes to the lecture notebooks and follow along. However, cloning/forking repos can be tricky. Note that since this video was posted, the primary branch on Github has been renamed from master to main. Therefore, everywhere in the walkthrough you see master, substitute it for main.

Prior Years' Course Repositories

Feel free to look for examples from prior years. In general, there is a lot of overlap between course content year over year. However, keep in mind that each time I teach this course, approximately 25-30% of the content changes (due to adjusting for feedback, industry advances, etc.)