tad

Text as data:

    - cleaning;
    - pre-processing;
    - post processing:
    - Topic Modelling: LDA, seeded LDA
    - Word2Vec and other static embedding
    - RNN, LSTM, and Seq2Seq
    - Attention and Transformers

Python Version: 3.7.16

Scikit-learn versions that are supported: 0.20 to 0.24 and 1.0, 1.0.1, and 1.0.2

To install the packages:

pip install -r requirements.txt

Ad-hoc materials:

Along with the lecture slides, we can also refer to the resources below:

0. Speech and Language Processing:

   • Book: Speech and Language Processing by Dan Jurafsky and James H. Martin
   • Book: Machine Learning for Text by Charu C. Aggarwal

1. Text Processing:

   • Blog: Jaccard vs Cosine Distance
   • Blog: Understanding the Levenshtein Distance Equation
   • Blog: Stemming vs Lemmatization

2. Fundamentals:

   • Tf-IDf:

     • Blog: TF-IDF

   • Zipf's Law:

     • Blog: Zipf's Law
     • YT: Zipf's Law

   • Heaps' Law:

     • YT: Heaps' law

3. Clustering:

   • SVD:

     • Basics, YT: Singular Value Decomposition (the SVD)
     • High Level Overview, YT: Singular Value Decomposition (SVD): Overview
     • Mathematical Overview: YT: Singular Value Decomposition (SVD): Mathematical Overview
     • Rank R Approximation, YT: Singular Value Decomposition (SVD): Matrix Approximation

   • LSA:

     • YT: LSA

4. Topic Modelling:

   • Data Pre-processing for Topic Modelling:

     • Blog: NLP Preprocessing and Latent Dirichlet Allocation (LDA) Topic Modeling with Gensim

   • LDA:

     • YT: Latent Dirichlet Allocation (Part 1 of 2)
     • YT: Training Latent Dirichlet Allocation: Gibbs Sampling (Part 2 of 2)
     • Blog: Topic Modeling and Latent Dirichlet Allocation (LDA) in Python
     • YT: LDA Plate Notation

5. Word2Vec:

   • YT: Word2Vec, GloVe, FastText- EXPLAINED!
   • YT: Word Vector Representations: word2vec
   • YT: GloVe: Global Vectors for Word Representation
   • YT: Word2Vec Detailed Explanation, Train custom Word2Vec Model using genism in Python
   • YT: Coding Word2Vec : Natural Language Processing
   • YT: Word Embedding and Word2Vec, Clearly Explained!!!
   • Extra, YT: Word Embeddings - EXPLAINED!

6. Doc2Vec:

   • Blog: How to Vectorize Text in DataFrames for NLP Tasks — 3 Simple Techniques
   • Blog: Practical Guide To Doc2Vec & How To Tutorial In Python
   • Blog: Multi-Class Text Classification with Doc2Vec & Logistic Regression

7. RNN and LSTM:

   • YT: Recurrent Neural Networks (RNNs), Clearly Explained!!!
   • YT:Long Short-Term Memory (LSTM), Clearly Explained
   • YT: Sequence-to-Sequence (seq2seq) Encoder-Decoder Neural Networks, Clearly Explained!!!

8. Attention and Transformers:

   • YT: The Attention Mechanism in LLMs a High Level Overview
   • YT: The math behind Attention: Keys, Queries, and Values matrices
   • YT: Attention for Neural Networks, Clearly Explained!!!
   • YT: What is Transformer Models and how do they work?
   • YT: Transformer Neural Networks, ChatGPT's foundation, Clearly Explained!!!
   • YT:Decoder-Only Transformers, ChatGPTs specific Transformer, Clearly Explained!!!
   • Blog: A Gentle Introduction to Positional Encoding in Transformer Models, Part 1