Current progress is updated in progress.txt

中文版欢迎关注我的CSDN

Wenqing's Machine Learning Blog

Foundational Concepts in Machine Learning

• Linear Algebra

  • Basic
  • Numpy Tips
  • Inverse matrix
  • Diagonal matrix
  • Determinants
  • Linear Transformations
  • Linear Systems
  • Change Of Basis
  • Eigenvector
  • Singular Vector
  • Eigenvalue
  • Singular Value
  • Singular Value Decomposition (SVD)

• Calculus

  • Calculus

• Statistical Measures

  • Standard Deviation vs Standard Error
  • Margin of Error
  • Statistical Inference
  • Confidence Intervals
  • Expected Value
  • Statistical Power
  • P-values
  • KL divergence
  • Statistical Significance
  • Bias
  • Variance&Standard Deviation
  • Covariance
  • Correlation
  • Joint Variability
  • Bessel’s Correction
  • Degrees of Freedom
  • Cardinality
  • Entropy
  • Naive Bayes
  • Hidden Markov Models (HMMs)
  • Probability Density Function (PDF)
  • Probability Distributions
    • Uniform Distribution
    • Normal Distribution
    • Cumulative Distribution Function (CDF)
    • Standard Gaussian Distribution
    • Bernoulli Distribution
    • Binomial Distribution

• Feature Engineering

  • Overview of Feature Engineering
  • One Hot Encoding
  • Feature Hashing

• Feature Selection

  • Overview of Feature Selection

• Data Preprocessing

  • Overview of Data Preprocessing
  • Data Normalization
  • Data Standardization
  • Missing Data Handling
  • Data Augmentation

• Model Evaluation

  • Model_Eval
  • Offline Metrics
    • Cross-Validation
    • Underfitting (High Bias) & Overfitting (High Variance)
    • AUC (Area Under Curve)
    • logloss
    • RMSE
    • MAPE
    • Confusion Matrix
    • Precision, Recall and F1 score
  • Online Metrics

• Training Pipeline

  • Training Pipeline Overview
  • Data Partitioning
  • Handle Imbalance Class Distribution
  • Choose the Right Loss Function
  • Retraining Requirements

• Classification Methods

  • Overview
  • Nearest Neighbor Classifiers
    • Nearest Neighbor Classifier Overview
    • k-Nearest Neighbor (kNN) Classifier
      • kNN Overview
      • kNN for Classification
      • kNN for Regression
  • Linear Classification
    • Linear Classification Overview
    • Support Vector Machine (SVM) Classifier
      • SVM Classifier Overview
      • Multiclass SVM Loss
    • Softmax Classifier
      • Softmax Classifier Overview
      • Softmax and Cross-Entropy Loss

• Linear Regression

  • Overview
  • Assumptions of Linear Regression
    • Linearity
    • Homoscedasticity (Constant Variance)
    • Normal Distribution of Errors
    • No Multicollinearity
    • Independence of Errors(Non-Random Residuals)
  • Performance Metrics for Linear Regression
    • R-squared
    • P-values
    • Statistical Significance
    • Prediction Interval
    • Standard Error of the Estimate

• Logistic Regression

  • Overview
  • Softmax-Regression

• Decision Trees

  • Overview

• Support Vector Machines (SVM)

  • Overview

• T-SNE

  • Overview

• Dimensionality Reduction

  • Principal Component Analysis (PCA)

• Jargon

  • Batch Size
  • Sequence Length

Deep Learning Concepts

Basic Neural Network Concepts

• Neural Network

  • Overview

• Feed Forward Neural Network (FFNN)

  • Overview
  • MLP (Multi-Layer Perceptron)
    • Overview

• Activation Functions

  • Overview
  • ReLU
  • Sigmoid
  • Tanh
  • GELU
  • Softmax

• Backpropagation

  • Overview

• Neural Network Layers

  • Linear Layer
  • Residual Connection
  • ReLU Activation
  • Flatten Layer
  • Batch Normalization (BatchNorm1d)
  • Layer Normalization (LayerNorm1d)
  • Dropout
  • Identity Layer
  • Sequential Layer
  • Embedding Layer

• Neural Network Models

  • Overview
  • MLP
  • ResNet

• Loss Functions

  • Overview
  • Classification Tasks:
    • Cross Entropy Loss
      • Binary Cross Entropy Loss
      • Normalized Cross Entropy
    • Ranking Loss
  • Regression Tasks:
    • MSE Loss
    • MAPE
    • SMAPE
    • Quantile Loss

• Optimization Algorithms

  • Overview
  • Convex Functions
  • Gradient Descent
  • Mini-Batch Gradient Descent
  • Stochastic Gradient Descent (SGD)
  • Stochastic Gradient Descent with momentum (SGD with momentum)
  • Adam
  • Advanced Optimization Algorithms
    • Momentum
    • Adam
    • RMSProp
    • AdaGrad

• Regularization

  • Overview
  • L1 Regularization
  • L2 Regularization
  • Dropout
  • Early Stopping

• Normalization

  • Batch Normalization
  • Layer Normalization

Hardware and Parallel Processing

• CPU vs GPU Performance
• CUDA Kernels and GPU Parallelism

Advanced Neural Network Architectures

• Convolutional Neural Networks (CNN)

  • Overview of CNNs
  • Convolutional Layers
  • Pooling Layers
  • Stride and Padding
  • CNN Architectures (e.g., LeNet, AlexNet)

• Recurrent Neural Networks (RNN)

  • Overview of RNNs
  • LSTM (Long Short-Term Memory)

• Transformer Networks

  • Overview of Transformers
  • Hidden States
  • Prefill vs Decode

• MoE

  • sparsely-gated-MoE-top-k.md
  • moe-block.md

Sequence Models

• Overview
• Language Model Evaluation Metrics
  • Perplexity
  • [Other relevant metrics...]
• RNN (Recurrent Neural Networks)
  • Standard RNN
  • LSTM (Long Short-Term Memory)
• CNN (Convolutional Neural Networks)
  • Overview
• Transformers
  • Original Transformer
  • BERT (Bidirectional Encoder Representations from Transformers)
    • Overview
  • GPT (Generative Pre-trained Transformer)
    • Overview
  • T5 (Text-to-Text Transfer Transformer)
    • Overview
• Model Comparisons: CNN vs. RNN vs. Transformer

Transfer Learning

• Overview
• Fine Tuning

Large Language Models

• Overview
• Weight Tying

Generative AI

• Overview
  • Overview
• Generative Adversarial Networks (GANs)
  • Introduction to GANs
• Variational Autoencoders (VAEs)
  • Introduction to VAEs
• Applications of Generative AI
  • Applications

Word Representations

• Embedding
  • Overview

Machine Learning System Design Notes From Educative

• Video Recommendation
  • Matrix Factorization
  • Collaborative Filtering
• Feed Ranking
• Ad Click Prediction
• Rental Search Ranking
• Estimate Delivery Time

Deployment and DevOps Tools

• Docker

  • Overview of Docker
  • Setting up Docker for Machine Learning Projects
  • Docker Commands and Usage

• CI/CD

  • Overview of CI/CD
  • Setting Up CI/CD for Machine Learning Projects
  • Best Practices for CI/CD in ML

Code Implementation

• Image Segmentation Task For CGG
• Decision Tree
• Logistic Regresssion
• Softmax Regresssion

NLP

• Text Classification
• Named-Entity-Recognition
• Corpus
• Text Normalization
• Support Vector Machines (SVM)
• Naive Bayes
• Conditional Random Fields (CRFs)
• LangChain

Recommendation System

• NMF (Non-Negative Matrix Factorization)
• Content-Based Filtering
• Collaborative Filtering
• FAISS Index

Other

• Object-Relational Mapping (ORM)

Machine Learning Interview Questions

• Useful Link
• Interview Questions
• NLP Interview Questions
• Transformer Interview Questions
• Machine Learning System Design Interview Questions