Machine Learning II (UMONS, 2023-2024)

This is a repository to organize the teaching material for Machine Learning II, to be taught by Souhaib Ben Taieb.

Course links

• Moodle (for discussion forum, assignments)

Project

• TBA

Online content

• (IAML) Intuition for the Algorithms of Machine Learning
• (PPA) Patterns, predictions, and actions: a story about machine learning
• (LFD) Learning from data

Content

• Week 1 (Feb. 5-9).

  • Lecture 1 (Feb. 9): Introduction Slides (PDF)
    • IAML Videos
      • Introduction to Intuition for the Algorithms of ML
      • Basic Problem of ML
    • PPA book reading for the next lecture
      • Chapter 1 (Introduction)
      • Chapter 2 (Fundamentals of Prediction)

• Week 2 (Feb. 12-16).

  • Lecture 2 (Feb. 14)
    • IAML Videos (margin, classification loss functions, expected training and test error as a function of model complexity or sample size, etc)
      • LossFunctions, OckhamsRazor, EvaluationMeasures, ROC1, ROC2, ImbalancedData, ROC3
    • PPA book reading for the next lecture
      • Chapter 3 (Supervised Learning)
  • Lab 1 (Feb. 15): The perceptron learning model [lab (PDF)]

• Week 3 (Feb. 19-23).

  • Lecture 3 (Feb. 22)
    • PPA book Chapter 2 Proof of 2 Lemmas:
      • Lemma 1: Optimal Classifier
      • Lemma 2: Neyman-Pearson Lemma
  • Lab 2 (Feb. 23): The perceptron learning model (Continued) [lab (PDF)] (Solution)

• Week 4 (Feb. 26 - Mar. 1).

  • Lecture 4 (Feb. 26)
    • PPA book Chapter 2
  • Lab 3 (Feb. 29)
    • Linear classification and Optimisation [lab (PDF)]
  • Lecture 5 (Mar. 1)
    • PPA book Chapter 3

• Week 5 (Mar. 4-8).

  • Lecture 6 (Mar. 4)
    • PPA book Chapter 3 + IAML Videos "Statistical Learning Theory"
  • Lecture 7 (Mar. 6)
    • Book reading: Chapter 4
  • Lab 4 (Mar. 8)
    • Linear classification and Optimisation (continue)
  • Lecture 8 (Mar. 8)
    • IAML Videos "Statistical Learning Theory"

• Week 6 (Mar. 11-15).

  • Lecture 9 (Mar. 13)
    • IAML Videos "Statistical Learning Theory" (8-11)
    • LFD Videos "Lecture 06 - Theory of Generalization" (Proof that the growth function is bounded by a polynomial when we have a breakpoint)
  • Lab 5 (Mar. 14)
    • Linear classification and Optimisation (continue) (Solution)
  • Lab 6 (Mar. 15)
    • Linear Regression [lab (PDF)]

• Week 7 (Mar. 18-22).

  • Lab 7 (Mar. 21)
    • Linear Regression (continue) (Solution) + Learning Theory [lab (PDF)]
  • Lecture 10 (Mar. 22)
    • Neural Networks (3Blue1Brown)

• Week 8 (Mar. 25-29).

  • Lab 8 (Mar. 25)

    • SLT

  • Lecture 11 (Mar. 28)

    • Backpropagation [Slides (PDF)] [Notes (PDF)]
    • Generalization in Neural Networks [Slides (PDF)] [Notes (PDF)]

  • Lab 9 (Mar. 29)

    • SLT (continue)

Spring Break

• Week 9 (Apr. 15-19).

  • Lecture 12 (Apr. 17)
    • Neural Networks (3Blue1Brown)
  • Lab 10 (Apr. 18)
    • Introduction to PyTorch [Lab (Notebook)] ()
    • Backpropagation [Lab (Notebook)] () Solution (PDF) ()

• Week 10 (Apr. 22-26).

  • Lecture 13 (Apr. 25)
    • Neural Networks (3Blue1Brown) + Deep Generative Models
  • Lab 11 (Apr. 25)
    • Generalization in Neural Networks [Lab (Notebook)] () [Solution] ()

• Week 11 (Apr. 29 - May 3).

  • Lab 12 (May 2)
    • Convolutional Neural Networks [Lab (Notebook)] () [Solution] ()
  • Lab 13 (May 3)
    • Recurrent Neural Networks Reference [Lab (Notebook)] () [Solution] ()

• Week 12 (May 6-10).

  • Lab 14 (May 6)
    • GPT
  • Lecture 14 (May 10). Deep Generative Models (Lecture 1)

• Week 13 (May 13-17).

  • Lab 15 (May 16)
    • Variational Autoencoders (9:32 - 38:48)
    • [Lab (Notebook)] () [Solution] ()
  • Lecture 15 (May 17). Deep Generative Models (Lecture 2)