Bayesian machine learning

Lecturers

• Rémi Bardenet (CNRS, Univ. Lille), https://rbardenet.github.io
• Julyan Arbel (Inria, Univ. Grenoble-Alpes), https://www.julyanarbel.com

Material

• Projects: group presentations are on a voluntary basis, register on the framacalc sheet.
• Reading list as a preparation for the Bayesian deep learning lecture (March 7): review paper on BNNs and position paper on future of BDL.
• Updated slides on Bayesian nonparametrics are in the slides folder.
• Annotated slides on Bayesics are here and here.
• Annotated slides on MCMC are here and here.
• Incomplete and drafty lecture notes are available in the notes folder. Any comment welcome, live or as a raised issue.
• Practicals and exercises are available in the corresponding folders. They are to be done on a voluntary basis. Solutions will be provided on demand.

Objective of the course

By the end of the course, the students should

• have a high-level view of the main approaches to making decisions under uncertainty.
• be able to detect when being Bayesian helps and why.
• be able to design and run a Bayesian ML pipeline for standard supervised or unsupervised learning.
• have a global view of the current limitations of Bayesian approaches and the research landscape.
• be able to understand the abstract of most Bayesian ML papers.

Topics

• Decision theory
• 50 shades of Bayes: Subjective and objective interpretations
• Bayesian supervised and unsupervised learning
• Bayesian computation for ML: Advanced Monte Carlo and variational methods
• Bayesian nonparametrics
• Bayesian methods for deep learning

Prerequisites

• An undergraduate course in probability.
• It is recommended to have followed either "Probabilistic graphical models" or "Computational statistics" during the first semester.

Organization of courses

• 8x3 hours of lectures, the last session being a student seminar.
• All classes and materials will be in English. Students may write their final report either in French or English.

Validation

• Students form groups. Each group reads and reports on a research paper from a list. We strongly encourage a dash of creativity: students should identify a weak point, shortcoming or limitation of the paper, and try to push in that direction. This can mean extending a proof, implementing another feature, investigating different experiments, etc.
• Deliverables are a small report and a short oral presentation in front of the class, in the form of a student seminar, which will take place during the last lecture.
• "Auditeurs libres" who need a grade will be given a different assignment, depending on their situation.

References

• Parmigiani, G. and Inoue, L. 2009. Decision theory: principles and approaches. Wiley.
• Robert, C. 2007. The Bayesian choice. Springer.
• Murphy, K. 2023. Probabilistic Machine Learning: Advanced Topics. MIT Press. pdf available at this link.
• Ghosal, S., & Van der Vaart, A. W. 2017. Fundamentals of nonparametric Bayesian inference. Cambridge University Press.