Introduction to Bayesian Statistical Learning

Table of Contents

1. Description
2. Information
3. File descriptions
4. Certificate

Description

When observing data, the key question is: What can I learn from the observation? Bayesian inference treats all parameters of the model as random variables. The main task is to update their distribution as new data is observed. Hence, quantifying uncertainty of the parameter estimation is always part of the task. In this course we will introduce the basic theoretical concepts of Bayesian Statistics and Bayesian inference. We discuss the computational techniques and their implementations, different types of models as well as model selection procedures. We will exercise on the existing datasets using the PyMC3 framework for practicals.

Information

The overall goals of this course were the following:

• Bayes theorem, Prior and Posterior distributions;
• Computational challenges and techniques: MCMC, variational approaches;
• Models: mixture models, Gaussian processes, neural networks;
• Bayesian model selection: Bayes factor and others;
• PyMC3 framework for Bayesian computation;
• Running Bayesian models on a Supercomputer;

More detailed information, links and software setup for the course can be found on the course website.

File descriptions

The description of the files in this repository can be found below:

• Day 1 and 2 - Bayes theorem, posterior distributions, working with PyM and more PyMC examples:
  • Lecture_1_examples - Notebook (Introduction);
  • BLcourse1.2 - Notebook (More PyMC3 examples);
  • separation_plot - Script;
  • daft_plot - Script;
  • LBLcourse1 - Slides;
    
    

---

• Day 3 - Markov chain Monte Carlo (MCMC) methods:
  • Lecture2_comp - Notebook (MCMC, Laplace approximation);
  • BLcourse2 - Slides;
    
    

---

• Day 4 - Bayesian optimization and variational inference:
  • avb_gaussian - Notebook (Analytic Variational Bayes, Inferring a single Gaussian);
  • svb_gaussian_tf2 - Notebook (Stochastic Variational Bayes);
  • svb_biexp_tf2 - Notebook (Stochastic Variational Bayes - example nonlinear model);
  • BLcourse3 - Slides;
    
    

    ---

• Day 5 - Bayes and generative ML models: Variational autoencoders, Normalizing flows, Gaussian processes, other topics not covered yet:
  • 01_simple_gp_regression - Notebook (Regression);
  • 01_simple_gp_regression - Script (Regression);
  • bayesian_neural_networks_wine - Notebook (Probabilistic Bayesian Neural Networks);
  • flows - Notebook (Normalizing Flows);
  • vae_mod - Notebook (Variational Autoencoder);
  • BLcourse4 - Slides;
    
    

Certificate

The certificate for the workshop can be found below:

"Introduction to Bayesian Statistical Learning" - Jülich Supercomputing Centre (JSC) (Issued On: April 2023)