Bayes_InfoTheory

This is a repository of Jupyter notebooks used by Florent Leclercq during lectures on Bayesian statistics and Information Theory. The homepage of the lectures is accessible here.

Contents

• Ignorance priors (exemplified with the lighthouse problem) and the maximum entropy principle
• Gaussian random fields:
  • Examples and a digression on non-Gaussianity
  • Bayesian signal processing and reconstruction: de-noising (example, example with the CMB)
  • Bayesian signal processing and reconstruction: de-blending
• Bayesian decision theory
• Markov Chain Monte Carlo:
  • Monte-Carlo integration, importance sampling, rejection sampling
  • Metropolis-Hastings algorithm & Gelman-Rubin test
  • Slice sampling
  • Gibbs sampling
  • Hamiltonian sampling
• Approximate Bayesian Computation:
  • Likelihood-free rejection sampling
  • Synthetic likelihood (parametric approximation)
  • Discrepancy and effective likelihood (non-parametric approximation)
• Information theory:
  • The noisy binary symmetric channel
  • Supervised Machine Learning basics: Titanic example

Acknowledgments

FL thanks Benjamin Wandelt for his own lectures, which have inspired a fraction of this material.

Warranty

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.