Bayesian Deep Learning

[howardyclo]

Last Update Date: 2020/03/22

Tutorials and Seminars

• Andrew Gordon Wilson on Bayesian deep learning; Primer: Pavel Izmailov and Polina Kirichenko. Broad Institute of MIT and Harvard. [Slides].
• Bayesian ML/DL Playlist by Bayesian Methods Research Group from Moscow, Russia.

PhD Thesis

• Bayesian Method for Adaptive Models. David J.C. MacKay (1992)
• Bayesian Learning for Neural Networks. Lecture Notes in Statistics. Neal, R. M. (1996)
• Uncertainty in Deep Learning. Yarin Gal (2016)

Uncertainty

2015

• Weight Uncertainty in Neural Networks by Charles Blundell. ICML 2015.

2016

• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning by Yarin Gal. ICML 2016.

2017

• On Calibration of Modern Neural Networks by Chuan Guo et al. ICML 2017.
• Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles by Lakshminarayanan et al. NIPS 2017.
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? by Alex Kendall & Yarin Gal. NIPS 2017.

2018

• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics by Alex Kendall et al. CVPR 2018.
• Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flow by Ilg et al. ECCV 2018.
• Accurate Uncertainties for Deep Learning Using Calibrated Regression by Volodymyr Kuleshov et al. ICML 2018.
• Predictive Uncertainty Estimation via Prior Networks by Andrey Malinin et al. NeurIPS 2018.

2019

• Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers by Geifman et al. ICLR 2019.
• Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem by Matthias Hein et al. CVPR 2019.
• Noise Contrastive Priors for Functional Uncertainty by Danijar Hafner et al. UAI 2019.
• Sampling-free Uncertainty Estimation in Gated Recurrent Units with Applications to Normative Modeling in Neuroimaging by Seong Jae Hwang et al. UAI 2019.
• Sampling-free Epistemic Uncertainty Estimation Using Approximated Variance Propagation by Janis Postels et al. ICCV 2019.
• Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift by Yaniv Ovadia et al. NeurIPS 2019.
• A Simple Baseline for Bayesian Uncertainty in Deep Learning by Maddox et al. NeurIPS 2019.
• Single-Model Uncertainties for Deep Learning by Natasa Tagasovska & David Lopez-Paz. NeurIPS 2019.
• Verified Uncertainty Calibration by Ananya Kumar, Percy Liang, Tengyu Ma. NeurIPS 2019.

2020

• Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning by Arsenii Ashukha, +2, Dmitry Vetrov. ICLR 2020.
• Scalable Uncertainty for Computer Vision with Functional Variational Inference by Carvalho et al. CVPR 2020.

Generalization

2017

• Stochastic Gradient Descent as Approximate Bayesian Inference by Stephan Mandt et al. JMLR 2017.

2018

• Averaging Weights Leads to Wider Optima and Better Generalization by Pavel Izmailov, +2, Dmitry Vetrov, Andrew Gordon Wilson. UAI 2018.

2019

• Understanding Priors in Bayesian Neural Networks at the Unit Level by Vladimirova et al. ICML 2019.
• The Deep Weight Prior by Andrei Atanov, +2, Dmitry Vetrov, Max Welling. ICLR 2019.
• Functional Variational Bayesian Neural Networks by Sun, +2, Roger Grosse. ICLR 2019.
• Subspace Inference for Bayesian Deep Learning by Pavel Izmailov, +3, Dmitry Vetrov, Andrew Gordon Wilson. UAI 2019.
• Practical Deep Learning with Bayesian Principles by Kazuki Osawa & Emtiyaz Khan. NeurIPS 2019.
• Approximate Inference Turns Deep Networks into Gaussian Processes by Emtiyaz Khan et al. NeurIPS 2019.

2020

• Bayesian Deep Learning and a Probabilistic Perspective of Generalization by Andrew Gordon Wilson.
• Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited by Wesley J. Maddox, +1, Andrew Gordon Wilson.