This is our code repository for our Final Project for ORIE 6741 Bayesian Machine Learning Class.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
experiments
Bayesian_CNN.png
LeNet- MCDropout-final version.ipynb
README.md
lenet5.ipynb

README.md

A Bayesian framework for Convolutional Neural Networks using Stochastic Gradient Hamiltonian Monte Carlo

This project we formulate a Bayesian framework for Convolutional Neural Networks using Stochastic Gradient Hamiltonian Monte Carlo(SGHMC)

Link to report here: https://www.overleaf.com/read/zcbkcbmgcvwp

This repository implements a Bayesian CNN framework using SGHMC. We compare our method to MC-Dropout. Experiments and models were developed using Tensorflow and Edward.

Abstract:

Convolutional Neural Networks(CNN's) are powerful image processing models that have no estimates of uncertainty. (Gal & Ghahramani, 2015) developed efficient estimation of CNN's weight uncertainty using variational approximation. Although this method is promising, variational approximation is known to underestimate the true distribution and does not fit well on multi-modal distributions (Gal & Ghahramani, 2015). The purpose of this study is to explore better inference methods, such as Stochastic Gradient Hamiltonian Monte Carlo, to better approximate model posterior of Convolutional Neural Networks architectures and improve class label prediction.

#ToDo

  1. Conducting model criticism by analyzing the posterior predictive distribution. Tutorial here: http://edwardlib.org/tutorials/criticism

One check would be a PPC: Posterior predictive checks (PPCs) analyze the degree to which data generated from the model deviate from data generated from the true distribution.

For more interesting links of probabalistic inference: Look at the Edward Library:http://edwardlib.org/ Bayesian Method for Hackers: https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers