Skip to content
Code for the NIPS paper 'Natural-Parameter Networks: A Class of Probabilistic Neural Networks'
MATLAB
Branch: master
Clone or download
Latest commit 3765d49 Jun 20, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
example first commit May 1, 2019
figures Add files via upload Jun 20, 2019
.gitignore update May 12, 2019
Esig.m first commit May 1, 2019
README.md Update README.md Jun 20, 2019
README.md~ update fig May 12, 2019
cv_mlp_classify_bayes.m first commit May 1, 2019
default_mlp.m first commit May 1, 2019
default_mlp_bayes.m first commit May 1, 2019
dkappa.m first commit May 1, 2019
dsigmoid.m first commit May 1, 2019
kappa.m first commit May 1, 2019
lapnn_get_m_s.m first commit May 1, 2019
logdiff.m first commit May 1, 2019
logsum.m first commit May 1, 2019
mc_mlp_classify.m first commit May 1, 2019
mc_mlp_classify_bayes.m first commit May 1, 2019
mcw_mlp_classify_bayes.m first commit May 1, 2019
mlp.m first commit May 1, 2019
mlp_bayes.m first commit May 1, 2019
mlp_classify.m first commit May 1, 2019
mlp_classify_bayes.m first commit May 1, 2019
print_n_updates.m first commit May 1, 2019
save_intermediate.m first commit May 1, 2019
sigmoid.m first commit May 1, 2019
softmax.m first commit May 1, 2019

README.md

Natural-Parameter Networks (NPN) in Matlab

This is the code for the NIPS paper 'Natural-Parameter Networks: A Class of Probabilistic Neural Networks'.

It is a class of probabilistic neural networks that treat both weights and neurons as distributions rather than just points in high-dimensional space. Distributions are first-citizens in the networks. The design allows distributions to feedforward and backprop across the network. Given an input data point, NPN will output a predicted distribution with information on both the prediction and uncertainty.

NPN can be used either independently or as a building block for Bayesian Deep Learning (BDL).

Note that this is the code for Gaussian NPN to run on the MNIST and Boston Housing datasets. For Gamma NPN or Poisson NPN please go to the other repo.

Neural networks v.s. natural-parameter-networks in two figures:

Distributions as first-class citizens:

Closed-form operations to handle uncertainty:

Example results:

Above is the predictive distribution for NPN. The shaded regions correspond to 3 standard deviations. The black curve is the data-generating function and blue curves show the mean of the predictive distributions. Red stars are the training data.

How to run the code:

To train the model, run the command: 'cd example' to go to the directory of the entry point and run './run.sh' or 'sh run.sh'

example/run.sh: entry point mlp_bayes.m: core model code default_mlp_bayes.m: initialize NPN

PyTorch implementation:

Official PyTorch version of NPN can be found here.

Other implementations (third-party):

Pytorch/Python code (with extension to GRU) by sohamghosh121.

Reference:

Natural-Parameter Networks: A Class of Probabilistic Neural Networks

@inproceedings{DBLP:conf/nips/WangSY16,
  author    = {Hao Wang and
               Xingjian Shi and
               Dit{-}Yan Yeung},
  title     = {Natural-Parameter Networks: {A} Class of Probabilistic Neural Networks},
  booktitle = {Advances in Neural Information Processing Systems 29: Annual Conference
               on Neural Information Processing Systems 2016, December 5-10, 2016,
               Barcelona, Spain},
  pages     = {118--126},
  year      = {2016}
}
You can’t perform that action at this time.