Skip to content
master
Switch branches/tags
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 

README.md

Code for the paper:

Gaussian Process Behaviour in Wide Deep Neural Networks Alexander G. de G. Matthews, Mark Rowland, Jiri Hron, Richard E. Turner and Zoubin Ghahramani

Which can be found at:

https://arxiv.org/abs/1804.11271

Note that this paper is substantially expanded from the original ICLR version.

The latest version of the experiments uses:

TensorFlow 1.9.0 PyTorch 0.3.0 GPflow 0.5.0

The code is released under the Apache 2.0 license.

The newer code uses a more numerically stable variant of the Deep ReLu kernel built on GPflow. This enables us to backpropagate gradients for type II maximum likelihood.

The random number generator in PyTorch has changed significantly across versions and this can effect numerical values, though not the qualitative conclusions.

An earlier version of the code had an inconsistency in the scaling of the input weight variances between the neural network and Gaussian process. This had a negligible effect on the experiments where it was used.

Note that the HMC experiments are particularly slow. Mixing is verified using the autocorrelation estimators included as part of PyHMC. We work in terms of the mixing of the predictive distribution since the weight posterior will have combinatorically many symmetric modes that are predictively equivalent.

About

Code for the paper Gaussian process behaviour in wide deep networks

Resources

License

Releases

No releases published

Packages

No packages published

Languages