Introduction to Deep Rewiring in tensorflow
This repository provides basic implementations of Deep Rewiring in Tensorflow. The scripts are showing how to solve Mnist with networks that are kept very sparse all along the training (less than 2% of the connections are active), more advanced simulations and mathematical analyses are described in our paper. The four scripts are:
script_mnist_deep_rewiring.py: Basic implementation of DEEP R where the number of non-zero in each individual matrix is contrained
script_mnist_deep_rewiring_with_global_constraint.py: Same with a constraint on the global number of connections
script_mnist_deep_rewiring_with_sparse_matrices.py: Same as the first script, but using the tensorflow sparse matrices
script_mnist_soft_deep_rewiring.py: An implementation of the soft-DEEP R algorithm used as baseline in our paper
"Deep Rewiring: Training very sparse deep networks"
Guillaume Bellec, David Kappel, Wolfgang Maass, Robert Legenstein