No description, website, or topics provided.
Switch branches/tags
Nothing to show
Clone or download
Latest commit 20b87dc Sep 13, 2016
Failed to load latest commit information.
.gitignore Initial commit Dec 29, 2015
LICENSE Initial commit Dec 29, 2015 Adding groups Jan 19, 2016 Initial commit Dec 29, 2015 Adding input layer Sep 12, 2016 Just load the net if --data unspecified (#1) May 16, 2016

Data-dependent initialization of convolutional neural networks

Created by Philipp Krähenbühl.


This code implements the initialization presented in our arXiv tech report, which is under submission at ICLR 2016.

This is a reimplementation and currently work in progress. Use at your own risk.


This code is released under the BSD License (refer to the LICENSE file for details).


If you find our initialization useful in your research, please consider citing:

  title={Data-dependent Initializations of Convolutional Neural Networks},
  author={Kr{\"a}henb{\"u}hl, Philipp and Doersch, Carl and Donahue, Jeff and Darrell, Trevor},
  journal={arXiv preprint arXiv:1511.06856},


Checkout the project and create a symlink to caffe in the magic_init directory:

ln -s path/to/caffe/python/caffe caffe


Here is a quick example on how to initialize alexnet:

python path/to/alexnet/deploy.prototxt path/to/output.caffemodel -d "path/to/some/images/*.png" -q -nit 10 -cs

Here -d flag allows you to initialize the network using your own images. Feel free to use imagenet, Pascal, COCO or whatever you have at hand, it shouldn't make a big difference. The -q (queit) flag suppresses all the caffe logging, -nit controls the number of batches used (while -bs controls the batch size). Finally -cs rescales the gradients accross layers. This rescaling currently works best for feed-forward networks, and might not work too well for DAG structured networks (we are working on that).

To run the k-means initialization use:

python path/to/alexnet/deploy.prototxt path/to/output.caffemodel -d "path/to/some/images/*.png" -q -nit 10 -cs -t kmeans

Finally, python -h should provide you with more help.

Pro tips

If you're numpy implementation is based on openblas, try disabeling threading export OPENBLAS_NUM_THREADS=1, it can improve the runtime performance a bit.