Skip to content
Unsupervised Learning by Predicting Noise
Branch: master
Clone or download
piotr-bojanowski Merge pull request #2 from facebookresearch/automated_fixup_code_of_c…

Adding Code of Conduct file
Latest commit ad8f9f7 May 27, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore Initial commit. Aug 23, 2017 OSS Automated Fix: Addition of Code of Conduct Mar 22, 2019 OSS Automated Fix: Addition of Contributing Mar 22, 2019
LICENSE Initial commit. Aug 23, 2017
Makefile Initial commit. Aug 23, 2017 changed the path to public models Jan 21, 2019
alexnet-specs.lua Initial commit. Aug 23, 2017
alexnet.lua Initial commit. Aug 23, 2017
hungarian.lua Initial commit. Aug 23, 2017
imagenet.lua Initial commit. Aug 23, 2017
main.lua Initial commit. Aug 23, 2017 Initial commit. Aug 23, 2017
make-index.lua Initial commit. Aug 23, 2017 Initial commit. Aug 23, 2017
optimutil.lua Initial commit. Aug 23, 2017 changed the path to public models Jan 21, 2019
test.lua Initial commit. Aug 23, 2017 Initial commit. Aug 23, 2017

Unsupervised Learning by Predicting Noise

This is the code used for unsupervised training of convolutional neural networks as described in the ICML 2017 paper Unsupervised Learning by Predicting Noise (arXiv).

The code is composed of two modules, one for unsupervised feature learning, and one for training a supervised classifier on top of them.


The code is in Lua Torch, and therefore requires a working torch installation. It was tested with a LuaJIT installation obtained using:

git clone
cd luajit-rocks
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=/your/prefix

where /your/prefix is the path where you want a local install of LuaJIT and all dependencies. In order to run this code, you will need the following modules:


You can install them using:

$ /your/prefix/bin/luarocks install [package]

Compiling the C code

Computing the optimal permutation of targets requires solving an assignment problem. We do so using the Hungarian algorithm, also known as the Kuhn-Munkres algorithm. We provide a C implementation and the corresponding Lua interface (using ffi). In order to use it, please run $ make which will compile the C code and create the shared library.

Getting data

This code uses an IndexedDataset to load data. This is a TorchNet data structure which provides an efficient way to bundle data into a single archive file, associated with an indexed file. Given a copy of ImageNet, you can create the archive using:

$ ./

Inside this script, please replace the source and destination paths as follows:


Where /data/users/bojanowski/data/imagenet-full-size is where you store your copy of ImageNet images:

$ ls -1 /data/users/bojanowski/data/imagenet-full-size

This will produce indexed archives (roughly 70Gb for training images) in the ${DEST} folder.

Running the unsupervised training

Unsupervised training can be launched by running:

$ ./

Please specify the location of your LuaJIT installation in the script:


Please also provide the path to the data folder where the imagenet-idx folder is located:


Finally, you can specify where you want to save the logs, checkpoints and models using:


During training, checkpoints and logs will be saved to the directory specified in ${EXP}. A new directory is created for each new set of parameters. This directory of results contains the following things:

$ ls -1 exp/unsup-dim-2048-perm-3-lr-1.0e-02

We have implemented rolling checkpoints, where a version of the model and the corresponding codes are saved every epoch, in a rolling fashion. The files checkpoint.bin and checkpointModel.bin are symbolic links to the latest checkpoint available. Models are also saved every other k epochs (set using the -saveperiod flag), and can be found in for instance in `epoch-00010'.

A complete list of options can be obtained using:

$ luajit main.lua -h

Learning the MLP on the supervised task

Once the unsupervised training is finished, you can launch the transfer task using:

$ ./

You need to specify the path to the ImageNet data and the directory in which your model lies. The code should run the supervised training of the MLP in the AlexNet and log the train and val performance along epochs. The output of this code will be saved in a subdirectory of where the model is.

The full set of parameters for this supervised training can be obtained using:

$ luajit test.lua -h

ImageNet classification with a pre-trained model

We provide a pre-trained model, available for download here. You can run the transfer task with a pre-trained model by running:

$ ./

This will download the pre-trained model and launch the transfer learning on ImageNet.

You can’t perform that action at this time.