Skip to content
Torch code for our CVPR 2016 paper "Joint Unsupervised LEarning of Deep Representations and Image Clusters"
Branch: master
Clone or download
jwyang Merge pull request #16 from crazylyf/master
Fix infinite iteration in `organize_samples`
Latest commit d43251d Sep 20, 2017
Type Name Latest commit message Commit time
Failed to load latest commit information.
affinity Delete affinity.lua~ Jun 2, 2016
agg_clustering remove files May 16, 2016
agg_clustering_c remove files May 16, 2016
criterions add code May 25, 2016
datasets add datasets May 25, 2016
evaluate add code May 25, 2016
matlab Delete nmi_jwy.m~ Jun 25, 2016
models_def modify YTF Jun 2, 2016
LICENSE Initial commit Apr 8, 2016 readme Mar 7, 2017
train.lua Fix infinite iteration in `organize_samples` Sep 19, 2017

Joint Unsupervised Learning (JULE) of Deep Representations and Image Clusters.


This project is a Torch implementation for our CVPR 2016 paper, which performs jointly unsupervised learning of deep CNN and image clusters. The intuition behind this is that better image representation will facilitate clustering, while better clustering results will help representation learning. Given a unlabeled dataset, it will iteratively learn CNN parameters unsupervisedly and cluster images.


This is a torch version reimplementation to the code used in our CVPR paper. There is a slight difference between the code used to report the results in our paper. The Caffe version code can be found here.


This code is released under the MIT License (refer to the LICENSE file for details).


If you find our code is useful in your researches, please consider citing:

    Author = {Yang, Jianwei and Parikh, Devi and Batra, Dhruv},
    Title = {Joint Unsupervised Learning of Deep Representations and Image Clusters},
    Booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    Year = {2016}


  1. Torch. Install Torch by:

    $ curl -s | bash
    $ git clone ~/torch --recursive
    $ cd ~/torch; 
    $ ./      # and enter "yes" at the end to modify your bashrc
    $ source ~/.bashrc

    After installing torch, you may also need install some packages using LuaRocks:

    $ luarocks install nn
    $ luarocks install image 

    It is preferred to run the code on GPU. Thus you need to install cunn:

    $ luarocks install cunn
  2. lua-knn. It is used to compute the distance between neighbor samples. Go into the folder, and then compile it with:

    $ luarocks make

Typically, you can run our code after installing the above two packages. Please let me know if error occurs.

Train model

  1. It is very simple to run the code for training model. For example, if you want to train on USPS dataset, you can run:

    $ th train.lua -dataset USPS -eta 0.9

    Note that it runs on fast mode by default. You can change it to regular mode by setting "-use_fast 0". In the above command, eta is the unfolding rate. For face dataset, we recommand 0.2, while for other datasets, it is set to 0.9 to save training time. During training, you will see the normalize mutual information (NMI) for the clustering results.

  2. You can train multiple models in parallel by:

    $ th train.lua -dataset USPS -eta 0.9 -num_nets 5

    By this way, you weill get 5 different models, and thus 5 possible different results. Statistics such as mean and stddev can be computed on these results.

  3. You can also get the clustering performance when using raw image data and random CNN by

    $ th train.lua -dataset USPS -eta 0.9 -updateCNN 0
  4. You can also change other hyper parameters for model training, such as K_s, K_c, number of epochs in each partial unrolled period, etc.


We upload six small datasets: COIL-20, USPS, MNIST-test, CMU-PIE, FRGC, UMist. The other large datasets, COIL-100, MNIST-full and YTF can be found in my google drive here.

Train on your own datasets

Alternatively, you can train the model on your own dataset. As preparations, you need:

  1. Create a hdf5 file with size of NxCxHxW, where N is the total number of images, C is the number of channels, H is the height of image, and W the width of image. Then move it to datasets/dataset_name/data4torch.h5

  2. Create a lua file to define the network architecture for your dataset. Put it in models_def/dataset_name.lua.

  3. Afterwards, you can run train.lua by specifying the dataset name as your own dataset. That's it!

Compared Approaches

We upload the code for the compared approaches in matlab folder. Please refer to the original paper for details and cite them properly. In this foler, we also attach the evaluation code for two metric: normalized mutual information (NMI) and clustering accuracy (AC).


You are welcome to send message to (jw2yang at if you have any issue on this code.

You can’t perform that action at this time.