Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Learning to See by Looking at Noise

This is the official implementation of Learning to See by Looking at Noise (NeurIPS 2021, Spotlight).

In this work, we investigate a suite of image generation models that produce images from simple random processes. These are then used as training data for a visual representation learner with a contrastive loss. We study two types of noise processes, statistical image models and deep generative models under different random initializations.

[Project page] [Paper] [arXiv]


This version of code has been tested with Python 3.7.7 and pytorch 1.6.0. Other versions of pytorch are likely to work out of the box. The contrastive training requires two GPU's with at least 12GB of memory for the small scale experiments, while the large scale experiments require the same computation resources as the facebookresearch implementation of MoCo.

To use this repo, first clone it and correct the permissions for the scripts with:

git clone
cd learning_with_noise
chmod 755 -R scripts

To install all the requirements, simply do:

pip intall -r requirements.txt

Small scale experiments

To download all the small scale datasets, first run:


Then you can launch the contrastive training for all the small scale experiments with:

./scripts/train_align_uniform/ <GPU_ID_0> <GPU_ID_1>

If you just want to test the linear evaluation of the models (or do something else with them), you can directly download our pretrained encoders with:


Finally, you can evaluate the linear performance with imagenet100 as:

./scripts/train_align_uniform/ <path-to-imagenet100> <GPU_ID>

Where is the path to the imagenet100 dataset dir, which should contain two dirs (train and val) each with the train and val samples respectively for the 100 imagenet100 classes. If you have imagenet1k, you can generate imagenet100 using the following command (which will create simlyncs to your imagenet1k dir):

./scripts/generate_datasets/ <path-to-imagenet1k> <path-to-imagenet100>

Large scale experiments

If you just want to test the linear evaluation of the models (or do something else with them), you can directly download our pretrained Resnet-50 encoders with:


To download all large scale datasets, first run:


Then to train and evaluate MoCo v2 run, substituting EXP_NAME for the desired experiment name and IMAGENET_PYTORCH for your local copy of Imagenet-1k containing train/val folders:


echo "Computing main moco!"
python moco/ --batch-size 256 --mlp --moco-t 0.2 --aug-plus --cos --multiprocessing-distributed --world-size 1 --rank 0 --dist-url tcp://localhost:10043 \
--epochs 200 --restart-latest --result-folder $RESULTS_FOLDER \
--dataset_type imagefolder --log-wandb True --workers 65 \

echo "Computing linear eval!"
python moco/ -a resnet50 --batch-size 256 \
  --dist-url 'tcp://localhost:10043' --multiprocessing-distributed --world-size 1 --rank 0 --lr 30.0 \
  --pretrained $RESULTS_FOLDER/checkpoint_0199.pth.tar \
  --restart-latest \
  --result-folder $RESULTS_FOLDER/main_lincls/epoch_199 \

Data generation

The scripts to generate the datasets are in generate_datasets folder. A README on how to execute them can be found in the folder.


  title={Learning to See by Looking at Noise},
  author={Manel Baradad and Jonas Wulff and Tongzhou Wang and Phillip Isola and Antonio Torralba},
  booktitle={Advances in Neural Information Processing Systems},


Learning to See by Looking at Noise






No releases published


No packages published