Skip to content
Code release for paper "Random Search and Reproducibility for NAS"
Branch: master
Clone or download
liam liam
liam and liam add supplement for UAI
Latest commit 6513a0a Jun 30, 2019


Code release for paper "Random Search and Reproducibility for NAS."

This code base requires the following additional repositories:

Random seeds and additional documentation needed to reproduce the results are provided in this spreadsheet.

You will need the following packages to run the code:
Python >= 3.5.5, PyTorch == 0.3.1, torchvision == 0.2.0, CUDA 9.0, CuDNN 7.1.3

Download the CIFAR-10 dataset and PTB dataset according to how DARTS expects the files to be.

Random Search with Weight Sharing

Be sure to set the right paths for the darts repository and data files in the benchmark files:

To run, issue the command with the desired arguments to replicate runs from the spreadsheet:
python --benchmark [ptb/cnn] --seed [X] --epochs [X] --batch_size [X] --grad_clip [X] --save_dir [X] --config [controls ptb hidden dim] --init_channels [controls cnn size]

Please use our fork of DARTS in order to get deterministic results for CNN evaluation. The output architecture for PTB can be copied directly into darts/rnn/ for eval. For CNN, run the following command to get the architecture to copy into darts/cnn/ for eval:
python "[arch_str]"

Note that the exact reproducibility of random search with weight sharing is hardware dependent; our CNN results are reproducible on Tesla V100 GPUs and our RNN results are reproducible on Tesla P100 GPUs.


Please follow the directions in the darts_asha repo to run these experiments.

You can’t perform that action at this time.