Skip to content
Adaptive Stochastic Natural Gradient Method for One-Shot Neural Architecture Search
Branch: master
Clone or download
Latest commit a13c482 May 29, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
classification
inpainting upload May 20, 2019
toy upload May 20, 2019
.gitignore upload May 20, 2019
LICENSE Initial commit May 9, 2019
README.md update README.md May 29, 2019

README.md

Adaptive Stochastic Natural Gradient Method for One-Shot Neural Architecture Search

This repository contains the code for the following paper:

Youhei Akimoto, Shinichi Shirakawa, Nozomu Yoshinari, Kento Uchida, Shota Saito, and Kouhei Nishida: Adaptive Stochastic Natural Gradient Method for One-Shot Neural Architecture Search, 36th International Conference on Machine Learning (ICML), pp. 171-180 (2019) [PMLR] [arXiv]

If you use this code for your research, please cite our paper:

@inproceedings{AkimotoICML2019,
    author = {Youhei Akimoto and Shinichi Shirakawa and Nozomu Yoshinari and Kento Uchida and Shota Saito and Kouhei Nishida},
    title = {Adaptive Stochastic Natural Gradient Method for One-Shot Neural Architecture Search},
    booktitle = {Proceedings of the 36th International Conference on Machine Learning (ICML)},
    pages = {171--180},
    year = {2019}
}

Directory structure

.
├── README.md
├── classification (source codes of 3.2. CIFAR-10 classification task)
├── inpainting (source codes of 3.3. Celeb-A inpainting task)
└── toy (source codes of 3.1. Toy Problem)

Requirements

We used the PyTorch version 0.4.1 for neural network implementation. We tested the codes on the following environment:

  • Ubuntu 16.04 LTS
  • GPU: NVIDIA GeForce GTX 1080Ti (for image classification and inpainting)
  • CUDA version 9.2
  • Python 3.6.5
  • Python package version
    • numpy 1.14.3
    • scipy 1.1.0
    • torch 0.4.1
    • torchvision 0.2.1
    • scikit-image 0.14

Usage

Classification (experiment of section 3.2)

$ cd classification
$ python main_classification.py
  1. move to the directory of classification
  2. run the program as python main_classification.py
    • If you want to run the code with a different setting, please directly modify the function call of experiment in python main_classification.py

Inpainting (experiment of section 3.3)

$ cd inpainting
$ python main_inpainting_cat.py -d celebA -p ~/data/ -c Center -g 0 -o ./result/

The main script main_inpainting_cat.py corresponds to the architecture encoding method using categorical variables, and main_inpainting_int.py corresponds to the architecture encoding method using the mix of categorical and integer variables. Please see our paper in details.

  1. move to the directory of inpainting
  2. prepare the celebA image dataset (train / test splitting)
  3. run the program as python main_inpainting_cat.py -d celebA -p ~/data/ -c Center -g 0 -o ./result/
    • -p : path to the celebA dataset
    • -c : mask type ('Center' or 'RandomPixel' or 'RandomHalf')
    • -g : gpu id
    • -o : output directory

Toy Problem (experiment of section 3.1)

$ cd toy
$ python main_toy.py
  1. move to the directory of toy
  2. run the program as python main_toy.py (this program is run on CPU)
    • default: ASNG with $eps_x=0.05$, $\alpha=1.5$, and $\delta^{0}_{\theta}=1.0$
    • If you want to run the code with different setting, please directly modify the following function call in python main_benchmark.py experiment(alg='ASNG', eta_x=0.05, eta_theta_factor=0., alpha=1.5, K=5, D=30, maxite=100000)
You can’t perform that action at this time.