Skip to content
master
Switch branches/tags
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
api
 
 
 
 
 
 
 
 
 
 
 
 

README.md

AutoShrink: A Topology-aware NAS for Discovering Efficient Neural Architecture

Paper

Tunhou Zhang, Hsin-Pai Cheng, Zhenwen Li, Feng Yan, Chengyu Huang, Hai Li, and Yiran Chen. "AutoShrink: A Topology-aware NAS for Discovering Efficient Neural Architecture." arXiv Link: https://arxiv.org/abs/1911.09251

Description

This is the official implementation of our AAAI'20 paper: AutoShrink: A Topology-aware NAS for Discovering Efficient Neural Architecture. AutoShrink targets on efficient neural architecture search within only 1.5 GPU hours, which is at least 6.7x faster than the fastest existing NAS methods (ENAS). Moreover, AutoShrink faciliates architecture search for both CNN and RNN architectures, leading to efficient architectures of comparable accuracy of the SOTA models.

Workflow

Understand the code

The structure of this project is shown as follows:

├── api
|   ├── dataset
|   └── network
|   └── ...
├── data
|   ├── ...
├── evaluation
|   ├── cnn
|		   ├── base_estimator.py
|		   ├── ...
|   └── rnn
|		   ├── model.py
|		   ├── ...
├── searching
|   ├── cnn
|		   ├── solver
|		   ├── autoshrink_cnn.py
|		   ├── ...
|   └── rnn
|		   ├── autoshrink_rnn.py
|		   ├── ...
├── README.md

api: This is the low-level implementation of layers, training schemes etc. for the ShrinkCNN-series networks and CIFAR/ImageNet dataset.

data: This folder is used to hold the training dataset (i.e. CIFAR, ImageNet, Penn-Treebank etc.).

evaluation: This folder contains the evaluation code of the derived architectures from AutoShrink. Please refer to README.md inside for more detailed instructions.

searching: This folder contains the searching code of the AutoShrink. The subfolder cnn and rnn corresponds to CNN and RNN architecture search, respectively. Please refer to README.md inside for more detailed instructions.

Experimental Results

ShrinkCNNs

ShrinkCNNs are crafted within only 1.5 GPU hours. ~75.1%(73.9%) top-1 accuracy can be achieved using Swish/ReLU activations.

Architecture Top-1 Accuracy #Parameters #Multiply-Accumulates
ShrinkCNN-A (ReLU) 73.9% 3.6M 384M
ShrinkCNN-B (Swish) 75.1% 3.6M 384M

ShrinkRNNs

ShrinkRNNs are crafted within only 1.5 GPU hours.

Architecture Perplexity(valid) Perplexity(test) #Parameters
ShrinkRNN 58.5 56.5 24M

The implementation of ShrinkRNN is in part intuited by the DARTS project (Liu et al.).

Citation

If you would like to use any part of this code for research, please kindly cite our paper:

@inproceedings{zhang2019autoshrink,
title={AutoShrink: A Topology-aware NAS for Discovering Efficient Neural Architecture},
author={Zhang, Tunhou and Cheng, Hsin-Pai and Li, Zhenwen and Yan, Feng and Huang, Chengyu and Li, Hai and Chen, Yiran},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 2020)},
year={2019}
}

About

Official implementation of AutoShrink: A Topology-aware NAS for Discovering Efficient Neural Architecture

Resources

Releases

No releases published

Packages

No packages published

Languages