Data-Free-NAS

This is the pytorch implementation of our paper "Data-Free Neural Architecture Search via Recursive Label Calibration", published in ECCV 2022.

This paper aims to explore the feasibility of neural architecture search (NAS) without original data, given only a pre-trained model. Our results demonstrate that the architectures discovered by our data-free NAS achieve comparable accuracy as architectures searched from the original natural data. This derives the conclusion that NAS can be done effectively and data-freely.

Citation

If you find our code useful for your research, please consider citing:

@inproceedings{liu2022data,
  title={Data-Free Neural Architecture Search via Recursive Label Calibration},
  author={Liu, Zechun and Shen, Zhiqiang and Long, Yun and Xing, Eric and Cheng, Kwang-Ting and Leichner, Chas},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2022}
}

Run

1. Requirements:

• python 3.6, pytorch 1.7.1, torchvision 0.8.2

2. Steps to run:

(1) Step 1: image synthesis

• Put the pretrained ResNet-50 into the folder ./Image_Synthesis/models
• Change directory to ./Image_Synthesis/
• run bash run.sh

(2) Step 2: neural architecture search

Step 2.0: split the synthesized data into the training set for supernet training and validation set for evolutionary search

• Change directory to ./NAS/
• Run python split.py

Step 2.1: supernet training

• Change directory to ./NAS/Supernet/
• Run bash run.sh

Step 2.2: evolutionary search

• Change directory to ./NAS/Search/
• Run bash run_search.sh

Step 2.3: evaluation

• Change directory to ./NAS/Evaluation/
• Run bash run_eval.sh and it will automatically generate a folder containing the searched architecture and the corresponding training code in ./data.
• Change directory to ./NAS/Evaluation/data/${architecture}
• Run python train.py

Models and synthesized data

1. Pretrained ResNet-50 model: ResNet-50

2. Synthesized images: Data

3. Searched model and final results:

Methods	Top1-Err (%)	FLOPs (M)	Data for NAS
Single Path One-Shot (SPOS)	25.7	319	ImageNet
Data-Free SPOS	25.8	316	Synthesized data

Contact

Zechun Liu, HKUST (zliubq at connect.ust.hk)