Skip to content

Latest commit

 

History

History
11 lines (8 loc) · 1.97 KB

Liu2020MemNAS.md

File metadata and controls

11 lines (8 loc) · 1.97 KB

Title

MemNAS: Memory-Efficient Neural Architecture Search With Grow-Trim Learning

Author

Peiye Liu, Bo Wu, Huadong Ma, Mingoo Seok

Abstract

Recent studies on automatic neural architecture search techniques have demonstrated significant performance, competitive to or even better than hand-crafted neural architectures. However, most of the existing search approaches tend to use residual structures and a concatenation connection between shallow and deep features. A resulted neural network model, therefore, is non-trivial for resource-constraint devices to execute since such a model requires large memory to store network parameters and intermediate feature maps along with excessive computing complexity. To address this challenge, we propose MemNAS, a novel growing and trimming based neural architecture search framework that optimizes not only performance but also memory requirement of an inference network. Specifically, in the search process, we consider running memory use, including network parameters and the essential intermediate feature maps memory requirement, as an optimization objective along with performance. Besides, to improve the accuracy of the search, we extract the correlation information among multiple candidate architectures to rank them and then choose the candidates with desired performance and memory efficiency. On the ImageNet classification task, our MemNAS achieves 75.4% accuracy, 0.7% higher than MobileNetV2 with 42.1% less memory requirement. Additional experiments confirm that the proposed MemNAS can perform well across the different targets of the trade-off between accuracy and memory consumption.

Bib

@INPROCEEDINGS{9156558, author={P. {Liu} and B. {Wu} and H. {Ma} and M. {Seok}}, booktitle={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, title={MemNAS: Memory-Efficient Neural Architecture Search With Grow-Trim Learning}, year={2020}, volume={}, number={}, pages={2105-2113}, doi={10.1109/CVPR42600.2020.00218}}