awesome_nas.md

Architecture Search

Labels

• Optimization Method
  • RL--> Reinforcement Learning
  • EA--> Evolution Algorithm
  • GD--> Gradient Descent
  • BO--> Bayesian Optimisation
  • MCTS --> Monte Carlo Tree Search
  • SMBO --> Sequential Model-Based Optimization
  • 1S --> 1-Shot Learning
• Objective Function
  • DE --> DEvice-related: inference time, memory usage, power consumption
• Training
  • FT --> FineTune on pretrained models
  • SR --> ScRatch
  • TL --> Transfer Learning between tasks
  • PS --> Parameter Sharing
  • NM --> Network Morphisms
  • KT --> Knowledge Transfer
• Search Level
  • TP --> ToPology of connection paths
  • SG --> SubGraph within a large computational graph
  • SM --> Frequent Computational Subgraph Mining
  • RNN
  • SE --> Shink and Expand
  • BC --> Block-wise Component
  • ML --> Modeling Language
  • MM --> Modularized Morphing
• Accurarcy Computation
  • PP --> Performance Prediction
  • ST --> STatistics derived from filter feature maps
  • WP --> Weight Prediction

2018

Neural Architecture Optimization
[arXiv:1808.07233]

Designing Adaptive Neural Networks for Energy-Constrained Image Classification
[arXiv:1808.01550]

Reinforced Evolutionary Neural Architecture Search
[arXiv:1808.00193]

MnasNet: Platform-Aware Neural Architecture Search for Mobile
[arXiv:1807.11626]

MaskConnect: Connectivity Learning by Gradient Descent
[arXiv:1807.11473] [ECCV'18]

Efficient Neural Architecture Search with Network Morphism
[arXiv:1806.10282] [code]

DARTS: Differentiable Architecture Search :star:
[arXiv:1806.09055] [code] --> GD

DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures
[arXiv:1806.08198] [ICLR'18 Workshop] --> DE

Path-Level Network Transformation for Efficient Architecture Search
[arXiv:1806.02639] [code] [ICML'18] --> FT TP RL

AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search
[arXiv:1805.07440] --> SR MCTS PP

Neural Architecture Construction using EnvelopeNets
[arXiv:1803.06744] --> ST

Transfer Automatic Machine Learning
[arXiv:1803.02780] --> RL TL

Neural Architecture Search with Bayesian Optimisation and Optimal Transport
[arXiv:1802.07191] --> BO

Efficient Neural Architecture Search via Parameter Sharing :star:
[arXiv:1802.03268] [code] [ICML'18] --> PS RL SG

Regularized Evolution for Image Classifier Architecture Search
[arXiv:1802.01548] --> EA

GitGraph - from Computational Subgraphs to Smaller Architecture Search Spaces
[arXiv:1801.05159] [ICLR'18 Workshop] --> SM

A Flexible Approach to Automated RNN Architecture Generation
[arXiv:1712.07316] [ICLR'18 Workshop] --> RNN

Peephole: Predicting Network Performance Before Training :star:
[arXiv:1712.03351] --> PP

Progressive Neural Architecture Search
[arXiv:1712.00559] --> SMBO

MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks :star:
[arXiv:1711.06798] --> SE

Simple And Efficient Architecture Search for Convolutional Neural Networks :star:
[arXiv:1711.04528] [ICLR'18 Workshop] --> NM

Hierarchical Representations for Efficient Architecture Search
[arXiv:1711.00436] [ICLR'18] --> EA TP

Practical Block-wise Neural Network Architecture Generation
[arXiv:1708.05552] [CVPR'18] --> BC

SMASH: One-Shot Model Architecture Search through HyperNetworks
[arXiv:1708.05344] [code] [ICLR'18] --> WP

Learning Transferable Architectures for Scalable Image Recognition
[arXiv:1707.07012] --> BC

Efficient Architecture Search by Network Transformation
[arXiv:1707.04873] [code] [AAAI'18] --> FT RL SE

Learning Time/Memory-Efficient Deep Architectures with Budgeted Super Networks :star:
[arXiv:1706.00046] [code] [CVPR'18] --> DE GD

Accelerating Neural Architecture Search using Performance Prediction
[arXiv:1705.10823] [ICLR'18 Workshop] --> PP

Understanding and Simplifying One-Shot Architecture Search :star:
[ICML'18] --> 1S

2017

DeepArchitect: Automatically Designing and Training Deep Architectures
[arXiv:1704.08792] [code] --> ML

Genetic CNN
[arXiv:1703.01513] [code] [ICCV'17] --> EA

Modularized Morphing of Neural Networks
[arXiv:1701.03281] [ICLR'17 Workshop] --> FT MM

Large-Scale Evolution of Image Classifiers
[arXiv:1703.01041] [ICML'17] --> EA

Designing Neural Network Architectures using Reinforcement Learning
[arXiv:1611.02167] [code] [ICLR'17] --> RL

Learning Curve Prediction with Bayesian Neural Networks
[ICLR'17] --> PP

Neural Architecture Search with Reinforcement Learning
[arXiv:1611.01578] [code (3rd)] [ICLR'17] --> RL

2016

Convolutional Neural Fabrics
[arXiv:1606.02492] [code:Caffe] [code:PyTorch] [NIPS'16]

Network Morphism :star:
[arXiv:1603.01670] [ICML'16] --> FT MM

Net2Net: Accelerating Learning via Knowledge Transfer :star:
[arXiv:1511.05641] [ICLR'16] --> KT

~ 2015

A Hypercube-Based Indirect Encoding for Evolving Large-Scale Neural Networks
[Artificial Life journal'09] [code] --> EA TP

Useful Link

1. https://www.ml4aad.org/automl/literature-on-neural-architecture-search/