NASLib is a modular and flexible framework created with the aim of providing a common codebase to the community to facilitate research on Neural Architecture Search (NAS). It offers high-level abstractions for designing and reusing search spaces, interfaces to benchmarks and evaluation pipelines, enabling the implementation and extension of state-of-the-art NAS methods with a few lines of code. The modularized nature of NASLib allows researchers to easily innovate on individual components (e.g., define a new search space while reusing an optimizer and evaluation pipeline, or propose a new optimizer with existing search spaces). It is designed to be modular, extensible and easy to use.

NASLib was developed by the AutoML Freiburg group and with the help of the NAS community, we are constantly adding new search spaces, optimizers and benchmarks to the library. Please reach out to zelaa@cs.uni-freiburg.de for any questions or potential collaborations.

Setup | Usage | Docs | Contributing | Cite

Setup

While installing the repository, creating a new conda environment is recomended. Install PyTorch GPU/CPU for your setup.

conda create -n mvenv python=3.7
conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia

Run setup.py file with the following command, which will install all the packages listed in requirements.txt

pip install --upgrade pip setuptools wheel
pip install -e .

To validate the setup, you can run tests:

cd tests
coverage run -m unittest discover -v

The test coverage can be seen with coverage report.

Usage

To get started, check out demo.py.

search_space = SimpleCellSearchSpace()

optimizer = DARTSOptimizer(config)
optimizer.adapt_search_space(search_space)

trainer = Trainer(optimizer, config)
trainer.search()        # Search for an architecture
trainer.evaluate()      # Evaluate the best architecture

For more examples see naslib tutorial, intro to search spaces and intro to predictors.

Contributing

We welcome contributions to the library along with any potential issues or suggestions. Please create a pull request to the Develop branch.

Cite

If you use this code in your own work, please use the following bibtex entries:

@misc{naslib-2020, 
  title={NASLib: A Modular and Flexible Neural Architecture Search Library}, 
  author={Ruchte, Michael and Zela, Arber and Siems, Julien and Grabocka, Josif and Hutter, Frank}, 
  year={2020}, publisher={GitHub}, 
  howpublished={\url{https://github.com/automl/NASLib}} }

NASLib has been used to run an extensive comparison of 31 performance predictors (figure above). See the separate readme: predictors.md and our paper: How Powerful are Performance Predictors in Neural Architecture Search?

@article{white2021powerful,
  title={How Powerful are Performance Predictors in Neural Architecture Search?},
  author={White, Colin and Zela, Arber and Ru, Binxin and Liu, Yang and Hutter, Frank},
  journal={arXiv preprint arXiv:2104.01177},
  year={2021}
}