Edit: The authors official implementation is now available on GitHub at github.com/NiccoloCavagnero/FreeREA.

Disclaimer This repo presents an implementation of FreeREA: Training-Free Evolution-based Architecture Search, by Cavagnero et al, 2022. This work is not associated in any way or form with the authors and only aims at reproducing the findings presented by the authors in the cited paper. The authors have been informed of this re-implementation.

Set-up

After having locally cloned this repo, the first step to use this code is installing the required dependencies. To set up dependancies, set up a virtual environment clonig the env.yml file.

$ conda create --name <env_name> --file requirements.txt

Once dependencies have been successfully installed, please go ahead and download in the main folder (that is, FreeREA) the archive folder, containing the actual NATS-Bench networks. archive is already in the .gitignore file of this repo.

To download NATS-Bench and create the archive folder simply run the following:

$ bash setup_nats.sh

Alternatively, one could download archive from here and then unzip the folder.

Please consider that downloading the search space only is more than sufficient as fully trained models are not needed, since the benchmark conveniently stores the model performance metrics. More than that, downloading the trained architectures (that is, the fully trained architectures with their weights) would download 200+ GB of architectures.

Run the experiments

To reproduce FreeREA experiments you simply need to launch experiments.sh. This file will create a grid of n x m experiments, with n=number of number_of_generations and m=number of datasets.

Note: All combinations will be executed in parallel!