This repository is meant to facilitate running experiments with spearmint.
Python Shell
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


This repository is meant to facilitate running experiments with spearmint.


spex requires a working spearmint repo, where the root directory of the cloned repo (containing bin, examples, spearmint) has to be appended to the PYTHONPATH environment variable, and jug which is used to process batch experiments and can simply be installed using pip, i.e.,

pip install jug


Once spearmint and jug are properly installed, all you need to do is write a configuration yaml file that corresponds to the experiment you wish to run. A sample configuration file can be found at spex/experiments/branin/branin.yaml.

Configuring and running an experiment can be accomplished using the following commands from the spex home directory:

python experiments/branin

bash experiments/branin -n 8

where in the second line we have set 8 processes to run (in the background).

Monitoring progress

You can monitor the progress of all tasks in an experiment by running:

jug status experiments/branin/

Visualizing results

You can plot the average performance with confidence bands (± 3 standard errors) with the following command:

python experiments/branin

Naturally, this can only be run when the jug tasks are done executing. If you prefer to plot or manipulate the results yourself, you can also run:

python experiments/branin

which will pickle the results in results/branin.pkl.

Submitting jobs to TORQUE

Alternatively, the script will give you the exact command to run on a cluster to submit a job to TORQUE. It will look something like:

qsub -l walltime=24:00:00,mem=4gb -t 1-10 path/to/execute.pbs

Cleaning up

The experiments can be fully cleaned up (leaving the config file intact) by running:

bash experiments/branin

or all of them by running bash experiments/*.