Skip to content
A black-box optimization benchmarking framework
Rust Python
Branch: master
Clone or download
Latest commit def94ae Dec 12, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Switch to GitHub Actions Nov 27, 2019
kurobako_core Bump version to v0.1.5 Dec 12, 2019
kurobako_problems Bump version to v0.1.3 Dec 12, 2019
kurobako_solvers
src
.gitignore Update .gitignore Nov 24, 2019
Cargo.lock Bump version to v0.1.3 Dec 12, 2019
Cargo.toml Bump version to v0.1.3 Dec 12, 2019
LICENSE Add LICENSE Mar 16, 2019
README.md

README.md

kurobako

kurobako Documentation Actions Status Coverage Status License: MIT

kurobako is a command-line tool to benchmark black-box optimization algorithms.

Features:

  • It has the capability to support wide range of optimization problems:
    • various search sapce:
      • Continuous numerical, discrete numerical and categorical
      • Uniform distribution and log uniform distribution
      • Conditional
    • Constrainted problems
    • Multi-objective problems
  • Generating a markdown report and PNG plots from benchmarking results
  • Easy to add user-defined optimization problems and solvers
  • Simulating a concurrent environment in which an optimization process is executed by multiple workers simultaneously
  • Reproducible

Installation

Precompiled binaries

Precompiled binaries for Linux are available in the releases page.

$ curl -L https://github.com/sile/kurobako/releases/download/${VERSION}/kurobako-${VERSION}.linux-amd64 -o kurobako
$ chmod +x kurobako
$ ./kurobako -h

Using Cargo

If you have already installed Cargo, you can install kurobako by executing the following command:

$ cargo install kurobako

Usage Example

// Define solver.
$ kurobako solver random | tee solver.json
{"random":{}}

// Define problem.
$ curl -OL http://ml4aad.org/wp-content/uploads/2019/01/fcnet_tabular_benchmarks.tar.gz
$ tar xf fcnet_tabular_benchmarks.tar.gz && cd fcnet_tabular_benchmarks/
$ kurobako problem hpobench fcnet_protein_structure_data.hdf5 | tee problem.json
{"hpobench":{"dataset":"fcnet_protein_structure_data.hdf5"}}

// Run benchmark.
$ kurobako studies --solvers $(cat solver.json) --problems $(cat problem.json) | kurobako run > result.json
(ALL) [00:00:01] [STUDIES     10/10 100%] [ETA  0s] done

// Report the benchmark result.
$ cat result.json | kurobako report
...abbrev...

// Plot the benchmark result.
$ cat result.json | kurobako plot curve
(PLOT) [00:00:01] [1/1 100%] [ETA  0s] done (dir="images/curve/")

Build-in Solvers and Problems

Solvers:

Problems:

Where does the name come from?

"kurobako" is a Japanese translation of "black box".

You can’t perform that action at this time.