BlackBoxOptimizationBenchmarking
Switch branches/tags
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data
scripts
src
test
.codecov.yml
.gitignore
.travis.yml
LICENSE.md
README.md
REQUIRE
appveyor.yml

README.md

BlackBoxOptimizationBenchmarking.jl

Build Status Coverage Status

A Julia implementation of the Black-Box-Optimization-Benchmarking (BBOB) functions.

Benchmark results

The average sucess rate (meaning the optimizer reached the minimum + 1e-6):

benchmark

Python CMA-ES is the best overall (mainly in higher dimension), while some of BlackBoxOptim.jl optimizers are better for large numbers of function evaluation.

The average sucess rate across the dimension of the function:

benchmark

benchmark

benchmark

The total relative run time of each optimizer

benchmark

Note that the Python algorithm are called from Julia, which might cause some overhead.

Functions

Functions can be accessed as BlackBoxOptimizationBenchmarking.F1, which returns a BBOBFunction with fields f containing the function itself, f_opt its minimal value, and x_opt its minimizer, i.e. f(x_opt) = f_opt.

Functions can be listed using enumerate:

julia> enumerate(BBOBFunction)
20-element Array{BlackBoxOptimizationBenchmarking.BBOBFunction,1}:
 Sphere                                           
 Ellipsoidal Function                             
 Discus Function                                  
 Bent Cigar Function                              
 Sharp Ridge Function                             
 Different Powers Function                        
 Rastrigin Function                               
 Weierstrass Function                             
 Schaffers F7 Function                            
 Schaffers F7 Function, moderately ill-conditioned
 Composite Griewank-Rosenbrock Function F8F2      
 Ellipsoidal                                      
 Schwefel Function                                
 Rastrigin                                        
 Buche-Rastrigin                                  
 Linear Slope                                     
 Attractive Sector                                
 Step Ellipsoidal Function                        
 Rosenbrock Function, original                    
 Rosenbrock Function, rotated

A benchmark for a single optimizer and function can be run with:

benchmark(optimizer::Any, f::BBOBFunction, run_lengths, Ntrials, dimensions, Δf)

Or for a collection of optimizers with:

benchmark(optimizers::Vector{T}, funcs, run_lengths, Ntrials, dimensions, Δf)

See test/runtests.jl

The optimizer must implement the methods optimize, minimum and minimizer, see

scripts/optimizers_interface.jl

Reference:

http://coco.lri.fr/downloads/download15.01/bbobdocfunctions.pdf