IOHprofiler: IOHexperimenter

This is the benchmarking platform for Iterative Optimization Heuristics (IOHs).

• Documentation: https://arxiv.org/abs/1810.05281
• Wiki page: https://iohprofiler.github.io
• General Contact: iohprofiler@liacs.leidenuniv.nl
• Mailing List: https://lists.leidenuniv.nl/mailman/listinfo/iohprofiler

IOHexperimenter provides:

• A framework for straightforward benchmarking of any iterative optimization heuristic
• A suite consisting of 23 pre-made Pseudo-Boolean benchmarking function, with easily accessible methods for adding custom functions and suites
• Logging methods to effortlesly store benchmarking data in a format compatible with IOHanalyzer, with future support for additional data logging options
• (Soon to come:) A framework which significantly simplifies algorithm design

IOHexperimenter is built on:

• C++

IOHexperimenter is available for:

• C++ on the current GitHub branch; Wiki Page.
• R on this GitHub branch or as a CRAN package; Wiki Page.
• Python on this GitHub branch or as a pip package; Wiki Page.

Using IOHexperimenter

Running Experiments

The IOHexperimenter has been built on C++.

Using IOHexperimenter in C++

Cmake is used to build this project, please make sure you have it installed.

If you are using the tool for the first time, please download or clone this branch, and run cmake .; make install at the directory where the project locates.

• If you want to set up the install directory, please run cmake -DCMAKE_INSTALL_PREFIX=your/path . before installation.
• three exectuable files will be generated in build/Cpp for test.

After installation, you can compile your project as follow (with linking IOH library):

g++ $CMPL_FLAGS -o IOHprofiler_run_experiment IOHprofiler_run_experiment.cpp -lIOH

For more details of how to use the C++ version, please visit this page.

Using IOHexperimenter in R

To use the IOHexperimenter within R, please visit the R branch of this repository.

Creating test problems

Benchmarking problems in IOHexperimenter are easy to create yourself. We provide support for any input type and any number of real-valued objectives. For a more detailed guidline of how to define a benchmarking problem within IOHexperimenter, please visit this page.

Configuring test suites

Suites are collections of benchmarking problems. By including problems into a suite, it is easier for users to maintain their experiments. If you create a set of similar problems, it is recommended to create a suite to collect them together, which can be done effortlesly within the IOHexperimenter. For detailed steps of creating and using suites, please visit this page.

Contact

If you have any questions, comments or suggestions, please don't hesitate contacting us IOHprofiler@liacs.leidenuniv.nl!

Our team

• Furong Ye, Leiden Institute of Advanced Computer Science,
• Diederick Vermetten, Leiden Institute of Advanced Computer Science,
• Hao Wang, Leiden Institute of Advanced Computer Science,
• Carola Doerr, CNRS and Sorbonne University,
• Thomas Bäck, Leiden Institute of Advanced Computer Science,

When using IOHprofiler and parts thereof, please kindly cite this work as

Carola Doerr, Hao Wang, Furong Ye, Sander van Rijn, Thomas Bäck: IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics, arXiv e-prints:1810.05281, 2018.

@ARTICLE{IOHprofiler,
  author = {Carola Doerr and Hao Wang and Furong Ye and Sander van Rijn and Thomas B{\"a}ck},
  title = {{IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics}},
  journal = {arXiv e-prints:1810.05281},
  archivePrefix = "arXiv",
  eprint = {1810.05281},
  year = 2018,
  month = oct,
  keywords = {Computer Science - Neural and Evolutionary Computing},
  url = {https://arxiv.org/abs/1810.05281}
}