v2.2.1

This patch release updates to qpsolvers v4.3.1 which fixes a bug in the Gurobi interface. Thanks @563925743 👍

Changed

• Update qpsolvers to v4.3.1 to fix the Gurobi interface

v2.2.0

The benchmark is now distributed on conda-forge:

conda install qpbenchmark

This release also adds a main-function hook to make it easier to run test sets directly from their repository as python my_test_set.py.

Added

• Distribute on conda-forge
• Test set path argument to the main function of the command-line interface

Changed

• Update environment file to install from conda-forge

v2.1.1

This patch release fixes a bug in the handling of known solver issues.

Changed

• Clarabel: set tol_gap_abs settings in set_eps_abs
• Clarabel: set tol_gap_rel settings in set_eps_rel
• Log a warning message when skipping known solver issue
• PIQP: set eps_duality_gap_abs settings in set_eps_abs
• PIQP: set eps_duality_gap_rel settings in set_eps_rel
• ProxQP: set eps_duality_gap_abs settings in set_eps_abs
• ProxQP: set eps_duality_gap_rel settings in set_eps_rel

Fixed

• Handling of known solver issues

v2.1.0

This release removes the cost error from benchmark metrics, and completes the spinning off of test sets to their own repositories. It also brings a few quality-of-use improvements.

Added

• Expose Problem from top-level module
• Link in report header to go to results tables directly
• Number of problems in report header
• Utility function to load a problem from file

Changed

• Breaking: Remove the cost error from benchmark metrics
• Include Python 3.8 in supported versions

Removed

• Move the GitHub Free-for-all test set to its own repository
• Move Maros-Meszaros test set to its own repository

v2.0.0

This release renames the benchmark to qpbenchmark and adds the QPALM solver.

Added

• New solver: QPALM

Changed

• Breaking: Rename the project "qpbenchmark"
• Don't install solvers by default from PyPI
• Move solver issue and timeout listings to test-set themselves

Removed

• Module-wide skip solver issue/timeout functions

v1.2.0

This release adds a mid_accuracy set of settings to the QP solvers benchmark.

Added

• Citation file to refer to the benchmark in scientific works
• Medium accuracy settings
• Note in the readme on CPU throttling

v1.1.0

This release adds two solvers to the benchmark: HPIPM and PIQP. It also invites everyone to submit results on their machines to the Results discussions.

Added

• Check consistency after loading results
• More CPU information in reports
• New solver: HPIPM
• New solver: PIQP

Changed

• Don't hard-wrap report lines, as it doesn't render well in Discussions
• Improve reporting of shifted geometric mean errors
• Make cpuinfo a proper dependency
• Refactor results class to allow finer check_results sessions
• Update to qpsolvers v4.0.0

Fixed

• Correct None values to False in found column
• Make sure found column has only boolean values

v1.0.0

This is the first release of the QP solvers benchmark, which ships a qpsolvers_benchmark command-line tool contributed by @ZAKIAkram.

The goal of this benchmark is to help users compare and select QP solvers. Its methodology is open to discussions. The benchmark ships standard and community test sets, as well as a qpsolvers_benchmark command-line tool to run any test set directly on your machine. For instance:

qpsolvers_benchmark maros_meszaros/maros_meszaros.py run

The outcome from running a test set is a standardized report evaluating all metrics of the benchmark across all available QP solvers. This repository also distributes results from running the benchmark on all test sets using the same computer.

New test sets are welcome. The benchmark is designed so that each test-set directory is standalone, so that the qpsolvers_benchmark command can be run on test sets from other repositories. Feel free to create ones that better represent the kind of problems you are working on.

Thanks to @aescande, @ottapav and @ZAKIAkram for contributing to this release 👍

Added

• Allow non-lowercase solver names in the command line (thanks to @ottapav)
• Command-line tool and standalone test sets (thanks to @ZAKIAkram)

Changed

• Plot: trim solutions that don't fulfill tolerance requirements
• Rename hist command to plot
• Update to qpsolvers v3.4.0

v0.1.0-beta

This is the last pre-release of qpsolvers_benchmark, a benchmark for quadratic programming (QP) solvers available in Python. The main upgrade since alpha is that the benchmark now checks both residuals and the duality gap, i.e., full optimality conditions.

The goal of this pre-release is to request feedback from QP solver developers before the initial release. The initial release will in no way be the end: it will be advertised around, but the benchmark is designed to be iteratively refined by the community in the long run.

You can already help at this stage! Here are some suggestions:

• Take a look at the README and current reports produced for the Maros-Meszaros and GitHub free-for-all test sets.
• Look at the benchmark methodology and join the discussions.
• Submit a new problem, e.g. one that reflects the applications you are working on.

Added

• Check dual residual
• Check duality gap
• Document all benchmark functions
• Main script: new hist plot command
• ProblemNotFound exception
• Results by settings in reports
• Write benchmark version in reports

Changed

• Benchmark script takes test set as first argument
• Maros-Meszaros: empty equality constraints are now set to None
• ProxQP: re-run benchmark with ProxQP 0.3.2
• Refactor Report class and run function
• Report encoding is now UTF-8
• Switch to qpsolvers v2.7
• Test set descriptions are now mandatory

Fixed

• Conform to linter standards
• Sparse matrix conversion

v0.1.0-alpha

This is a pre-release of qpsolvers_benchmark, a benchmark for quadratic programming (QP) solvers available in Python. This benchmark is still a work in progress. The goal of this pre-release is to show the current status and gather feedback. In particular, feel free to:

• Take a look at the README and current reports produced for the Maros-Meszaros and GitHub free-for-all test sets.
• Look at the benchmark methodology and join the discussions.
• Submit a new problem, e.g. one that reflects the applications you are working on.

Towards a tool to help us compare and select QP solvers!