[New solver] DAQP solver initial interface #2312

enzbus · 2023-12-24T07:52:24Z

Description

This is a first implementation of an interface to the DAQP active set (dense) QP solver, an open source solver published in 2023. This was requested in issue #2310.

Notes

This is a straightforward implementation of the Python interface described in the documentation. The main catch is that DAQP with its default parameters can't solve QPs whose cost matrix has zero eigenvalues (that includes LPs). It can do so if we change the eps_prox option; I implemented the following logic:

check if the cost matrix is positive definite; if so leave default eps_prox = 0 which gives best performance (both numerical and execution time)
if instead the cost matrix has null eigenvalues, set eps_prox=1e-5; this was obtained by benchmarking with a typical use-case -> https://gist.github.com/enzbus/3d0236c7ed93cff5f0ec3f8587bcd67e

Type of change

[X ] New feature (backwards compatible)
New feature (breaking API changes)
Bug fix
Other (Documentation, CI, ...)

Contribution checklist

[X ] Add our license to new files.
[ X] Check that your code adheres to our coding style.
Write unittests.
[X ] Run the unittests and check that they’re passing.
Run the benchmarks to make sure your change doesn’t introduce a regression.

CLAassistant · 2023-12-24T07:52:29Z

All committers have signed the CLA.

enzbus · 2023-12-27T10:25:36Z

@SteveDiamond @phschiele @PTNobel @rileyjmurray any comments? Also @bstellato should be happy (OSQP is both faster and more robust). Thanks and happy holidays.

PTNobel · 2023-12-27T10:34:28Z

Hey Enzo, I'm out of the country and won't be able to review this till after I'm back.

enzbus · 2023-12-27T12:42:48Z

For more context here's another little benchmark, on linearly constrained least squares (so there's not proximal step involved). OSQP clearly in the lead.

https://gist.github.com/enzbus/426bd635d3dd7df559855c934495c899

SteveDiamond

This looks great to me! Thanks, Enzo! Let's have a another review before we merge, but seems like everything is there.

doc/source/tutorial/advanced/index.rst

enzbus · 2023-12-29T12:01:18Z

Thanks! Fixed that, and DAQP's author approves this initial implementation (see darnstrom/daqp#31 ). Thanks and happy new year!

PTNobel

Overall, this seems like a good interface; thank you Enzo!

I am a bit nervous about adding a "sparse -> dense" step in the solver interface. It feels like using this solver is a big performance footgun, and at the least it should be called out in the docs. Thoughts?

PTNobel · 2024-01-09T07:11:53Z

cvxpy/reductions/solvers/defines.py

@@ -42,9 +42,10 @@
 from cvxpy.reductions.solvers.conic_solvers.xpress_conif import XPRESS as XPRESS_con
 from cvxpy.reductions.solvers.qp_solvers.copt_qpif import COPT as COPT_qp


Shouldn't this and the line below be moved down below the comment # QP interfaces too?

I don't remember but I think the import reorder was done by ruff, is there a logic I was missing?

I think the QP solver interfaces were supposed to be below the comment about # QP solvers and the conic programs above...

PTNobel · 2024-01-09T07:15:35Z

cvxpy/reductions/solvers/qp_solvers/daqp_qpif.py

+                DAQP.VAR_ID:
+                intf.DEFAULT_INTF.const_to_matrix(np.array(xstar))
+            }
+            # need to skip unused variable bounds constraints


What are these unused variable bounds constraints? Why are they being passed to the solver?

To my understanding variable bounds are mandatory, this point also the author was particular about and asked whether Cvxpy can't handle those natively (I told him there are people looking into those but it's not there yet).

Ah okay that makes sense.

enzbus · 2024-01-09T09:56:46Z

Overall, this seems like a good interface; thank you Enzo!

I am a bit nervous about adding a "sparse -> dense" step in the solver interface. It feels like using this solver is a big performance footgun, and at the least it should be called out in the docs. Thoughts?

Agreed, I've done only limited execution time benchmarking but it didn't look to me that the sparse-dense conversion was too costly, worth investigating though. I don't see an alternative, though, other than perhaps factoring it out to be re-used by other interfaces, I think only one other at this point, or writing a new canonicalizer with dense output which is much more work I guess.

Edit: let me double check on your comments and add that to the docs.

SteveDiamond · 2024-01-09T15:34:34Z

For what it's worth, there is a dense canonicalization backend now if you use canon_backend=NUMPY_CANON_BACKEND

I don't think it's necessary or desirable to couple the canonicalization backend to the solver, but it might be worth trying this interface with that backend or documenting that the dense backend is recommended.

Also, I will post some thoughts about propagating variable bounds in the discord

enzbus · 2024-01-09T17:15:42Z

For what it's worth, there is a dense canonicalization backend now if you use canon_backend=NUMPY_CANON_BACKEND

I don't think it's necessary or desirable to couple the canonicalization backend to the solver, but it might be worth trying this interface with that backend or documenting that the dense backend is recommended.

Also, I will post some thoughts about propagating variable bounds in the discord

Ok! I should have thought of that, will do some more interesting benchmarks then. Also, what's the policy for SolverStats.solve_time and SolverStats.setup_time ? We put there the time reported by the solver (but I'm not sure it's accurate, probably ignores the time spent in its own Python interface)? Would it be better to time it in the interface? In any case will get back to it ASAP.

SteveDiamond · 2024-01-09T19:16:49Z

SolverStats.solve_time

SolverStats.solve_time is always from the solver. Sometimes solvers also break out setup time, in which case you should record it in SolverStats.setup_time. If the solver doesn't break out setup time, no need to record anything.

PTNobel · 2024-01-10T06:08:47Z

For what it's worth, there is a dense canonicalization backend now if you use canon_backend=NUMPY_CANON_BACKEND

I don't think it's necessary or desirable to couple the canonicalization backend to the solver, but it might be worth trying this interface with that backend or documenting that the dense backend is recommended.

@SteveDiamond correct me if I'm wrong, but the NumPy backend will convert to sparse before passing it into the solver interface right? We should probably have a way to skip that conversion if the solver interface wants a dense object.

SteveDiamond · 2024-01-10T21:47:34Z

For what it's worth, there is a dense canonicalization backend now if you use canon_backend=NUMPY_CANON_BACKEND
I don't think it's necessary or desirable to couple the canonicalization backend to the solver, but it might be worth trying this interface with that backend or documenting that the dense backend is recommended.

@SteveDiamond correct me if I'm wrong, but the NumPy backend will convert to sparse before passing it into the solver interface right? We should probably have a way to skip that conversion if the solver interface wants a dense object.

That could be right, I'm not sure though. @phschiele @Transurgeon how does the NumPy backend work in this regard?

phschiele · 2024-01-10T22:32:34Z

For what it's worth, there is a dense canonicalization backend now if you use canon_backend=NUMPY_CANON_BACKEND
I don't think it's necessary or desirable to couple the canonicalization backend to the solver, but it might be worth trying this interface with that backend or documenting that the dense backend is recommended.

@SteveDiamond correct me if I'm wrong, but the NumPy backend will convert to sparse before passing it into the solver interface right? We should probably have a way to skip that conversion if the solver interface wants a dense object.

That could be right, I'm not sure though. @phschiele @Transurgeon how does the NumPy backend work in this regard?

That's right, the dense backend has the same (sparse) output format as the other backends. We'd need to have a flag for the output and also update a few transformations that happen after the backend (or at least, verify that all overloaded operators work for sparse and dense matrices).

enzbus · 2024-01-12T09:19:50Z

Do you guys want to look into amending the numpy canonicalizer to produce a dense problem matrix? It would be cool but out of scope for this solver interface I guess, can always be added later. I haven't had time to benchmark it better yet but I was under the impression the sparse-dense conversion wasn't too costly. As an aside, the solver author mentioned he was interested in cvxpygen, I guess having a dense canonicalizer in CVXPY would be helpful for that side project too...

SteveDiamond · 2024-01-12T15:40:56Z

We can look into that later. This is good to go.

SteveDiamond · 2024-01-12T16:07:49Z

I did some work on propagating variable upper and lower bounds to the solver:
#2321

If the general design seems good, it will be easy to integrate with the DAQP interface.

enzbus · 2024-02-07T05:50:21Z

Hello @Transurgeon, looks like #2309 overwrote the changes to doc/advanced/solvers/index.rst introduced by this PR (adding DAQP). Should be fixable by a copy-paste but maybe you should have a look. Cc @SteveDiamond

enzbus · 2024-02-12T05:53:02Z

Thank you @Transurgeon @SteveDiamond ! PS I encountered the same issue recently (splitting a git-tracked file into many ones breaks automatic merge) and with some digging I found the solution, if you use this script it keeps the correct git history and allows for automatic merge.

enzbus added 3 commits December 24, 2023 10:22

initial daqp intf

12b8291

initial daqp intf passes pytest

ef70b24

added deps and docs, should be ok for first draft

3990155

enzbus added 4 commits December 24, 2023 11:57

ruff check --fix

ffa5d3d

trying eps_prox conditional on det(cost_matrix)

fd32562

better way to check PD matrix

6b97525

changed eps_prox default after some benchmarking

5defdda

SteveDiamond approved these changes Dec 27, 2023

View reviewed changes

doc/source/tutorial/advanced/index.rst Outdated Show resolved Hide resolved

enzbus mentioned this pull request Dec 27, 2023

Cvxpy interface help/suggestions darnstrom/daqp#31

Closed

Capitalize CVXPY

24b6399

PTNobel reviewed Jan 9, 2024

View reviewed changes

SteveDiamond merged commit 797bc9a into cvxpy:master Jan 12, 2024
44 checks passed

SteveDiamond mentioned this pull request Jan 12, 2024

Propagate variable bounds to the solver #2321

Merged

9 tasks

phschiele changed the title ~~DAQP solver initial interface~~ [New solver] DAQP solver initial interface Jan 26, 2024

phschiele mentioned this pull request Jan 26, 2024

MindOpt solver backend #2296

Open

Transurgeon mentioned this pull request Feb 11, 2024

Daqp solver info and other doc updates #2347

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[New solver] DAQP solver initial interface #2312

[New solver] DAQP solver initial interface #2312

enzbus commented Dec 24, 2023 •

edited

CLAassistant commented Dec 24, 2023 •

edited

enzbus commented Dec 27, 2023

PTNobel commented Dec 27, 2023

enzbus commented Dec 27, 2023

SteveDiamond left a comment

enzbus commented Dec 29, 2023

PTNobel left a comment

PTNobel Jan 9, 2024

enzbus Jan 9, 2024

PTNobel Jan 10, 2024

PTNobel Jan 9, 2024

enzbus Jan 9, 2024

PTNobel Jan 10, 2024

enzbus commented Jan 9, 2024 •

edited

SteveDiamond commented Jan 9, 2024

enzbus commented Jan 9, 2024

SteveDiamond commented Jan 9, 2024

PTNobel commented Jan 10, 2024

SteveDiamond commented Jan 10, 2024

phschiele commented Jan 10, 2024

enzbus commented Jan 12, 2024

SteveDiamond commented Jan 12, 2024

SteveDiamond commented Jan 12, 2024

enzbus commented Feb 7, 2024

enzbus commented Feb 12, 2024

		@@ -42,9 +42,10 @@
		from cvxpy.reductions.solvers.conic_solvers.xpress_conif import XPRESS as XPRESS_con
		from cvxpy.reductions.solvers.qp_solvers.copt_qpif import COPT as COPT_qp

[New solver] DAQP solver initial interface #2312

[New solver] DAQP solver initial interface #2312

Conversation

enzbus commented Dec 24, 2023 • edited

Description

Notes

Type of change

Contribution checklist

CLAassistant commented Dec 24, 2023 • edited

enzbus commented Dec 27, 2023

PTNobel commented Dec 27, 2023

enzbus commented Dec 27, 2023

SteveDiamond left a comment

Choose a reason for hiding this comment

enzbus commented Dec 29, 2023

PTNobel left a comment

Choose a reason for hiding this comment

PTNobel Jan 9, 2024

Choose a reason for hiding this comment

enzbus Jan 9, 2024

Choose a reason for hiding this comment

PTNobel Jan 10, 2024

Choose a reason for hiding this comment

PTNobel Jan 9, 2024

Choose a reason for hiding this comment

enzbus Jan 9, 2024

Choose a reason for hiding this comment

PTNobel Jan 10, 2024

Choose a reason for hiding this comment

enzbus commented Jan 9, 2024 • edited

SteveDiamond commented Jan 9, 2024

enzbus commented Jan 9, 2024

SteveDiamond commented Jan 9, 2024

PTNobel commented Jan 10, 2024

SteveDiamond commented Jan 10, 2024

phschiele commented Jan 10, 2024

enzbus commented Jan 12, 2024

SteveDiamond commented Jan 12, 2024

SteveDiamond commented Jan 12, 2024

enzbus commented Feb 7, 2024

enzbus commented Feb 12, 2024

enzbus commented Dec 24, 2023 •

edited

CLAassistant commented Dec 24, 2023 •

edited

enzbus commented Jan 9, 2024 •

edited