Atom for the Von Neumann Entropy #1789

aryamanjeendgar · 2022-05-28T06:50:15Z

In this PR I add support for the Von Neumann Entropy as an Atom. The vn_entr class itself lies in cvxpy/atoms, and its respective Canonicalization method lies in cvxpy/reductions/dcp2cone/atom_canonicalizers, finally, the tests reside in cvxpy/tests/test_vn_entr.py. I chose to omit the _grad method for this PR as per @SteveDiamond's advice.

The implementation of the function itself is wrapped in the numeric procedure in vn_entr.py--- $S(\rho)=-\sum_x \lambda_x\log\lambda_x$ --- this form can be found in equation (11.40) of Neilsen & Chuang's QIC text.

The Canonicalization procedure is based off of Proposition-4 of THIS paper.

get vn_entr canonicalization *probably* working

keithbriggs · 2022-05-28T07:33:16Z

"it's respective Canonicalization method" should be "its respective Canonicalization method".

rileyjmurray · 2022-05-28T22:32:32Z

The failing tests seem to be for versions of Python where SCS 2.1.2 is installed, rather than SCS 3. I ran the tests on my laptop with SCS 3 and they passed, but the return code was "inaccurate" even after I increased max_iters to 500,000. So I think we'll need MOSEK for these tests to pass.

rileyjmurray

@aryamanJgl here are some things to be aware of. I'm investigating other stuff and hopefully will provide more comments soon.

rileyjmurray · 2022-05-28T22:50:33Z

cvxpy/tests/test_vn_entr.py

+        expect_N = np.array([[0.36787973, 0.00099987],
+                             [0.00099987, 0.36787973]])
+        eps = 1e2
+        cons1 = N >> 0


The constraint that N >> 0 is redundant, since you already declared PSD = True for N. Same comment goes for other tests.

rileyjmurray · 2022-05-28T22:52:02Z

cvxpy/tests/test_vn_entr.py

+        N = U.T@N@U
+        objective = cp.Maximize(vn_entr(N))
+        obj_pair = (objective, 1.1033501770078251)
+        con_pairs = [
+            (cons1, None),
+            (cons2, None),
+            (cons3, None)
+        ]
+        var_pairs = [
+            (N, expect_N)


Here you're reassigning N. That should cause tests to fail since expect_N is supposed to be with respect to the original variable.

rileyjmurray · 2022-05-28T22:57:02Z

cvxpy/tests/test_vn_entr.py

+        eps = 1e3
+        cons1 = N >> 0
+        cons2 = N << eps*np.eye(N.shape[0])
+        cons3 = N[2][0] == 1e-2


This is inconsistent with expect_N.

rileyjmurray · 2022-05-28T23:14:52Z

@aryamanJgl here is a tiny example of a test that's failing:

N = cp.Variable(shape=(2, 2), PSD=True)
constraints = [cp.trace(N) <= 1,
               N[0, 1] == -0.1,
               ]
objective = cp.Maximize(cp.vn_entr(N))
prob = cp.Problem(objective, constraints)
prob.solve(verbose=True)  # infeasible

The solver (SCS or MOSEK) says the problem is infeasible, but N = [[0.5, -0.1],[-0.1, 0.5]] is feasible (its eigenvalues are 0.4 and 0.6).

Here's another option:

N = cp.Variable(shape=(2, 2), PSD=True)
N_val = np.array([[0.6, -0.1], [-0.1, 0.4]])
constraints = [N == N_val]
objective = cp.Maximize(cp.vn_entr(N))
prob = cp.Problem(objective, constraints)
prob.solve(verbose=True)  # again, infeasible.

In fact, you can make that example even simpler by setting N_val = np.array([[0.6, 0.0], [0.0, 0.4]]).

Progress toward working vn_entr canonicalization

CLAassistant · 2022-05-29T06:57:04Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ rileyjmurray
✅ aryamanJgl
❌ Riley Murray

Riley Murray seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

keithbriggs · 2022-05-29T07:06:06Z

Isn't this a another case where a long name is needed? vn_entr is completely cryptic. I would go for von_neumann_entropy.

rileyjmurray · 2022-05-29T15:07:22Z

Isn't this a another case where a long name is needed? vn_entr is completely cryptic. I would go for von_neumann_entropy.

That's a good point. Although I'd vote for von_neumann_entr, since basic (Shannon) entropy is just entr. In the near future we'll have a bunch of other atoms where we'll want to continue to use existing abbreviations, like quantum_rel_entr and operator_rel_entr.

@aryamanJgl if you don't object, how about you change it to von_neumann_entr?

aryamanjeendgar · 2022-06-01T05:11:08Z

@aryamanJgl if you don't object, how about you change it to von_neumann_entr?

Yeah, that makes a lot of sense! I will change it in the next commit.

rileyjmurray

Mostly looks good! The main thing that stands out is the need for documentation. The docstring for von_neumann_entr is missing, and the web docs need to be updated to mention this new atom. I also left minor comments on the tests.

cvxpy/atoms/von_neumann_entr.py

cvxpy/tests/test_von_neumann_entr.py

rileyjmurray

Looks good to me!

rileyjmurray · 2022-06-21T14:38:06Z

@SteveDiamond can you review this PR? We need to project maintainers to approve before merging, since this affects the public API.

cvxpy/atoms/von_neumann_entr.py

cvxpy/reductions/dcp2cone/atom_canonicalizers/von_neumann_entr_canon.py

SteveDiamond

Approved subject to my comments. Also shouldn't we add it to the table of atoms?

rileyjmurray · 2022-06-23T13:40:58Z

@aryamanJgl please address my response to Steven's comment. Also, there are two things to do for the docs.

von_neumann_entr isn't an elementwise atom. It should be documented here, not at doc/source/api_reference/cvxpy.atoms.elementwise.rst.
Add von_neumann_entr to the table of "Scalar Functions" here.

* origin/master: Add autoflake pre-commit; remove unused variable; remove unused pass (#1756) [New atom] dotsort (#1803) Change exception type when unsupported SCIP 4.x is installed (#1807) Atom for the Von Neumann Entropy (#1789) Exclude examples from deployment (#1804) make upload jobs pass (#1802) [CI] Replace manylinux_2_24 image (#1792)

aryamanjeendgar and others added 5 commits April 30, 2022 08:12

Skeleton for vn_entr

53807b0

get vn_entr canonicalization *probably* working

273602b

Merge pull request #1 from rileyjmurray/local_vn_entr

c266239

get vn_entr canonicalization *probably* working

Added tests

6f3999e

cosmetic changes

845d161

modify test settings for vn_entr

33a1793

rileyjmurray reviewed May 28, 2022

View reviewed changes

rileyjmurray and others added 2 commits May 28, 2022 17:56

correct vn_entr canonicalization

6598322

Merge pull request #2 from rileyjmurray/vn_entr

8bf4f1a

Progress toward working vn_entr canonicalization

aryamanjeendgar and others added 3 commits June 5, 2022 10:16

Changed test cases + name

676d300

Merge branch 'cvxpy:master' into vn_entr

7cc5d10

new test case + name fix

01e9f6f

rileyjmurray requested changes Jun 7, 2022

View reviewed changes

cvxpy/atoms/von_neumann_entr.py Outdated Show resolved Hide resolved

cvxpy/tests/test_von_neumann_entr.py Outdated Show resolved Hide resolved

cvxpy/tests/test_von_neumann_entr.py Outdated Show resolved Hide resolved

aryamanjeendgar added 2 commits June 8, 2022 09:31

Final changes

1ef2347

Linter correction

d81bd06

rileyjmurray approved these changes Jun 21, 2022

View reviewed changes

SteveDiamond reviewed Jun 21, 2022

View reviewed changes

cvxpy/atoms/von_neumann_entr.py Show resolved Hide resolved

SteveDiamond reviewed Jun 21, 2022

View reviewed changes

cvxpy/reductions/dcp2cone/atom_canonicalizers/von_neumann_entr_canon.py Show resolved Hide resolved

SteveDiamond approved these changes Jun 21, 2022

View reviewed changes

Doc changes, canon. optim

b9cf994

rileyjmurray merged commit f788cce into cvxpy:master Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Atom for the Von Neumann Entropy #1789

Atom for the Von Neumann Entropy #1789

aryamanjeendgar commented May 28, 2022 •

edited

keithbriggs commented May 28, 2022

rileyjmurray commented May 28, 2022

rileyjmurray left a comment

rileyjmurray May 28, 2022

rileyjmurray May 28, 2022

rileyjmurray May 28, 2022

rileyjmurray commented May 28, 2022 •

edited

CLAassistant commented May 29, 2022 •

edited

keithbriggs commented May 29, 2022

rileyjmurray commented May 29, 2022

aryamanjeendgar commented Jun 1, 2022

rileyjmurray left a comment

rileyjmurray left a comment

rileyjmurray commented Jun 21, 2022

SteveDiamond left a comment

rileyjmurray commented Jun 23, 2022

Atom for the Von Neumann Entropy #1789

Atom for the Von Neumann Entropy #1789

Conversation

aryamanjeendgar commented May 28, 2022 • edited

keithbriggs commented May 28, 2022

rileyjmurray commented May 28, 2022

rileyjmurray left a comment

Choose a reason for hiding this comment

rileyjmurray May 28, 2022

Choose a reason for hiding this comment

rileyjmurray May 28, 2022

Choose a reason for hiding this comment

rileyjmurray May 28, 2022

Choose a reason for hiding this comment

rileyjmurray commented May 28, 2022 • edited

CLAassistant commented May 29, 2022 • edited

keithbriggs commented May 29, 2022

rileyjmurray commented May 29, 2022

aryamanjeendgar commented Jun 1, 2022

rileyjmurray left a comment

Choose a reason for hiding this comment

rileyjmurray left a comment

Choose a reason for hiding this comment

rileyjmurray commented Jun 21, 2022

SteveDiamond left a comment

Choose a reason for hiding this comment

rileyjmurray commented Jun 23, 2022

aryamanjeendgar commented May 28, 2022 •

edited

rileyjmurray commented May 28, 2022 •

edited

CLAassistant commented May 29, 2022 •

edited