Fix how trainable args are counted for gradients in `GradientDescentOptimizer` and `NesterovMomentumOptimizer` #1495

antalszava · 2021-08-04T15:35:05Z

Context
The GradientDescentOptimizer and NesterovMomentumOptimizer optimizers seem to have an issue with a cost function that takes one trainable and one non-trainable argument:

import pennylane as qml
from pennylane import numpy as np

dev = qml.device('default.qubit', wires=2)

@qml.qnode(dev, diff_method='parameter-shift')
def circuit(x):
    qml.RX(x, wires=0)
    return qml.expval(qml.PauliZ(0))

def cost(x, target):
    return circuit(x)

opt = qml.GradientDescentOptimizer(stepsize=10)
x = np.tensor(0.0, requires_grad=True)
ev = np.tensor(0.7781, requires_grad=False)

x, cost_val = opt.step_and_cost(cost, x, ev)

Raises

~/xanadu/pennylane/pennylane/optimize/gradient_descent.py in apply_grad(self, grad, args)
    153             if getattr(arg, "requires_grad", True):
    154                 x_flat = _flatten(arg)
--> 155                 grad_flat = _flatten(grad[trained_index])
    156                 trained_index += 1
    157 

IndexError: invalid index to scalar variable.

Although the cost function does not depend on target, the error is still raised. If we remove that argument, the gradient is computed well:

import pennylane as qml
from pennylane import numpy as np

dev = qml.device('default.qubit', wires=2)

@qml.qnode(dev, diff_method='parameter-shift')
def circuit(x):
    qml.RX(x, wires=0)
    return qml.expval(qml.PauliZ(0))


def cost(x):
    return circuit(x)

opt = qml.GradientDescentOptimizer(stepsize=10)
x = np.tensor(0.0, requires_grad=True)
ev = np.tensor(0.7781, requires_grad=False)

x, cost_val = opt.step_and_cost(cost, x)

For both optimizers, there is a part of the logic, where the output gradient is adjusted:

        if len(args) == 1:
            grad = (grad,)

Changes made

Changes how the output gradient is adjusted such that only the trainable arguments are considered.

Related issues

PennyLaneAI/qml#309

codecov · 2021-08-04T15:46:36Z

Codecov Report

Merging #1495 (ad0eb38) into master (c0cdff2) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #1495   +/-   ##
=======================================
  Coverage   98.36%   98.36%           
=======================================
  Files         183      183           
  Lines       13163    13171    +8     
=======================================
+ Hits        12948    12956    +8     
  Misses        215      215

Impacted Files	Coverage Δ
pennylane/optimize/gradient_descent.py	`100.00% <100.00%> (ø)`
pennylane/optimize/nesterov_momentum.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c0cdff2...ad0eb38. Read the comment docs.

josh146 · 2021-08-04T17:19:19Z

pennylane/optimize/gradient_descent.py

+        num_trainable_args = 0
+        for arg in args:
+            if getattr(arg, "requires_grad", True):
+                num_trainable_args += 1


@antalszava you might be able to use the new

num_trainable_args = len(qml.math.get_trainable_indices(args))

functionality I just merged into the math module!

(optional, though)

This didn't seem to have worked for some tests 🤔

No worries! We'll need to remember to update this in the future, if requires_grad=False by default

The issue seems to be that if the argument is not a PennyLane NumPy tensor, then it'll be considered to belong to the "numpy" interface (instead of the "autgrad" interface), and all "numpy" arguments automatically evaluate as False when calling utils.requires_grad(arg). So, whenever an arg is simply a float get_trainable_indices would consider it non-trainiable, while that seems to not be what's wanted here.

josh146

Really nice catch @antalszava 💯

I've left some minor comments - only important one is to add one additional test case.

tests/test_optimize.py

josh146 · 2021-08-04T17:25:49Z

pennylane/optimize/gradient_descent.py

+        num_trainable_args = 0
+        for arg in args:
+            if getattr(arg, "requires_grad", True):
+                num_trainable_args += 1


(optional, though)

Co-authored-by: Josh Izaac <josh146@gmail.com>

…AI/pennylane into ch7853-fix_grad_desc_mult_args

This reverts commit 386eb77.

…se; the previous version left a state because the object lived on

tests/test_optimize.py

antalszava · 2021-08-04T20:16:04Z

tests/test_optimize.py

@@ -747,16 +747,41 @@ def reset(opt):
        opt.reset()


+@pytest.fixture


Changed to this structure, as it seems that the previous code left state in between test cases. This made 2 tests fail:

Before: https://github.com/PennyLaneAI/pennylane/runs/3244508688

josh146

Thanks @antalszava 👨‍🍳

josh146 · 2021-08-05T05:07:31Z

pennylane/optimize/gradient_descent.py

+        num_trainable_args = 0
+        for arg in args:
+            if getattr(arg, "requires_grad", True):
+                num_trainable_args += 1


No worries! We'll need to remember to update this in the future, if requires_grad=False by default

thisac

Nice fix @antalszava!

thisac · 2021-08-05T22:20:03Z

tests/test_optimize.py

@@ -747,16 +747,41 @@ def reset(opt):
        opt.reset()


+@pytest.fixture


thisac · 2021-08-05T22:46:59Z

pennylane/optimize/gradient_descent.py

+        num_trainable_args = 0
+        for arg in args:
+            if getattr(arg, "requires_grad", True):
+                num_trainable_args += 1


The issue seems to be that if the argument is not a PennyLane NumPy tensor, then it'll be considered to belong to the "numpy" interface (instead of the "autgrad" interface), and all "numpy" arguments automatically evaluate as False when calling utils.requires_grad(arg). So, whenever an arg is simply a float get_trainable_indices would consider it non-trainiable, while that seems to not be what's wanted here.

antalszava added 2 commits August 4, 2021 17:27

count the trainable args, not simply the args

caed76a

Merge branch 'master' into ch7853-fix_grad_desc_mult_args

d586c24

antalszava added 5 commits August 4, 2021 17:46

changelog

c79f5c0

update nesterov opt too

b07357f

update nesterov opt too

08fa4eb

comment

c57a3fb

changelog update

4751c69

antalszava marked this pull request as ready for review August 4, 2021 16:02

antalszava added 2 commits August 4, 2021 18:04

remove redundant enumerate

9d2af7c

format

cbe3457

antalszava changed the title ~~Update gradient descent~~ Fix how trainable args are counted for gradients in GradientDescentOptimizer and NesterovMomentumOptimizer Aug 4, 2021

add another test for two trainable args

eba42af

antalszava requested a review from josh146 August 4, 2021 16:24

antalszava mentioned this pull request Aug 4, 2021

[BUG] Noisy circuits demo doesn't work when run on Braket devices PennyLaneAI/qml#309

Closed

antalszava added 2 commits August 4, 2021 19:04

extract trainable args

0ea45d7

enumerate trainable args

c45226e

josh146 reviewed Aug 4, 2021

View reviewed changes

josh146 requested changes Aug 4, 2021

View reviewed changes

josh146 mentioned this pull request Aug 4, 2021

Update tutorial_noisy_circuits.py PennyLaneAI/qml#312

Merged

antalszava and others added 10 commits August 4, 2021 19:36

Update tests/test_optimize.py

d8ca734

Co-authored-by: Josh Izaac <josh146@gmail.com>

Update tests/test_optimize.py

a0d468c

Co-authored-by: Josh Izaac <josh146@gmail.com>

apply suggestion

386eb77

Merge branch 'ch7853-fix_grad_desc_mult_args' of github.com:PennyLane…

e29725f

…AI/pennylane into ch7853-fix_grad_desc_mult_args

add new test case

f448a4d

test docstring

210a32c

Revert "apply suggestion"

0b58d3e

This reverts commit 386eb77.

format

5e400fe

remove extra dimensionality from non-trainable

7a02ff8

create a fixture that returns a new optimizer object for each test ca…

2d85634

…se; the previous version left a state because the object lived on

format test

9e5526e

antalszava commented Aug 4, 2021

View reviewed changes

tests/test_optimize.py Outdated Show resolved Hide resolved

Update tests/test_optimize.py

675b7fc

antalszava commented Aug 4, 2021

View reviewed changes

antalszava requested review from josh146 and thisac August 4, 2021 20:33

josh146 approved these changes Aug 5, 2021

View reviewed changes

thisac approved these changes Aug 5, 2021

View reviewed changes

Merge branch 'master' into ch7853-fix_grad_desc_mult_args

ad0eb38

antalszava merged commit 53e1dc8 into master Aug 6, 2021

antalszava deleted the ch7853-fix_grad_desc_mult_args branch August 6, 2021 09:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix how trainable args are counted for gradients in `GradientDescentOptimizer` and `NesterovMomentumOptimizer` #1495

Fix how trainable args are counted for gradients in `GradientDescentOptimizer` and `NesterovMomentumOptimizer` #1495

antalszava commented Aug 4, 2021 •

edited

Loading

codecov bot commented Aug 4, 2021 •

edited

Loading

josh146 Aug 4, 2021

josh146 Aug 4, 2021

antalszava Aug 4, 2021

josh146 Aug 5, 2021

thisac Aug 5, 2021

josh146 left a comment

josh146 Aug 4, 2021

antalszava Aug 4, 2021

thisac Aug 5, 2021

josh146 left a comment

josh146 Aug 5, 2021

thisac left a comment

thisac Aug 5, 2021

thisac Aug 5, 2021

		@@ -747,16 +747,41 @@ def reset(opt):
		opt.reset()


		@pytest.fixture

Fix how trainable args are counted for gradients in GradientDescentOptimizer and NesterovMomentumOptimizer #1495

Fix how trainable args are counted for gradients in GradientDescentOptimizer and NesterovMomentumOptimizer #1495

Conversation

antalszava commented Aug 4, 2021 • edited Loading

codecov bot commented Aug 4, 2021 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josh146 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josh146 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thisac left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Fix how trainable args are counted for gradients in `GradientDescentOptimizer` and `NesterovMomentumOptimizer` #1495

Fix how trainable args are counted for gradients in `GradientDescentOptimizer` and `NesterovMomentumOptimizer` #1495

antalszava commented Aug 4, 2021 •

edited

Loading

codecov bot commented Aug 4, 2021 •

edited

Loading