Use the new `multi_dispatch` decorator in the math module. Add `tensordot` tests. #2096

dwierichs · 2022-01-18T10:14:58Z

Context:
This PR makes use of the multi_dispatch decorator that was introduced in #2084 throughout pennylane/math/multi_dispatch.py. It also introduces tests of differentiability in all interfaces for qml.math.tensordot, as well as a minimal test of tensordot with Sequences as inputs.

#1865 was partially closed by #1725, but a qml.math.coerce is introduced here to tensordot to complete this patch of tensordot.

Description of the Change:
Some more tests; Usage of multi_dispatch.

Benefits:
Concise codebase and usage examples of the new decorator; Test coverage

Possible Drawbacks:
Tests take longer.
Performance might be reduced minimally by using a higher-level object (the decorator) rather than hard-coding the usage of _multi_dispatch to determine the interface.

Related GitHub Issues:
closes #1865

josh146 · 2022-01-18T10:31:34Z

pennylane/math/multi_dispatch.py

+# Note that diag is not eligible for the multi_dispatch decorator because
+# it is used sometimes with iterable `values` that need to be interpreted
+# as a list of tensors, and sometimes with a single tensor `values` that
+# might not be iterable (for example a TensorFlow `Variable`)


@dwierichs is it possible to modify the decorator to allow for this?

I'm not sure. It would have to try to iterate over the arguments in (via the list method extend) that are marked via tensor_list, and if it fails, attempt to just append the respective argument to the arguments to dispatch over.
I'd consider this an unreasonable overhead given the currently single use case. What do you think?

Oh, I was thinking something simpler, although maybe it is not in the spirit of 'multiple dispatch'.

Basically, check not isinstance(arg, (list, tuple)), and if this is the case, call qml.math.get_interface(arg).

Basically, we assume that a tensor_list argument must be a built-in Python sequence (either list or tuple), and if this is not the case, we assume it must be a tensor and simply single dispatch

Ah I see, that sounds cool! I'll give it a go :)

I implemented this now, a bit more lightweight, even: If an argument marked as tensor list is not a tuple or list, it is simply treated as if it was not marked as a tensor list.

ah, nice call!

…ne into multi_dispatch-usage

codecov · 2022-01-19T00:11:22Z

Codecov Report

Merging #2096 (58c6661) into master (dd50a7f) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #2096   +/-   ##
=======================================
  Coverage   99.19%   99.19%           
=======================================
  Files         228      228           
  Lines       17534    17534           
=======================================
  Hits        17393    17393           
  Misses        141      141

Impacted Files	Coverage Δ
pennylane/devices/default_qubit.py	`100.00% <100.00%> (ø)`
pennylane/math/is_independent.py	`100.00% <100.00%> (ø)`
pennylane/math/multi_dispatch.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dd50a7f...58c6661. Read the comment docs.

josh146

Thanks @dwierichs! I must admit I am a bit confused, I can't see the changes as described in the PR comment --- it looks mostly like test changes. Are there commits that haven't been pushed?

pennylane/devices/default_qubit.py

josh146 · 2022-01-19T06:42:32Z

pennylane/math/is_independent.py

@@ -203,7 +203,7 @@ def _get_random_args(args, interface, num, seed, bounds):
        if interface == "autograd":

            # Mark the arguments as trainable with Autograd
-            rnd_args = pnp.array(rnd_args, requires_grad=True)
+            rnd_args = [tuple(pnp.array(a, requires_grad=True) for a in arg) for arg in rnd_args]


Also curious about the change here 🤔

This was producing ragged arrays, raising the corresponding warning. It was very easy to fix and is not
related to the ragged array issues of QNodes, it was just not so nicely written by me back then, producing an array of a list of tuples of different sizes.

pennylane/math/multi_dispatch.py

tests/math/test_basic_math.py

tests/math/test_functions.py

josh146 · 2022-01-19T06:43:56Z

tests/math/test_functions.py

@@ -1362,14 +1452,14 @@ def test_array(self):
        x = np.array(self.x, requires_grad=True)
        y = np.array(self.y, requires_grad=True)

-        def cost(weights):
+        def cost(*weights):


How come this test was changed?

While x and y are markes as trainable above, the list [x, y] of course is not. This triggered the warning that with Autograd, soon parameters will need to be marked as trainable explicitly. I'm aware that this is in an intermediate state across the codebase currently, but I thought it might be nice to change it here already, removing warnings from the test suite :)

tests/math/test_functions.py

josh146 · 2022-01-19T06:44:26Z

tests/math/test_is_independent.py

+    lambda x: qml.math.log(1 + qml.math.exp(100.0 * x)) / 100.0,  # Softplus is okay
+    lambda x: qml.math.log(1 + qml.math.exp(100.0 * x)) / 100.0,  # Softplus is okay


What was the reason for this change?

With the chosen parameters and this massive prefactor of 1000, the exponential function was raising an overflow warning, actually 😅
This change essentially does not affect what is tested.

dwierichs · 2022-01-19T09:41:31Z

tests/math/test_functions.py

@@ -1362,14 +1452,14 @@ def test_array(self):
        x = np.array(self.x, requires_grad=True)
        y = np.array(self.y, requires_grad=True)

-        def cost(weights):
+        def cost(*weights):


While x and y are markes as trainable above, the list [x, y] of course is not. This triggered the warning that with Autograd, soon parameters will need to be marked as trainable explicitly. I'm aware that this is in an intermediate state across the codebase currently, but I thought it might be nice to change it here already, removing warnings from the test suite :)

dwierichs · 2022-01-19T09:41:57Z

tests/math/test_functions.py

@@ -1871,7 +1961,7 @@ def test_autograd(self):
            np.array([[x, 1.2 * y], [x ** 2 - y / 3, -x / y]]),
        ]
        f = lambda x, y: fn.block_diag(tensors(x, y))
-        x, y = 0.2, 1.5
+        x, y = np.array([0.2, 1.5], requires_grad=True)


This is to remove the same warning as above (i.e. explicit trainability marking in Autograd)

dwierichs · 2022-01-19T09:42:09Z

tests/math/test_functions.py

+        values = [
+            onp.array([0.1, 0.2]),
+            np.tensor(0.1, dtype=np.float64, requires_grad=True),
+            np.tensor([0.5, 0.2], requires_grad=True),
+        ]
+        grad = qml.grad(cost_fn, argnum=[1, 2])(*values)


tests/math/test_functions.py

dwierichs · 2022-01-19T09:43:22Z

tests/math/test_is_independent.py

+    lambda x: qml.math.log(1 + qml.math.exp(100.0 * x)) / 100.0,  # Softplus is okay
+    lambda x: qml.math.log(1 + qml.math.exp(100.0 * x)) / 100.0,  # Softplus is okay


With the chosen parameters and this massive prefactor of 1000, the exponential function was raising an overflow warning, actually 😅
This change essentially does not affect what is tested.

dwierichs · 2022-01-19T09:44:04Z

tests/math/test_is_independent.py

@@ -167,7 +167,7 @@ def dependent_circuit(x, y, z):
        dependent_circuit,
        np.array,
        lambda x: np.array(x * 0.0),
-        lambda x: (1 + qml.math.tanh(1000 * x)) / 2,
+        lambda x: (1 + qml.math.tanh(100 * x)) / 2,


This again is to avoid overflow warnings.

dwierichs · 2022-01-19T09:45:02Z

tests/math/test_is_independent.py

@@ -519,7 +519,7 @@ def dependent_circuit(x, y, z):

        dependent_functions = [
            dependent_circuit,
-            torch.tensor,
+            torch.as_tensor,


Applying torch.tensor to a Torch tensor raises a warning to use an appropriate copying method. We here just want to test some form of identity function, so as_tensor is as good as tensor and it does not raise the warning :)

pennylane/devices/default_qubit.py

dwierichs · 2022-01-19T09:47:10Z

pennylane/math/is_independent.py

@@ -203,7 +203,7 @@ def _get_random_args(args, interface, num, seed, bounds):
        if interface == "autograd":

            # Mark the arguments as trainable with Autograd
-            rnd_args = pnp.array(rnd_args, requires_grad=True)
+            rnd_args = [tuple(pnp.array(a, requires_grad=True) for a in arg) for arg in rnd_args]


This was producing ragged arrays, raising the corresponding warning. It was very easy to fix and is not
related to the ragged array issues of QNodes, it was just not so nicely written by me back then, producing an array of a list of tuples of different sizes.

dwierichs · 2022-01-19T09:48:34Z

tests/math/test_functions.py

+            lambda *weights: cost(*weights)[self.indices[0][0], self.indices[1][0]]
+            + cost(*weights)[self.indices[0][1], self.indices[1][1]]


josh146

Thanks @dwierichs, this looks great to me! Thank you for clarifying some of my questions from earlier

josh146 · 2022-01-19T13:54:32Z

pennylane/math/multi_dispatch.py

@@ -376,11 +378,12 @@ def tensordot(tensor1, tensor2, axes=None):
    Returns:
        tensor_like: the tensor product of the two input tensors
    """
-    interface = _multi_dispatch([tensor1, tensor2])
-    return np.tensordot(tensor1, tensor2, axes=axes, like=interface)
+    tensor1, tensor2 = np.coerce([tensor1, tensor2], like=like)


This is minor, but do you think we should include coerce inside the decorator? Or is the required behaviour to specific to different functions?

The reason I suggest this is we could simplify even more, and just do

tensordot = multi_dispatch(argnum=[0, 1])(np.tensordot)

I was thinking about this as well 😆 Maybe as an optional kwarg?

tensordot = multi_dispatch(argnum=[0, 1], do_coerce=True)(np.tensordot)

There are multiple functions to which this would apply, e.g. block_diag, stack, tensordot. For other functions, it would just safe a line in the patched function, e.g. in frobenius_inner_product and dot.

I like it 😎 Although, I might even just have coerce=True?

Discussed off-PR: Including this in multi_dispatch is less straight-forward than previously thought, due to unpacking and re-packing of arguments in that are sequences of tensors. Therefore it will not be included.

dwierichs added 7 commits January 17, 2022 23:42

use multi_dispatch decorator across math module

a5e2746

tests

308ddef

more tests

270837d

changelog

fe6b027

Merge branch 'master' into multi_dispatch-usage

8ac4d09

filename type

ea4296b

filename typo

f02fc56

josh146 reviewed Jan 18, 2022

View reviewed changes

dwierichs added 5 commits January 18, 2022 11:43

Merge branch 'multi_dispatch-usage' of github.com:PennyLaneAI/pennyla…

a8e1604

…ne into multi_dispatch-usage

intermediate

cd8ac5a

Merge branch 'master' into multi_dispatch-usage

b80bb81

Merge branch 'multi_dispatch-usage' of github.com:PennyLaneAI/pennyla…

5b3b255

…ne into multi_dispatch-usage

tmp check

2c7ef26

reduce warnings across math module test suite

12abf98

dwierichs requested a review from josh146 January 19, 2022 01:21

josh146 reviewed Jan 19, 2022

View reviewed changes

dwierichs commented Jan 19, 2022

View reviewed changes

include multi_dispatch applications, again.

3afc884

dwierichs requested a review from josh146 January 19, 2022 10:35

Merge branch 'master' into multi_dispatch-usage

58c6661

josh146 approved these changes Jan 19, 2022

View reviewed changes

dwierichs merged commit a19c744 into master Jan 19, 2022

dwierichs deleted the multi_dispatch-usage branch January 19, 2022 15:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use the new `multi_dispatch` decorator in the math module. Add `tensordot` tests. #2096

Use the new `multi_dispatch` decorator in the math module. Add `tensordot` tests. #2096

dwierichs commented Jan 18, 2022

josh146 Jan 18, 2022

dwierichs Jan 18, 2022

josh146 Jan 18, 2022 •

edited

Loading

dwierichs Jan 18, 2022

dwierichs Jan 19, 2022

josh146 Jan 19, 2022

codecov bot commented Jan 19, 2022 •

edited

Loading

josh146 left a comment

josh146 Jan 19, 2022

dwierichs Jan 19, 2022

josh146 Jan 19, 2022

dwierichs Jan 19, 2022

josh146 Jan 19, 2022

dwierichs Jan 19, 2022

dwierichs Jan 19, 2022

dwierichs Jan 19, 2022

dwierichs Jan 19, 2022

dwierichs Jan 19, 2022

dwierichs Jan 19, 2022

dwierichs Jan 19, 2022

dwierichs Jan 19, 2022

dwierichs Jan 19, 2022

josh146 left a comment

josh146 Jan 19, 2022

dwierichs Jan 19, 2022 •

edited

Loading

josh146 Jan 19, 2022

dwierichs Jan 19, 2022

		lambda x: qml.math.log(1 + qml.math.exp(100.0 * x)) / 100.0, # Softplus is okay
		lambda x: qml.math.log(1 + qml.math.exp(100.0 * x)) / 100.0, # Softplus is okay

		lambda weights: cost(weights)[self.indices[0][0], self.indices[1][0]]
		+ cost(*weights)[self.indices[0][1], self.indices[1][1]]

Use the new multi_dispatch decorator in the math module. Add tensordot tests. #2096

Use the new multi_dispatch decorator in the math module. Add tensordot tests. #2096

Conversation

dwierichs commented Jan 18, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josh146 Jan 18, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jan 19, 2022 • edited Loading

Codecov Report

josh146 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josh146 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dwierichs Jan 19, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Use the new `multi_dispatch` decorator in the math module. Add `tensordot` tests. #2096

Use the new `multi_dispatch` decorator in the math module. Add `tensordot` tests. #2096

josh146 Jan 18, 2022 •

edited

Loading

codecov bot commented Jan 19, 2022 •

edited

Loading

dwierichs Jan 19, 2022 •

edited

Loading