Refactor template tests

[mariaschuld]

Context: This PR is part of the templates redesign.

Description of the Change: Heavily improves and extends the integration tests. They are now fully automated, so that a developer only has to add the template and inputs to be tested into a list, and it gets automatically tested for different interfaces and different ways to pass parameters.

Benefits: Simpler, more elegant and more extensive tests.

Possible Drawbacks: This PR necessarily contains the small changes in PR #403, which fixes how the templates iterate through weight arrays (making it compatible with tensorflow). I would therefore recommend merging #403 first.

Tests that will be added are integrations for different hyperparameters. However, these tests will fail at the moment, an issue addressed in a separate PR (which will also update the tests).

Related GitHub Issues: -

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@9c4eb72). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master     #409   +/-   ##
=========================================
  Coverage          ?   99.24%           
=========================================
  Files             ?       42           
  Lines             ?     3030           
  Branches          ?        0           
=========================================
  Hits              ?     3007           
  Misses            ?       23           
  Partials          ?        0

Impacted Files	Coverage Δ
pennylane/templates/layers.py	100% <100%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9c4eb72...97dd76b. Read the comment docs.

[co9olguy]

Left a few minor suggestions. My main question is whether the list comprehension which explicitly casts each entry to a numpy array might be masking anything, i.e., shouldn't we also check the cases where the inputs are standard python numbers? Or are there no situations like this, due to how the parameters of a layer are structured?

[josh146]

Nice work @mariaschuld!

There are two things I think that are missing though:

• An integration test for the def circuit(weights): pattern a user might try

• Gradient tests. Something simple, such as doing qml.grad(), torch.backwards() and tape.grad() for each of the three interfaces, and making sure this does not raise an exception or return None. Especially working on the VQE module, I found cases where evaluating a circuit with templates would work fine, but then the gradient would raise an exception with TF.

  Note: with the latter, interface parametrization won't work --- better to have a gradient test class per interface

[josh146]

Oh, another thought - could you also add the test case that you know fails (using the class instead of the decorator), but mark it as xfail using Pytest? That way we have a benchmark for when it is fixed 🙂

[mariaschuld]

Working on the grad tests and the xfail one.

Isn't the case circuit(weights) covered by test_integration_XX_args in test_templates.py? Only that the number of weights is dynamically inferred.

[mariaschuld]

Thinking about it - it is major code duplication to add the xfail tests, which would then test something that is completely unrelated to templates.

I suggest we rather add a test to qnode itself.

Copying the MWE here:

import pennylane as qml
import torch

inpts = [[1], [2]]
inpts = [torch.tensor(i) for i in inpts]
dev = qml.device('default.qubit', wires=2)

# Decorator version works well: 
qml.qnode(dev, interface='torch')
def circuit1(*inp_):
    return qml.expval(qml.PauliX(1))
print("Works:", circuit1(*inpts))

# Manual version fails: 
def circuit2(*inp_):
    return qml.expval(qml.PauliX(1))
circuit2 = qml.QNode(circuit2, dev)
circuit2.to_torch()
print("Fails:", circuit2(*inpts))

[mariaschuld]

From discussion with @josh146 :

• The xfail to keep track of a bug in manual qnode construction has to be added in another module.
• For now we will limit the test to the elegant circuit(*weights) signature, since it would require a very complex rewrite to also include circuit(weights) [however, we should keep an eye on this because interfaces may have some exotic differences in how the cases are handled].
• Gradient tests are now added

[co9olguy]

Thanks @mariaschuld. I only have a couple small questions and comments, but nothing to block merging

[mariaschuld]

This one is smuggled in from a later PR, but it makes the tests run with 1 wire as well...

[mariaschuld]

From here on the diff becomes unreadable...I recommend looking at it in the file...

[mariaschuld]

See smuggled in fix, now SEL can take 1 wire too