specs transform #1245

albi3ro · 2021-04-21T16:09:08Z

This PR introduces a "transform" that calculates circuit resources. The function qml.transforms.resource_estimation takes a qnode, and returns a function. When that function is called with circuit arguments, it returns a dictionary of information.

import pennylane as qml
from pennylane import numpy as np

n_wires = 3
dev = qml.device('default.qubit', wires=n_wires)

weights = np.array([[0.5, 0.6, 1.0]])

@qml.qnode(dev)
def circuit(weights):
    qml.templates.BasicEntanglerLayers(weights=weights, wires=range(n_wires))
    return [qml.expval(qml.PauliZ(wires=i)) for i in range(n_wires)]

resources = qml.transforms.resource_estimation(circuit)
resources(weights)

{'dev_short_name': 'default.qubit.autograd',
 'num_wires': 3,
 'num_gates': 6,
 'num_ops_by_size': {1: 3, 2: 3}}

The current values are

dev_short_name
num_wires: the number of wires on the device
num_gates: total number of gates
num_ops_by_size: a dictionary that gives the number of operations of a certain size. For example, the above circuit has three single qubit gates and three two-qubit gates.

The qml.transforms.resource_estimation function can take in a keyword expand_depth, but it looks like qnode.construct expanded everything anyway.

In the future, I would like to add time scaling for applicable devices given relevant circuit structure information.

github-actions · 2021-04-21T16:09:27Z

Hello. You may have forgotten to update the changelog!
Please edit .github/CHANGELOG.md with:

A one-to-two sentence description of the change. You may include a small working example for new features.
A link back to this PR.
Your name (or GitHub username) in the contributors section.

codecov · 2021-04-21T16:18:01Z

Codecov Report

Merging #1245 (e6b6e92) into master (2230611) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #1245   +/-   ##
=======================================
  Coverage   98.20%   98.21%           
=======================================
  Files         159      160    +1     
  Lines       11838    11888   +50     
=======================================
+ Hits        11626    11676   +50     
  Misses        212      212

Impacted Files	Coverage Δ
pennylane/__init__.py	`98.57% <ø> (ø)`
pennylane/transforms/draw.py	`100.00% <ø> (ø)`
pennylane/qnode.py	`97.96% <100.00%> (+0.11%)`	⬆️
pennylane/tape/qubit_param_shift.py	`100.00% <100.00%> (ø)`
pennylane/tape/tape.py	`97.79% <100.00%> (+0.06%)`	⬆️
pennylane/transforms/__init__.py	`100.00% <100.00%> (ø)`
pennylane/transforms/specs.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2230611...e6b6e92. Read the comment docs.

chaserileyroberts · 2021-04-21T16:17:47Z

pennylane/transforms/resource_estimation.py

+        qnode.construct(args, kwargs)
+        tape = qnode.qtape


Suggested change

qnode.construct(args, kwargs)

tape = qnode.qtape

tape = qnode.construct(args, kwargs)

You can skip a step here.

yeah that doesn't work.

chaserileyroberts · 2021-04-21T16:19:10Z

pennylane/transforms/resource_estimation.py

+"""Code for resource estimation"""
+
+
+def resource_estimation(qnode, expand_depth=1):


Hmm... not sure how I feel about expand_depth being here. But that leads into a whole other conversation around how tapes get expanded on the device, and that's probably outside of the design scope here....

chaserileyroberts · 2021-04-21T16:19:33Z

pennylane/transforms/resource_estimation.py

+        qnode (qml.QNode): a PL Qnode
+
+    Returns:
+        function: a function of the same parameters as the qnode


Suggested change

function: a function of the same parameters as the qnode

A function that has same inputs as the given qnode and returns a resource estimation dictionary.

chaserileyroberts · 2021-04-21T16:25:22Z

Yeah this looks pretty good! Now all we need is a function tape -> float that estimates the single shot cost to run on braket.

glassnotes · 2021-04-21T16:27:54Z

(Just a fly-by comment): It might be worth absorbing the existing tape.get_resources and tape.get_depth functions into this, since it looks like it's going to be a more general resource estimator, which is awesome!

mariaschuld · 2021-06-10T15:39:36Z

pennylane/transforms/specs.py

+            executed on a device. Expansion occurs when an operation or measurement is not
+            supported, and results in a gate decomposition. If any operations in the decomposition
+            remain unsupported by the device, another expansion occurs. Defaults to
+            ``qnode.max_expansion``.


Just to understand, why do we give access to exactly that one? For example, I would find it much more useful to give user access to the stopping criterion for expansion, if anything...

I'm just mirroring how QNode's work.

I also wondered 🤔. If we have to expand to depth N for the device to execute, what would happen if max_expansion < N? Wouldn't it just not work on the device? Although I guess you can still access the specs?

E.g.,

dev = qml.device('default.qubit', wires=2) @qml.qnode(dev) def circuit(): qml.templates.StronglyEntanglingLayers(np.random.random((3, 2, 3)), wires=range(2)) return qml.probs(wires=(0,1)) qml.specs(circuit, max_expansion=0)()

pennylane/transforms/specs.py

mariaschuld · 2021-06-10T15:42:18Z

tests/tape/test_qnode.py

+
+        assert info["num_device_wires"] == 4
+        assert info["device_name"] == "default.qubit"
+        assert info["diff_method"] == "adjoint"


I think it would be nice to add tests for:

a qnode without gates

a qnode where the tape changes with the arguments

But nice to have the tests in different diff methods!

Some slight variation across diff methods, like adding parameter shift executions or deleting "num_trainbable_params".

Would also be nice to have a case for the behaviour of having multiple measurements in the return statement of the qfunc.

tests/tape/test_qubit_param_shift.py

mariaschuld · 2021-06-10T15:43:35Z

tests/tape/test_tape.py

+        assert tape.specs["num_trainable_params"] == 0
+        assert tape.specs["depth"] == 0
+
+        assert len(tape.specs) == 7


Oh nice, ignore my comment above then!

As I found out, empty circuits have their own peculiarties, so I'm adding tests for empty qnodes.

tests/tape/test_tape.py

mariaschuld · 2021-06-10T15:45:46Z

.github/CHANGELOG.md

+ 'by_name': defaultdict(int, {'RX': 1, 'Toffoli': 1, 'CRY': 1, 'Rot': 1}),
+ 'total_operations': 4,
+ 'total_observables': 2,
+ 'num_tape_wires': 3,


I would not talk about tapes, since this is not user facing at all. Can we say num_used_wires then?

mariaschuld · 2021-06-10T15:46:01Z

.github/CHANGELOG.md

+ 'total_operations': 4,
+ 'total_observables': 2,
+ 'num_tape_wires': 3,
+ 'depth': 3,


What about "circuit_depth"?

mariaschuld

I did not to the deepest review, but went through everything. Giving approval because I think as it is it will improve the code base, and there was nothing wrong I could see (plus I will be on vacation). My only suggestions (non-blocking, but strongly recommended) are namings.

Very nice addition @albi3ro , and it uncovered some unelegant parts of the deepest qnode code base.

trbromley

Thanks @albi3ro! So far I have just looked at the changelog and tried out this new feature. It's very cool and will be fun to use! I left a few questions that would be good to understand.

.github/CHANGELOG.md

trbromley · 2021-06-10T15:48:47Z

.github/CHANGELOG.md

+ 'depth': 3,
+ 'num_trainable_params': 4,
+ 'num_parameter_shift_executions': 7,
+ 'num_device_wires': 4,


Side note: it'd be great to finally get rid of device wires for simulators, and only use as many wires as there are in the tape!

trbromley · 2021-06-10T15:51:23Z

.github/CHANGELOG.md

+ 'num_tape_wires': 3,
+ 'depth': 3,
+ 'num_trainable_params': 4,
+ 'num_parameter_shift_executions': 7,


Does this include the forward pass execution? E.g., if you set requires_grad=False for x, this number is 1.

Also, how is this number 7? Since we have 4 params shouldn't it be 9?

Would this make sense as num_device_executions?

.github/CHANGELOG.md

trbromley

Thanks @albi3ro!

trbromley · 2021-06-10T16:09:02Z

pennylane/qnode.py

@@ -720,6 +720,63 @@ def circuit():
            charset=charset, wire_order=wire_order, show_all_wires=show_all_wires
        )

+    @property
+    def specs(self):
+        """


Think it might be nicer to have a single-sentence summary here (e.g., same as what you have below), and then in the Returns: it can just be a word or two. Currently the docs renders the summary weirdly:

trbromley · 2021-06-10T16:11:33Z

pennylane/qnode.py

+        # tape's do not accurately track parameters for backprop
+        # TODO: calculate number of trainable parameters in backprop
+        # find better syntax for determining if backprop
+        if info["diff_method"] == "backprop":
+            del info["num_trainable_params"]


Nice, probably answers my question above.

trbromley · 2021-06-10T16:55:36Z

pennylane/tape/qubit_param_shift.py

+    def specs(self):
+
+        if "grad_method" not in self._par_info[0]:
+            self._update_gradient_info()


Is this for if someone asks for specs before actually having evaluated the gradient?

trbromley · 2021-06-10T17:03:19Z

pennylane/tape/tape.py

-                else:
-                    self._resources[op.name] += 1
+        warnings.warn(
+            "``tape.get_resources`` will be deprecated after v0.16 "


Do we want to say when it will be removed?
E.g.
"tape.get_resources is now deprecated and will be removed in v0.17"

pennylane/tape/tape.py

trbromley · 2021-06-10T17:17:33Z

pennylane/transforms/specs.py

+
+
+def specs(qnode, max_expansion=None):
+    """


It'd be good to have a one sentence summary line, otherwise the docs look weird:

Indeed, and worth including that a QNode construction is required to obtain data here.

trbromley · 2021-06-10T17:23:03Z

pennylane/tape/qubit_param_shift.py

+        info = super().specs
+        info["num_parameter_shift_executions"] = num_executions
+
+        return info


I just realised specs doesn't work so well with circuits that include templates:

import pennylane as qml from pennylane import numpy as np x = np.array([0.1, 0.2]) with qml.tape.QubitParamShiftTape() as tape: qml.templates.StronglyEntanglingLayers(np.random.random((3, 2, 3)), wires=range(2)) tape.specs

Gives

{'by_size': defaultdict(int, {2: 1}), 'by_name': defaultdict(int, {'StronglyEntanglingLayers': 1}), 'total_operations': 1, 'total_observables': 0, 'num_tape_wires': 2, 'depth': 1, 'num_trainable_params': 1, 'num_parameter_shift_executions': 1}

Though in practice, there are many trainable params and there will be many param shift executions 🤔.
There may not be an easy fix for this though. And also things work nicely on the QNode level as expansion will have happened.

Do you think this is something that needs fixing for this PR? Most users are not going to working with tapes, and there are multiple shortcomings to a tape with an unexpanded template.

I agree, maybe something we can think about as we improve the Operation class. (I believe this subtlety is due to moving templates to operations, since they'd be automatically expanded before?)

trbromley · 2021-06-10T17:27:50Z

pennylane/transforms/specs.py

+            executed on a device. Expansion occurs when an operation or measurement is not
+            supported, and results in a gate decomposition. If any operations in the decomposition
+            remain unsupported by the device, another expansion occurs. Defaults to
+            ``qnode.max_expansion``.


I also wondered 🤔. If we have to expand to depth N for the device to execute, what would happen if max_expansion < N? Wouldn't it just not work on the device? Although I guess you can still access the specs?

trbromley · 2021-06-10T17:28:09Z

pennylane/transforms/specs.py

+            executed on a device. Expansion occurs when an operation or measurement is not
+            supported, and results in a gate decomposition. If any operations in the decomposition
+            remain unsupported by the device, another expansion occurs. Defaults to
+            ``qnode.max_expansion``.


E.g.,

dev = qml.device('default.qubit', wires=2) @qml.qnode(dev) def circuit(): qml.templates.StronglyEntanglingLayers(np.random.random((3, 2, 3)), wires=range(2)) return qml.probs(wires=(0,1)) qml.specs(circuit, max_expansion=0)()

pennylane/transforms/specs.py

antalszava

This is looking great! 😊 Will be very useful. My comments are mostly for consideration.

pennylane/qnode.py

pennylane/tape/tape.py

antalszava · 2021-06-10T19:24:54Z

pennylane/tape/tape.py

+
+            for op in self.operations:
+                # don't use op.num_wires to allow for flexible gate classes like hermitian
+                self._specs["by_size"][len(op.wires)] += 1


Perhaps this was decided on previously, but the names "by_size" and "by_name" do not seem completely descriptive, how about pretending "ops_" to them? e.g., "ops_by_size", etc.

Due to Maria's suggestion, we are now using "gate_sizes" and "gate_types"

pennylane/tape/tape.py

antalszava · 2021-06-10T19:27:37Z

pennylane/tape/tape.py

+
+            self._specs["total_operations"] = len(self.operations)
+            self._specs["total_observables"] = len(self.observables)
+            self._specs["num_tape_wires"] = self.num_wires


How about the "num_all_wires" or simply "all_wires" here? "num_tape_wires" assumes knowledge about having quantum tapes represent quantum circuits. When querying a QNode, a user might not possess the knowledge that there's an underlying quantum tape in a QNode and might become confused.

tests/tape/test_qnode.py

antalszava · 2021-06-10T19:47:36Z

tests/tape/test_qnode.py

+        assert info["num_tape_wires"] == 3
+        assert info["depth"] == 3
+        assert info["num_trainable_params"] == 4
+        assert info["num_parameter_shift_executions"] == 7


Maybe these three tests could be parametrized, as this seems to be the only line that is different. This one could be checked with an if statement.

antalszava · 2021-06-10T19:48:31Z

pennylane/tape/qubit_param_shift.py

+        # Loop over all variables
+        for _, info in self._par_info.items():
+
+            if info["grad_method"] == "A":


How about finite diff and CV circuits? Maybe not too important for now, but might be worth considering later.

I've thought about it a little and will probably try to add them in later.

antalszava · 2021-06-10T19:50:22Z

tests/tape/test_qnode.py

+
+        assert info["num_device_wires"] == 4
+        assert info["device_name"] == "default.qubit"
+        assert info["diff_method"] == "adjoint"


Would also be nice to have a case for the behaviour of having multiple measurements in the return statement of the qfunc.

Co-authored-by: Tom Bromley <49409390+trbromley@users.noreply.github.com> Co-authored-by: Maria Schuld <mariaschuld@gmail.com> Co-authored-by: antalszava <antalszava@gmail.com>

…pennylane into resource_estimation

trbromley

Thanks @albi3ro! Just left some suggestions, otherwise looks awesome!

pennylane/qnode.py

pennylane/tape/qubit_param_shift.py

trbromley · 2021-06-11T20:29:33Z

pennylane/tape/qubit_param_shift.py

+
+        info = super().specs
+
+        if any(m.return_type is qml.operation.State for m in self.measurements):


Just to check, this also works for density matrix too right? 🤔

Yep. It was a state thing.

trbromley · 2021-06-11T20:31:53Z

pennylane/tape/qubit_param_shift.py

+        for _, grad_info in self._par_info.items():
+
+            # if this variable uses parameter-shift
+            if grad_info["grad_method"] == "A":


Do we ever have finite-diff gates in the qubit setting? 🤔

Adding finite-diff is on the todo list.

trbromley · 2021-06-11T20:33:45Z

pennylane/tape/qubit_param_shift.py

+        info = super().specs
+        info["num_parameter_shift_executions"] = num_executions
+
+        return info


I agree, maybe something we can think about as we improve the Operation class. (I believe this subtlety is due to moving templates to operations, since they'd be automatically expanded before?)

trbromley · 2021-06-11T20:38:30Z

pennylane/transforms/specs.py

+            * ``"total_operations"``
+            * ``"total_observables"``
+            * ``"by_size"``: dictionary mapping gate number of wires to number of occurances
+            * ``"by_name"``: dictionary mapping gate types to number of occurances
+            * ``"num_tape_wires"``: number of wires used by the circuit
+            * ``"num_wires"``: number of wires in device
+            * ``"depth"``: longest path in directed acyclic graph representation
+            * ``"dev_short_name"``: name of QNode device
+            * ``"diff_method"``
+
+        Potential Additional Information:
+            * ``"num_trainable_params"``: number of individual scalars that are trainable
+            * ``"num_parameter_shift_executions"``: number of times circuit will execute when


Just to check, is this up to date? Maybe the diagonalizing gates one.

Not up to date. Thanks for catching that! Tiny little changes having to get propagated to a million places.

pennylane/transforms/specs.py

trbromley · 2021-06-11T20:45:30Z

tests/transforms/test_specs.py

+        assert info["depth"] == 1
+        assert info["num_device_wires"] == 5
+        assert info["device_name"] == "default.qubit.autograd"
+        assert info["diff_method"] == "backprop"


My only worry would be if there's any more edge cases, but the tests pass and coverage is 100% so that should be good!

Co-authored-by: Tom Bromley <49409390+trbromley@users.noreply.github.com>

albi3ro added 2 commits April 21, 2021 10:23

initial resource estimation

a3c5160

initial resource estimation

60b61eb

albi3ro added the WIP 🚧 Work-in-progress label Apr 21, 2021

black

358eac4

chaserileyroberts reviewed Apr 21, 2021

View reviewed changes

albi3ro mentioned this pull request May 11, 2021

[WIP] Resource estimation Option 2: context manager #1295

Closed

albi3ro and others added 5 commits May 18, 2021 18:41

properties, ps num executions, qnode methods

dc2de57

Merge branch 'master' into resource_estimation

fe9e845

change I forgot to add

116b528

Merge branch 'master' into resource_estimation

f713ecd

specs transform

535dfa6

albi3ro changed the title ~~[WIP] Resource estimation~~ [WIP] specs transform Jun 4, 2021

albi3ro and others added 3 commits June 3, 2021 22:22

docstrings

7e51f07

Merge branch 'master' into resource_estimation

ca8a56b

some progress

b1cba49

antalszava added this to the v0.16.0 milestone Jun 8, 2021

albi3ro and others added 5 commits June 9, 2021 11:08

tape specs tests, total_observables

0d473f1

Merge branch 'master' into resource_estimation

b11571f

parametershift exeuctions on tape tested

e62c740

qnode tests

c410204

transform tests

b0b874a

albi3ro changed the title ~~[WIP] specs transform~~ specs transform Jun 10, 2021

albi3ro removed the WIP 🚧 Work-in-progress label Jun 10, 2021

albi3ro and others added 3 commits June 9, 2021 21:11

Merge branch 'master' into resource_estimation

cb5c7e1

changelog

18b5d1f

remove duplicated function

5e2ee88

mariaschuld reviewed Jun 10, 2021

View reviewed changes

pennylane/transforms/specs.py Show resolved Hide resolved

mariaschuld reviewed Jun 10, 2021

View reviewed changes

tests/tape/test_qubit_param_shift.py Outdated Show resolved Hide resolved

mariaschuld reviewed Jun 10, 2021

View reviewed changes

tests/tape/test_tape.py Show resolved Hide resolved

mariaschuld reviewed Jun 10, 2021

View reviewed changes

mariaschuld approved these changes Jun 10, 2021

View reviewed changes

trbromley requested changes Jun 10, 2021

View reviewed changes

trbromley reviewed Jun 10, 2021

View reviewed changes

antalszava reviewed Jun 10, 2021

View reviewed changes

albi3ro and others added 8 commits June 11, 2021 08:52

Apply suggestions from code review

2912d49

Co-authored-by: Tom Bromley <49409390+trbromley@users.noreply.github.com> Co-authored-by: Maria Schuld <mariaschuld@gmail.com> Co-authored-by: antalszava <antalszava@gmail.com>

code review comments updated

1c87df3

test updates

86196d2

Merge branch 'master' into resource_estimation

73cdc8b

updated tests, review comments

c42aa14

Merge branch 'resource_estimation' of https://github.com/PennyLaneAI/…

6da7a2c

…pennylane into resource_estimation

black and docstrings

6a4fbae

docstring

6e5637c

albi3ro requested review from trbromley and antalszava June 11, 2021 17:35

albi3ro and others added 2 commits June 11, 2021 12:41

black

82fde2b

Merge branch 'master' into resource_estimation

8e8bcbb

trbromley approved these changes Jun 11, 2021

View reviewed changes

albi3ro and others added 3 commits June 11, 2021 15:55

Apply suggestions from code review

dce091e

Co-authored-by: Tom Bromley <49409390+trbromley@users.noreply.github.com>

total to num

6f19e42

total to num

e6b6e92

albi3ro merged commit 8e69187 into master Jun 11, 2021

albi3ro deleted the resource_estimation branch June 11, 2021 21:36

	qnode.construct(args, kwargs)
	tape = qnode.qtape
	tape = qnode.construct(args, kwargs)

		"""Code for resource estimation"""


		def resource_estimation(qnode, expand_depth=1):

	function: a function of the same parameters as the qnode
	A function that has same inputs as the given qnode and returns a resource estimation dictionary.


		info = super().specs

		if any(m.return_type is qml.operation.State for m in self.measurements):

specs transform #1245

specs transform #1245

Conversation

albi3ro commented Apr 21, 2021

github-actions bot commented Apr 21, 2021

codecov bot commented Apr 21, 2021 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chaserileyroberts commented Apr 21, 2021 • edited Loading

glassnotes commented Apr 21, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mariaschuld Jun 10, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mariaschuld left a comment

Choose a reason for hiding this comment

trbromley left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trbromley left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antalszava left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trbromley left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Apr 21, 2021 •

edited

Loading

chaserileyroberts commented Apr 21, 2021 •

edited

Loading

mariaschuld Jun 10, 2021 •

edited

Loading