Add BackendEstimatorV2 #11931

t-imamichi · 2024-03-01T14:26:18Z

Summary

This PR adds BackendEstimatorV2 as a BaseEstimatorV2 implementation using a backend.
This is another approach to #11899.
This PR provides an implementation that directly relies on backend.run while #11899 provides a converter from EstimatorV1 to EstimatorV2.
This PR can do qubit-wise grouping taking into account broadcasting, e.g., estimator.run([(circuit, ["IX", "YI"])]) -> run 1 circuit.
I made tests based on those for StatevectorEstimator.

Notes (same as #11928)

I intentionally specified only BackendV2 as backend option though it still supports BackendV1 because I think we need to support only the latest version of backends. If requested, I can add BackendV1.
- I added BackendV1 support as well as BackendV2
I didn't add transpile feature because Runtime Primitives V2 do not support transpile during execution. Users need to transform input circuits into ISA format beforehand.

Details and comments

qiskit-bot · 2024-03-01T14:26:22Z

One or more of the the following people are requested to review this:

@Qiskit/terra-core
@ajavadia
@ikkoham
@levbishop
@t-imamichi

qiskit/primitives/backend_estimator.py

coveralls · 2024-03-01T14:43:32Z

Pull Request Test Coverage Report for Build 8293552364

Details

139 of 142 (97.89%) changed or added relevant lines in 3 files are covered.
18 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.003%) to 89.282%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
qiskit/primitives/backend_estimator_v2.py	135	138	97.83%

Files with Coverage Reduction	New Missed Lines	%
crates/qasm2/src/lex.rs	6	91.94%
crates/qasm2/src/parse.rs	12	97.15%

Totals
Change from base Build 8285398834:	-0.003%
Covered Lines:	59709
Relevant Lines:	66877

💛 - Coveralls

t-imamichi · 2024-03-02T06:43:35Z

test/python/primitives/test_estimator.py

+        # calculate [ <psi2(theta2)|H1|psi2(theta2)> ]
        result2 = estimator.run([psi2], [hamiltonian1], [theta2]).result()


H2 should be H1 because the test uses hamiltonian1. Same for other estimator tests.

t-imamichi · 2024-03-02T07:07:11Z

qiskit/primitives/backend_estimator_v2.py

+        basis = PassManagerConfig.from_backend(backend).basis_gates
+        self._passmanager = PassManager(
+            [Optimize1qGatesDecomposition(basis=basis, target=backend.target)]


This is a workaround to support both BasicSimulator and AerSimulator when translating measurement circuits to their basis gates.

If I don't set basis, some tests for BasicSimulator fail as follows.

ERROR: test_run_numpy_params_1 (test.python.primitives.test_backend_estimator_v2.TestBackendEstimatorV2)
test.python.primitives.test_backend_estimator_v2.TestBackendEstimatorV2.test_run_numpy_params_1
...
qiskit.providers.basic_provider.exceptions.BasicProviderError: 'basic_simulator encountered unrecognized operation "sdg"'

On the other hand, if I use generate_preset_passmanager instead of Optimize1qGatesDecomposition, it causes an error with AerSimulator as follows.

ERROR: test_aer_1 (test.python.primitives.test_backend_estimator_v2.TestBackendEstimatorV2)
test.python.primitives.test_backend_estimator_v2.TestBackendEstimatorV2.test_aer_1
...
qiskit.transpiler.exceptions.TranspilerError: "Unable to translate the operations in the circuit: ['sdg', 'h', 'measure'] to the backend's (or manually specified) target basis: ['barrier', 'measure', 'h', 'snapshot']. This likely means the target basis is not universal or there are additional equivalence rules needed in the EquivalenceLibrary being used. For more details on this error see: https://docs.quantum.ibm.com/api/qiskit/transpiler_passes.BasisTranslator#translation-errors"

t-imamichi · 2024-03-02T07:09:44Z

qiskit/primitives/backend_estimator_v2.py

+        return PrimitiveResult([self._run_pub(pub) for pub in pubs])
+
+    def _run_pub(self, pub: EstimatorPub) -> PubResult:
+        shots = int(np.ceil(self._VARIANCE_UPPER_BOUND / pub.precision**2))


Since I'm not sure the details of precison, this shots estimation is quite rough one. Could you tell me how can we determine the number of shots depending on precision, @ihincks and @chriseclectic?

FWIW, I find confusing that the default precision parameter is 0 but precision should be larger than 0.

I will update it with the same default value with qiskit-ibm-runtime.
https://github.com/Qiskit/qiskit-ibm-runtime/blob/50797a909f376a5ab051b4cefd787ced7a2160e0/qiskit_ibm_runtime/options/estimator_options.py#L44

I left a different comment about the default value before reading this. Please go ahead and set it to any reasonable >0 value you like.

how can we determine the number of shots

What you've written looks like a good first stab, and essentially the same as the one in the IBM runtime at the moment. For this reason I think it's okay as-is.

If we wanted to make a better guess we could do this: suppose a sparse Pauli sum observable A=\sum_i a_i P_i. Then Var[A]=\sum_i |a_i|^2 Var[P_i]. If we take the worst case that each P_i has expectation value 0, then each Var[P_i] will be about 1, so that Var[A]=\sum_i |a_i|^2. So a conservative guess would be to go through every observable in the array, find the one with the worst such variance, and use that to decide the shots.

Thank you for the details. I copied the API doc of qiskit-ibm-runtime and simplified the code by removing _VARIANCE_UPPER_BOUND.

1ucian0 · 2024-03-11T11:43:05Z

The following example with Aer:

import numpy as np
from qiskit.circuit.library import IQP
from qiskit import transpile
from qiskit.quantum_info import SparsePauliOp, random_hermitian
from qiskit.primitives import BackendEstimatorV2
from qiskit_aer import Aer

qasm_simulator = Aer.get_backend('qasm_simulator')
qasm_simulator_estimator = BackendEstimatorV2(backend=qasm_simulator)

n_qubits = 12
 
mat = np.real(random_hermitian(n_qubits, seed=1234))
circuit = IQP(mat)
observable = SparsePauliOp("Z" * n_qubits)
 
isa_circuit = transpile(circuit, backend=qasm_simulator, optimization_level=1)
isa_observable = observable.apply_layout(isa_circuit.layout) 
 
job = qasm_simulator_estimator.run([(isa_circuit, isa_observable)], precision=0.01)
result = job.result()
 
print(f" > Expectation value: {result[0].data.evs}")
print(f" > Metadata: {result[0].metadata}")

 > Expectation value: -0.00385
 > Metadata: {'precision': 0.01}

1ucian0 · 2024-03-11T11:51:30Z

Here an example with FakeBackendV2:

import numpy as np
from qiskit.circuit.library import IQP
from qiskit import transpile
from qiskit.quantum_info import SparsePauliOp, random_hermitian
from qiskit.primitives import BackendEstimatorV2
from qiskit_ibm_runtime.fake_provider import FakeAlmadenV2 

fake_almaden = FakeAlmadenV2()
fake_almaden_estimator = BackendEstimatorV2(backend=fake_almaden)

n_qubits = 12
 
mat = np.real(random_hermitian(n_qubits, seed=1234))
circuit = IQP(mat)
observable = SparsePauliOp("Z" * n_qubits)
 
isa_circuit = transpile(circuit, backend=fake_almaden, optimization_level=1)
isa_observable = observable.apply_layout(isa_circuit.layout) 
 
job = fake_almaden_estimator.run([(isa_circuit, isa_observable)], precision=0.01)
result = job.result()
 
print(f" > Expectation value: {result[0].data.evs}")
print(f" > Metadata: {result[0].metadata}")

 > Expectation value: -0.00575
 > Metadata: {'precision': 0.01}

ihincks

Thank-you @t-imamichi this looks very good! I don't have any major comments.

qiskit/primitives/__init__.py

qiskit/primitives/backend_estimator_v2.py

ihincks · 2024-03-12T10:37:03Z

qiskit/primitives/backend_estimator_v2.py

+        return PrimitiveResult([self._run_pub(pub) for pub in pubs])
+
+    def _run_pub(self, pub: EstimatorPub) -> PubResult:
+        shots = int(np.ceil(self._VARIANCE_UPPER_BOUND / pub.precision**2))


I left a different comment about the default value before reading this. Please go ahead and set it to any reasonable >0 value you like.

ihincks · 2024-03-12T10:44:21Z

qiskit/primitives/backend_estimator_v2.py

+        return PrimitiveResult([self._run_pub(pub) for pub in pubs])
+
+    def _run_pub(self, pub: EstimatorPub) -> PubResult:
+        shots = int(np.ceil(self._VARIANCE_UPPER_BOUND / pub.precision**2))


how can we determine the number of shots

What you've written looks like a good first stab, and essentially the same as the one in the IBM runtime at the moment. For this reason I think it's okay as-is.

If we wanted to make a better guess we could do this: suppose a sparse Pauli sum observable A=\sum_i a_i P_i. Then Var[A]=\sum_i |a_i|^2 Var[P_i]. If we take the worst case that each P_i has expectation value 0, then each Var[P_i] will be about 1, so that Var[A]=\sum_i |a_i|^2. So a conservative guess would be to go through every observable in the array, find the one with the worst such variance, and use that to decide the shots.

qiskit/primitives/backend_estimator_v2.py

Co-authored-by: Ian Hincks <ian.hincks@gmail.com>

t-imamichi · 2024-03-12T14:18:16Z

qiskit/primitives/backend_estimator_v2.py

+        self,
+        *,
+        backend: BackendV1 | BackendV2,
+        default_precision: float = 0.015625,


I updated it with the same value with qiskit-ibm-runtime
https://github.com/Qiskit/qiskit-ibm-runtime/blob/16e90f475e78a9d2ae77daa139ef750cfa84ca82/qiskit_ibm_runtime/options/estimator_options.py#L44

t-imamichi · 2024-03-12T14:18:35Z

qiskit/primitives/backend_estimator_v2.py

+        Args:
+            backend: The backend to run the primitive on.
+            default_precision: The default precision to use if none are specified in :meth:`~run`.
+                Default: 0.015625 (1 / sqrt(4096)).


I added Default copied from qiskit-ibm-runtime
https://github.com/Qiskit/qiskit-ibm-runtime/blob/16e90f475e78a9d2ae77daa139ef750cfa84ca82/qiskit_ibm_runtime/options/estimator_options.py#L44

t-imamichi · 2024-03-12T14:22:34Z

qiskit/primitives/backend_estimator_v2.py

+        return PrimitiveResult([self._run_pub(pub) for pub in pubs])
+
+    def _run_pub(self, pub: EstimatorPub) -> PubResult:
+        shots = int(np.ceil(1.0 / pub.precision**2))


I removed _VARIANCE_UPPER_BOUND and simplified this part since the numerator is 1.

t-imamichi · 2024-03-12T14:24:25Z

I added BackendV1 support and tests with Fake7QPulseV1.

t-imamichi · 2024-03-12T14:29:02Z

test/python/primitives/test_backend_estimator_v2.py

+from qiskit.transpiler.preset_passmanagers import generate_preset_pass_manager
+from qiskit.utils import optionals
+
+BACKENDS = [BasicSimulator(), Fake7QPulseV1(), BackendV2Converter(Fake7QPulseV1())]


It would be great if we could set random seed number via constructor of these simulators. I currently set a conservative target precision value. But it may fail in the worst case.

t-imamichi · 2024-03-13T10:58:00Z

I'm wondering how to add seed.

add seed parameter to the constructor, i.e., BackendEstimator(self, *, backend, default_precision, seed).

this approach looks similar to StatevectorEstimator.

qiskit/qiskit/primitives/statevector_estimator.py

Line 106 in 9f228fa

self, *, default_precision: float = 0.0, seed: np.random.Generator | int | None = None

add seed parameter to run method, i.e., BackendEstimator.run(self, pubs, *, default_precision, seed).
We will pass seed to a simulator via _run_circuit. Then, we can set a random seed number though some simulator behavior might be deterministic.

t-imamichi · 2024-03-13T13:30:27Z

I made a PoC to add seed_simulator option torun method and constructor. I confirmed that it fixes deviation of expectation values of the unit tests.
If it looks good, I will merge the branch to this PR and make the same stuff for BackendSamplerV2.

run method
t-imamichi/qiskit-terra@backend-estimator-v2...t-imamichi:qiskit-terra:backend-estimator-v2-seed

constructor
t-imamichi/qiskit-terra@backend-estimator-v2...t-imamichi:qiskit-terra:backend-estimator-v2-seed2

jyu00 · 2024-03-13T20:40:12Z

I made a PoC to add seed_simulator option torun method and constructor.

@t-imamichi We had said in the options RFC that we didn't want to pass options to run() (except those that are required, i.e. shots and precision) anymore. The primitive should instead have an options attribute, preferably a dataclass, e.g.

estimator = BackendEstimatorV2(...)
estimator.options.seed_simulator = 42

This would also be more consistent with what qiskit-ibm-runtime has.

t-imamichi · 2024-03-14T08:02:32Z

Thank you for your suggestion. I made an option based on dataclass. Could you take a look at it?

jyu00 · 2024-03-14T13:45:31Z

@t-imamichi thanks for the update! I'd prefer having default_precision in options as well, since unlike precision, default_precision is not defined in the base interface. I think it's nicer to have all options not defined in the base to live together in options.

1ucian0 · 2024-03-14T14:04:40Z

test/python/primitives/test_backend_estimator_v2.py

+        # calculate [ <psi2(theta2)|H1|psi2(theta2)> ]
+        ham1 = hamiltonian1.apply_layout(psi2.layout)
+        result2 = estimator.run([(psi2, ham1, theta2)]).result()
+        np.testing.assert_allclose(result2[0].data.evs, [2.97797666], rtol=self._rtol)


now you can seed the execution, do you still need assert_allclose?

Yes. It is necessary because unit tests use both an exact simulator and noise simulators.

releasenotes/notes/add-backend-estimator-v2-26cf14a3612bb81a.yaml

1ucian0 · 2024-03-14T20:33:10Z

Some non-blocking comments. It they can be addressed in this PR, great. Otherwise, in a follow up. The comment in #11931 (comment) seems to be the only blocking issue on this PR.

t-imamichi · 2024-03-15T08:52:35Z

I moved default_precision to Options and update reno to include an example.

I have a question. I used dataclass to define Options instead of using qiskit.providers.Options to allow autocompletion and expect a validation of a wrong option name. If qiskit.providers.Options is preferable, I would update the options.

1ucian0 · 2024-03-15T12:22:08Z

On my side, LGTM and this can be merged.

I have a question. I used dataclass to define Options instead of using qiskit.providers.Options to allow autocompletion and expect a validation of a wrong option name. If qiskit.providers.Options is preferable, I would update the options.

I don't think it matters. @jyu00 ?

jyu00 · 2024-03-15T12:44:52Z

I have a question. I used dataclass to define Options instead of using qiskit.providers.Options to allow autocompletion and expect a validation of a wrong option name. If qiskit.providers.Options is preferable, I would update the options.

Dataclass is actually more preferable than qiskit.providers.Options, so this looks good to me!

t-imamichi added 2 commits March 1, 2024 23:14

add BackendEstimatorV2

a688bf4

reno

6edfce0

t-imamichi requested review from a team as code owners March 1, 2024 14:26

t-imamichi commented Mar 1, 2024

View reviewed changes

qiskit/primitives/backend_estimator.py Show resolved Hide resolved

t-imamichi added mod: primitives Related to the Primitives module Changelog: New Feature Include in the "Added" section of the changelog labels Mar 1, 2024

t-imamichi added 2 commits March 2, 2024 15:38

fix error for aer

49d1ebd

fix comments of tests

eae8fe8

t-imamichi commented Mar 2, 2024

View reviewed changes

revise aer test

27c920d

t-imamichi commented Mar 2, 2024

View reviewed changes

increase presion for test

e5c09a6

1ucian0 mentioned this pull request Mar 6, 2024

Epic - Implementation of RFC 0015: Extended Estimator Interface Qiskit/RFCs#69

Open

2 tasks

1ucian0 self-assigned this Mar 7, 2024

ihincks reviewed Mar 12, 2024

View reviewed changes

t-imamichi and others added 3 commits March 12, 2024 22:50

Apply suggestions from code review

abf87a5

Co-authored-by: Ian Hincks <ian.hincks@gmail.com>

allow BackendV1

a0f1628

Merge branch 'main' into backend-estimator-v2

8354449

t-imamichi commented Mar 12, 2024

View reviewed changes

add seed_simulator to constructor

571c079

introduce options

dd4d8a6

use dataclass for Options

aea3460

t-imamichi force-pushed the backend-estimator-v2 branch from 7d8d7c2 to aea3460 Compare March 14, 2024 08:20

1ucian0 reviewed Mar 14, 2024

View reviewed changes

releasenotes/notes/add-backend-estimator-v2-26cf14a3612bb81a.yaml Show resolved Hide resolved

t-imamichi added 3 commits March 15, 2024 15:34

Merge branch 'main' into backend-estimator-v2

28e86d5

move default_precision to options

c809af7

update docstring and reno

6aaa98d

t-imamichi mentioned this pull request Mar 15, 2024

Add BackendSamplerV2 #11928

Merged

jyu00 approved these changes Mar 15, 2024

View reviewed changes

1ucian0 approved these changes Mar 15, 2024

View reviewed changes

1ucian0 added this pull request to the merge queue Mar 15, 2024

Merged via the queue into Qiskit:main with commit 7370ed0 Mar 15, 2024
12 checks passed

t-imamichi deleted the backend-estimator-v2 branch March 15, 2024 13:54

1ucian0 mentioned this pull request Mar 19, 2024

Add EstimatorV2Converter and BackendEstimatorV2 #11899

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add BackendEstimatorV2 #11931

Add BackendEstimatorV2 #11931

t-imamichi commented Mar 1, 2024 •

edited

qiskit-bot commented Mar 1, 2024

coveralls commented Mar 1, 2024 •

edited

t-imamichi Mar 2, 2024 •

edited

t-imamichi Mar 2, 2024 •

edited

t-imamichi Mar 2, 2024

1ucian0 Mar 11, 2024

t-imamichi Mar 11, 2024

ihincks Mar 12, 2024

ihincks Mar 12, 2024

t-imamichi Mar 12, 2024

1ucian0 commented Mar 11, 2024 •

edited

1ucian0 commented Mar 11, 2024 •

edited

ihincks left a comment

ihincks Mar 12, 2024

ihincks Mar 12, 2024

t-imamichi Mar 12, 2024

t-imamichi Mar 12, 2024

t-imamichi Mar 12, 2024

t-imamichi commented Mar 12, 2024

t-imamichi Mar 12, 2024

t-imamichi commented Mar 13, 2024 •

edited

t-imamichi commented Mar 13, 2024 •

edited

jyu00 commented Mar 13, 2024

t-imamichi commented Mar 14, 2024

jyu00 commented Mar 14, 2024

1ucian0 Mar 14, 2024

t-imamichi Mar 15, 2024

1ucian0 commented Mar 14, 2024

t-imamichi commented Mar 15, 2024

1ucian0 commented Mar 15, 2024

jyu00 commented Mar 15, 2024

		# calculate [ <psi2(theta2)\|H1\|psi2(theta2)> ]
		result2 = estimator.run([psi2], [hamiltonian1], [theta2]).result()

Add BackendEstimatorV2 #11931

Add BackendEstimatorV2 #11931

Conversation

t-imamichi commented Mar 1, 2024 • edited

Summary

Details and comments

qiskit-bot commented Mar 1, 2024

coveralls commented Mar 1, 2024 • edited

Pull Request Test Coverage Report for Build 8293552364

Details

💛 - Coveralls

t-imamichi Mar 2, 2024 • edited

Choose a reason for hiding this comment

t-imamichi Mar 2, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

1ucian0 commented Mar 11, 2024 • edited

1ucian0 commented Mar 11, 2024 • edited

ihincks left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

t-imamichi commented Mar 12, 2024

Choose a reason for hiding this comment

t-imamichi commented Mar 13, 2024 • edited

t-imamichi commented Mar 13, 2024 • edited

jyu00 commented Mar 13, 2024

t-imamichi commented Mar 14, 2024

jyu00 commented Mar 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

1ucian0 commented Mar 14, 2024

t-imamichi commented Mar 15, 2024

1ucian0 commented Mar 15, 2024

jyu00 commented Mar 15, 2024

t-imamichi commented Mar 1, 2024 •

edited

coveralls commented Mar 1, 2024 •

edited

t-imamichi Mar 2, 2024 •

edited

t-imamichi Mar 2, 2024 •

edited

1ucian0 commented Mar 11, 2024 •

edited

1ucian0 commented Mar 11, 2024 •

edited

t-imamichi commented Mar 13, 2024 •

edited

t-imamichi commented Mar 13, 2024 •

edited