Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add better error messages for typical SamplerV2 and EstimatorV2 error cases #12031

Merged
merged 5 commits into from
Mar 19, 2024

Conversation

t-imamichi
Copy link
Member

@t-imamichi t-imamichi commented Mar 18, 2024

Summary

If users want to run a circuit with SamplerV2, they need to call SamplerV2.run([circuit]) according to the spec.

def run(
self, pubs: Iterable[SamplerPubLike], *, shots: int | None = None

But, they may miss [] but the corresponding error message was not comprehensive as follows. This PR detects such a case and shows more comprehensive message.

from qiskit import QuantumCircuit
from qiskit.circuit.library import RealAmplitudes
from qiskit.primitives import StatevectorSampler

qc = QuantumCircuit(1, 1)
qc.h(0)
qc.measure(0, 0)

sampler = StatevectorSampler()
result = sampler.run([qc]).result()
# OK

result = sampler.run(qc).result()
# NG

output
main branch:

AttributeError: '_SingletonHGate' object has no attribute 'parameters'

this PR:

ValueError: An invalid Sampler pub-like was given (<class 'qiskit._accelerate.quantum_circuit.CircuitInstruction'>). If you want to run a single circuit, you need to wrap it with [] like sampler.run([circuit]) instead of sampler.run(circuit).


I also added a similar error message for Estimator V2 as suggested by @ElePT.

from qiskit import QuantumCircuit
from qiskit.circuit.library import RealAmplitudes
from qiskit.primitives import StatevectorEstimator

qc = QuantumCircuit(1, 1)
qc.h(0)
qc.h(0)
qc.h(0)

estimator = StatevectorEstimator()
result = estimator.run([(qc, "Z")]).result()
# OK

result = estimator.run((qc, "Z")).result()
# NG

output

main branch

TypeError: Invalid observable type: <class 'qiskit._accelerate.quantum_circuit.CircuitInstruction'>

this PR

ValueError: An invalid Estimator pub-like was given (<class 'qiskit.circuit.quantumcircuit.QuantumCircuit'>). If you want to run a single pub-like, you need to wrap it with [] like estimator.run([(circuit, observables, param_values)]) instead of estimator.run((circuit, observables, param_values)).

Details and comments

In addition to the error messages, I fixed random seed for primitives tests (replaced np.random.rand with np.random.default_rng).

@t-imamichi t-imamichi added stable backport potential The bug might be minimal and/or import enough to be port to stable mod: primitives Related to the Primitives module labels Mar 18, 2024
@t-imamichi t-imamichi requested review from a team as code owners March 18, 2024 05:46
@qiskit-bot
Copy link
Collaborator

One or more of the the following people are requested to review this:

  • @Qiskit/terra-core
  • @ajavadia
  • @ikkoham
  • @levbishop
  • @t-imamichi

@coveralls
Copy link

coveralls commented Mar 18, 2024

Pull Request Test Coverage Report for Build 8344800410

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 10 of 10 (100.0%) changed or added relevant lines in 2 files are covered.
  • 5 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.3%) to 89.319%

Files with Coverage Reduction New Missed Lines %
crates/qasm2/src/lex.rs 5 91.94%
Totals Coverage Status
Change from base Build 8320404144: -0.3%
Covered Lines: 59811
Relevant Lines: 66963

💛 - Coveralls

Copy link
Contributor

@ElePT ElePT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this PR, I think it makes a lot of sense. I added a suggestion to make the message even more clear (specifying single circuit). Would it make sense to extend this to the Estimator too? I am not sure if there is also a case with a single circuit and observable where this type of issue would happen.

qiskit/primitives/containers/sampler_pub.py Outdated Show resolved Hide resolved
Co-authored-by: Elena Peña Tapia <57907331+ElePT@users.noreply.github.com>
@t-imamichi t-imamichi changed the title Add a better error message for a SamplerV2 error case Add a better error message for a typical SamplerV2 and EstimatorV2 error cases Mar 18, 2024
@t-imamichi t-imamichi added the Changelog: None Do not include in changelog label Mar 18, 2024
@t-imamichi
Copy link
Member Author

I made the following changes since the last review comment.

@t-imamichi t-imamichi changed the title Add a better error message for a typical SamplerV2 and EstimatorV2 error cases Add better error messages for typical SamplerV2 and EstimatorV2 error cases Mar 18, 2024
Copy link
Contributor

@ElePT ElePT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot! I agree that exposing the pub could make the message too long, so using the type is a good call. It might still be a bit unexpected for users to see a report of <class 'qiskit._accelerate.quantum_circuit.CircuitInstruction'> (for example) as their input, but I think that overall the message is much much clearer and hopefully the suggestion about adding [ ] covers a lot of these initial user errors. Mostly LGTM (I left a tiny tiny comment). The changes to the test seeds/rng also look good.

qiskit/primitives/containers/estimator_pub.py Outdated Show resolved Hide resolved
Copy link
Contributor

@ElePT ElePT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, one more question, would a single sampler pub input like sampler.run((circuit,parameters)) run successfully? or would it also have to be wrapped in [] like the estimator one (sampler.run([(circuit,parameters)]))? Asking in case this also needs to be covered by the error messages.

@t-imamichi
Copy link
Member Author

t-imamichi commented Mar 18, 2024

sampler.run((circuit,parameters)) isn't correct. It raises an error ValueError: The number of values (0) does not match the number of parameters (X) for the circuit. (X is the actual number of parameters). In this case, the first item is a valid pub-like ((circuit,parameters) is an iterable and the first object is QuantumCircuit, which is a pub-like), but it is missing parameter values. So, the error message covers the case.
This error message is also raised in the case of sampler.run((circuit1, circuit2)) where circuit1 has some parameters.

Co-authored-by: Elena Peña Tapia <57907331+ElePT@users.noreply.github.com>
@ElePT
Copy link
Contributor

ElePT commented Mar 18, 2024

sampler.run((circuit,parameters)) isn't correct. It raises an error ValueError: The number of values (0) does not match the number of parameters (X) for the circuit. (X is the actual number of parameters). In this case, the first item is a valid pub-like ((circuit,parameters) is an iterable and the first object is QuantumCircuit, which is a pub-like), but it is missing parameter values. So, the error message covers the case.
This error message is also raised in the case of sampler.run((circuit1, circuit2)) where circuit1 has some parameters.

I see... it's difficult to tell apart both cases to have a specific message for each case. What do you think if we also add a second part to the error message in SamplerPub.validate so that it says the following??:

  raise ValueError(
      f"The number of values ({num_parameters}) does not match "
      f"the number of parameters ({self.circuit.num_parameters}) for the circuit. "
      f"Note that if you want to run a single pub, you need to wrap it with `[]` like "
      f"`sampler.run([(circuit, param_values)])` instead of `sampler.run((circuit, param_values))`."
  )

@t-imamichi
Copy link
Member Author

Thank you for your suggestion. I added your additional message if the number of parameters do not match and num_parameters == 0.

Copy link
Contributor

@ElePT ElePT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for your changes, I think this looks very good now. Approving!

@ElePT ElePT enabled auto-merge March 19, 2024 15:00
@ElePT ElePT added this pull request to the merge queue Mar 19, 2024
Merged via the queue into Qiskit:main with commit 194cc8f Mar 19, 2024
12 checks passed
mergify bot pushed a commit that referenced this pull request Mar 19, 2024
… cases (#12031)

* add an early error for a case

* Update qiskit/primitives/containers/sampler_pub.py

Co-authored-by: Elena Peña Tapia <57907331+ElePT@users.noreply.github.com>

* add better message for estimator pub

* Update qiskit/primitives/containers/estimator_pub.py

Co-authored-by: Elena Peña Tapia <57907331+ElePT@users.noreply.github.com>

* add an additional message

---------

Co-authored-by: Elena Peña Tapia <57907331+ElePT@users.noreply.github.com>
(cherry picked from commit 194cc8f)

# Conflicts:
#	test/python/primitives/test_backend_estimator_v2.py
#	test/python/primitives/test_backend_sampler_v2.py
@t-imamichi t-imamichi deleted the sampler-pub-validation branch March 20, 2024 09:25
github-merge-queue bot pushed a commit that referenced this pull request Mar 20, 2024
… cases (backport #12031) (#12041)

* Add better error messages for typical SamplerV2 and EstimatorV2 error cases (#12031)

* add an early error for a case

* Update qiskit/primitives/containers/sampler_pub.py

Co-authored-by: Elena Peña Tapia <57907331+ElePT@users.noreply.github.com>

* add better message for estimator pub

* Update qiskit/primitives/containers/estimator_pub.py

Co-authored-by: Elena Peña Tapia <57907331+ElePT@users.noreply.github.com>

* add an additional message

---------

Co-authored-by: Elena Peña Tapia <57907331+ElePT@users.noreply.github.com>
(cherry picked from commit 194cc8f)

# Conflicts:
#	test/python/primitives/test_backend_estimator_v2.py
#	test/python/primitives/test_backend_sampler_v2.py

* remove tests for backend primitives V2 because they will appear for 1.1

---------

Co-authored-by: Takashi Imamichi <31178928+t-imamichi@users.noreply.github.com>
Co-authored-by: Takashi Imamichi <imamichi@jp.ibm.com>
@jakelishman jakelishman modified the milestones: 1.1.0, 1.0.3 Apr 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Changelog: None Do not include in changelog mod: primitives Related to the Primitives module stable backport potential The bug might be minimal and/or import enough to be port to stable
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants