Add support for pulse reference to QPY

[nkanazawa1989]

Summary

This PR adds support for Reference pulse instruction to QPY. In addition, ScheduleBlock with assigned subroutine after round-trip keeps original reference data structure.

from qiskit import pulse, qpy
import io

with pulse.build() as sub:
    pulse.play(pulse.Constant(100, 0.1), pulse.DriveChannel(0))

with pulse.build() as schedule:
    pulse.call(sub, name="subroutine")  # this immediately assigns subroutine to the reference

# Do QPY roundtrip
with io.BytesIO() as container:
    qpy.dump(schedule, container)
    binary = container.getvalue()
with io.BytesIO(binary) as container:
    schedule_load = qpy.load(container)[0]

Original reference structure:

>>> schedule.references
ReferenceManager(references=[('subroutine',)])

Conventional behavior:

>>> schedule_load.references
ReferenceManager(references=[])

With this PR:

>>> schedule_load.references
ReferenceManager(references=[('subroutine',)])

This doesn't impact program execution (because reference doesn't appear in the wire format), but it may help saving a template program for calibration task to preserve the original context.

Close #9889

Details and comments

This PR avoids key conflict issue described in #9889 in bit fragile way. This introduces new reference key for string o that appears in the context of pulse instruction operands, rather than delegating serialization to the standard value encoding in QPY. Since current type key doesn't have namespace, it could cause further key conflicts in future as we add more data types. However, adding namespace requires major refactoring of the current QPY module, and I wanted avoid this effort because of the limited bandwidths.

[qiskit-bot]

Thank you for opening a new pull request.

Before your PR can be merged it will first need to pass continuous integration tests and be reviewed. Sometimes the review process can be slow, so please be patient.

While you're waiting, please feel free to review other open PRs. While only a subset of people are authorized to approve pull requests for merging, everyone is encouraged to review open pull requests. Doing reviews helps reduce the burden on the core team and helps make the project's code better for everyone.

One or more of the the following people are requested to review this:

• @Qiskit/terra-core
• @mtreinish
• @nkanazawa1989

[coveralls]

Pull Request Test Coverage Report for Build 4653901271

• 41 of 44 (93.18%) changed or added relevant lines in 3 files are covered.
• 610 unchanged lines in 70 files lost coverage.
• Overall coverage increased (+0.03%) to 85.438%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
qiskit/qpy/binary_io/schedules.py	34	35	97.14%
qiskit/qpy/type_keys.py	6	8	75.0%

Files with Coverage Reduction	New Missed Lines	%
qiskit/algorithms/eigen_solvers/eigen_solver.py	1	97.78%
qiskit/algorithms/phase_estimators/hamiltonian_phase_estimation_result.py	1	96.43%
qiskit/algorithms/time_evolvers/variational/solvers/ode/abstract_ode_function.py	1	93.33%
qiskit/primitives/backend_estimator.py	1	95.37%
qiskit/primitives/backend_sampler.py	1	97.73%
qiskit/qpy/binary_io/schedules.py	1	86.36%
qiskit/transpiler/passes/layout/full_ancilla_allocation.py	1	97.73%
qiskit/transpiler/passes/utils/check_map.py	1	97.78%
qiskit/algorithms/amplitude_amplifiers/amplification_problem.py	2	85.14%
qiskit/algorithms/aux_ops_evaluator.py	2	95.56%

Totals
Change from base Build 4576589370:	0.03%
Covered Lines:	67833
Relevant Lines:	79394

---

💛 - Coveralls

[mtreinish]

Overall the code LGTM, the approach for storing the references seems sensible. I just had a few comments in the documentation.

[mtreinish]

We probably should also document that the SCHEDULE_BLOCK_HEADER now has a num_references field in it too?

[nkanazawa1989]

Thanks for catching this. I forgot to remove SCHEDULE_BLOCK_HEADER_v7 (I was thinking different data model in the beginning). Since we already have mapping data model in QPY (key value must be string), I just reused it to express the reference mapping. Namely schedule block and references are stored back to back but they are different data section.

[mtreinish]

Heh, well it's not going to be so temporary if we push this out as version 7 we'll need to support o as a type key for string forever as we'll always need to be able to load version 7. But it's also not the end of the world from my perspective, we're just using letters because it's easier to reason about compared to 0x6F. So the fact that we're already using s in this context it doesn't seem to be such a big deal we have to use a different character for string here. I think we already have a conflict like this for something on the circuit side.

[nkanazawa1989]

Fair enough. I updated the comments.

[nkanazawa1989]

Suggested change

	common.write_mapping(
	file_obj=file_obj,
	mapping=flat_key_refdict,
	serializer=_dumps_reference_item,
	metadata_serializer=metadata_serializer,
	)
	common.write_mapping(
	file_obj=file_obj,
	mapping=flat_key_refdict,
	serializer=_dumps_reference_item,
	metadata_serializer=metadata_serializer,
	)

We have QPY function to write mapping object here.

[mtreinish]

This LGTM now, thanks for making the updates. The cleanups make it a bit more clear to me how the new format works. I was getting a bit confused by the new header type before that I didn't see used anywhere.

I left an inline comment about having an upgrade release note, but we can do that as part of the release notes roundup.

[mtreinish]

A note to myself (or anyone who is doing release notes) we should have an upgrade release note here about bumping to version 7.

[nkanazawa1989]

Another note. @mtreinish could you also replace this with

pulse.reference("cr45m", "q0", "q1")

cr45"p" -> cr45"m" when you cleanup the release note? This is just a typo but not critical. Usually ECR pulse sequence has negative rotation (subscripted with "m") on the second pulse.