Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature faster testing stuff #403

Merged
merged 19 commits into from
Mar 25, 2024
Merged

Feature faster testing stuff #403

merged 19 commits into from
Mar 25, 2024

Conversation

coreyostrove
Copy link
Contributor

@coreyostrove coreyostrove commented Feb 23, 2024

This PR contains primarily a set of updates for the unit tests and for the tutorial/example notebooks aimed at reducing their runtime in the CI setting. This also includes some changes modernizing tests/notebooks moving away from use of legacy modelpacks. See issue #380 for a checklist summarizing the notebooks and tests which were identified as bottlenecks which have been addressed. In all cases the changes made are ones that reduce computational requirements without meaningfully impacting either the test coverage, or the message conveyance of a tutorial. E.g. switching from the XYI to the XY gate set in an example to reduce parameter counts (making stuff run more quickly) while still demoing the intended feature/analysis. With all of these changes in place the notebook regression checks now run in almost exactly half the time (at least from spot checking a couple ubuntu runners comparing two-weeks ago to today).

In the mix are also a couple of minor bugfixes/codebase changes. These are:

  • Set the default maximum iteration count for the optimizer to 100 when performing GST using the driver methods. Down from 100000 (essentially unlimited). This brings the default behavior inline with the defaults when using the OO interface using Protocol objects.
  • Some modernizations and tweaks to the qutrit model building helpers to conform with new line labeling behavior, and a switch to using TP constrained modelmembers by default.
  • Bugfix for StandardGST that ensures we make a copy of the target model rather than optimizing it in-place. I swear we had already done this fix well over a year ago, but perhaps this was inadvertently overwritten at some point.

Corey Ostrove added 15 commits February 20, 2024 15:30
Initial round of updates to speed up unit tests and modernize them.
Update unit tests for time dependent GST to speed those up.
Switch from loading old models and datasets from disk to spinning these up from scratch.
I swear we had changed this to have the model copied, but for some reason that ended up reverted...
The default max iteration count for the driver based calls was different than the default behavior for the OO approach. This update brings those two in line.
This commit updates the QutritGST demo to run significantly faster for automated testing purposes. Also includes some changes to the qutrit model building helper utilities to modernize those by switching default parameterization to full TP, from unconstrained full. (This also has the benefit of reducing parameter counts which speeds up runtime).
Updates the tutorial notebook for the drift analysis code to run on a smaller experiment design and thus take a lot less time to run.
More notebook updates aimed at speeding up testing. Main changes here are to switch from XYI to XY in a few places where the gate set isn't particularly important, and to decrease the maximum circuit depth for GST experiment designs in some places.
Minor fix to a unit test to account for changes to qutrit construction code.
Streamline this notebook to speed up execution.
Modernize it to not use a legacy modelpack and use a slightly smaller model to run a bit faster.
Reduce from 3 to two-qubits, and reduce maximum depth a bit.
@coreyostrove coreyostrove added this to the 0.9.13 milestone Feb 23, 2024
@coreyostrove coreyostrove self-assigned this Feb 23, 2024
Corey Ostrove added 3 commits February 22, 2024 18:36
Needed to update dof counting to account for smaller edesign
Another minor typo fix...
@coreyostrove coreyostrove marked this pull request as ready for review February 23, 2024 03:24
Copy link
Contributor

@sserita sserita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for all your hard work on optimizing these tests Corey. I feel pretty good about our runtime on the workers now.

@sserita sserita merged commit f53e0a4 into develop Mar 25, 2024
4 checks passed
@sserita sserita deleted the feature-faster-testing-stuff branch March 25, 2024 20:24
@sserita sserita mentioned this pull request Mar 25, 2024
86 tasks
@sserita sserita modified the milestones: 0.9.13, 0.9.12.2 Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants