Feature germsel enhancements #270
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 30 commits
May 29, 2022 23:07
This commit implements a rudimentary form of caching for the effective states and preps to avoid having to repeatedly calculate the transfer matrix for the fiducial circuits.
…e available fiducials This commit adds basic caching support to avoid re-calculating the PTMs for the available fiducial list when generating the prep cache and measurement cache, which typically use the same set of available circuits and as such the same PTMs. Also add a function for removing effective identities and circuits with duplicate PTMs from the available fiducial list to decrease the search space.
Updated the scoring routine to inline the calculation to enable avoiding duplicate work. Modest performance boost.
Updated caching to avoid calling to_vector of POVMs and state preps for key generation too often which was proving expensive.
This commit is a one-line change of how the number of non-zero eigenvalues is counted which works out to be a few times faster than the old way I implemented.
Biggest changes are to the deduping routine which is now more efficient because we now winnow the list of circuits to search through for duplicates on the fly, reducing the number of pairwise checks along the way, big speedboost from this. Also switched to using the frobenious norm to identify duplicate PTMs as this was much faster in practice than np.allclose. Also updates the cache to use the string reps of circuits as keys instead of the circuits themselves to reduce the cost of repeatedly hashing circuits which can be more expensive than straight strings, modest performance enhancement from this
Add a basic check through the list of candidate germs to remove any with duplicate PTMs. Modest reduction in the size of the search space/performance.
…formance gains Adds support for user specified numpy floating point data types. This advanced option can enable a factor of 2-4 savings on the memory footprint of germ selection. Also exposes at the top level the ability to switch between a fully cached and on-the-fly calculation of the Jacobians for each germ.
1. Hybrid caching feature built on using the compact SVD representation of each of the Jacobians of each of the germs. 2. low-rank update based optimization loop using a hybrid approach based on results from M. Brand and N. Minamide. 3. Force-rank-update mode which allows you to force a short circuit of an update calculation if it doesn't increase the rank of the overall jacobian. (Still work in progress) 4. Functionality for dropping a user specified random fraction of candidate germs. 5. Minor change to the projection onto the kite code which uses a faster kronecker product implementation.
Adds progress logging to the cache generation stage and also modifies the way the inner loop progression is reported (by changing the progress logging level context).
The gauge parameter calculation is kind of slow so print out the result of this so we can re-use it later to circumvent the need to calculate this again.
One more minor printing change.
Update the code to allow circumventing the calculation of the number of gauge parameters entirely when both the number of gauge and non gauge parameters are manually specified.
I was originally returning both the components of the rank decomposition and the eigenvalues for the cache, but I never actually used the eigenvalues so I am dropping them from this structure. If I think of an application where I need to know what these are I can always reverse these changes very easily.
…disk This adds support for saving and loading compact EVD caches to disk to accelerate future calls to the germ selection algorithm. In the future I'll look into a more robust solution that can, for example, allow for incomplete coverage of the current candidate germ list etc...
When the SVD calculation fails to converge using numpy and it's default lapack driver of gesdd, fall back to scipy's implementation using the non-default gesvd driver option. Supposedly gesvd is more stable at the expense of being a bit slower.
…d toggle on and off prep and measurement selection Add option to return the deduped candidate list as well as the ability to manually specify the candidate list as a kwarg to avoid redoing the dedupe/ckt gen procedure. Also adds toggles to enable the generation of preps and measurements separately.
Adds an additional penalty term to the objective function where in a penalty is added to the score for each instance of a particular gate. This is passed in by the user in the form of a dictionary where the keys are gates and the values are the penalty weights.
Add a fallback behavior that tries instead to use a more stable implementation of the pinv based on scipy's SVD using a different lapack driver.
minor typo fix
…acobians onto the commutant Updates the implementation of the construction of the super-duper operator for projecting Jacobians onto the commutant to avoid doing double work when forming the projector onto degenenerate eigenspaces. Modest performance improvement for 2-qubits. Should return an effectively identical twirling super-duper operator otherwise.
One of the Cholesky decompositions is failing for mysterious reasons. Catch those errors and print out some diagnostic info.
At least one of the Cholesky decompositions was failing because a matrix was not PD, catch this, print some diagnostics, and fall back to calculation that doesn't use Cholesky (this was put in place for performance and is not strictly necessary).
Adds additional handles for setting and piping in a use specified value of the minimum eigenvalue threshold tolerance. I think the Cholesky decompositions we were seeing fail were effectively due to the Jacobian whose psuedoinverse we were updating being poorly conditioned due to an overly generous (i.e. too small) minimum eigenvalue tolerance. This was technically a variable that could be set but didn't have a way to do so at the top level and the hardcoded value resulted in issues.
This updates the final germ set score calculation to use a more memory efficient construction of the Jacobian.
…ure-germsel-enhancements
Adds unit tests to the germ selection test package to ensure the algorithm runs through to completion. Currently doesn't check for a particular set to be returned (none of the germ selection test do at the moment though).
added 4 commits
November 7, 2022 19:29
fix linting issue
Add additional unit test for low-rank update germ sel
enielse
approved these changes
Nov 11, 2022
Removes the argument num_gauge_params from find_germs and child functions since this is extraneous if we are already specifying the number of non-gauge parameters.
enielse
approved these changes
Nov 11, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
First, and primary, PR for a bunch of branches that revamp the germ selection code. This update adds a number of additional performance enhancements:
Because I practiced terrible branch hygiene (but also in part because some changes to the germ sel code were in service of other algorithms), there are a number of additional updates in the pipeline, specifically on the branches 'feature-fpr-algorithm-performance' and 'feature-fidsel-performance'. This includes better docstrings, logging/exception messages and the removal of a bunch of debugging print statements.