Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete serialization overhaul #263

Merged
merged 99 commits into from
Jul 17, 2018
Merged

Complete serialization overhaul #263

merged 99 commits into from
Jul 17, 2018

Commits on May 15, 2018

  1. Configuration menu
    Copy the full SHA
    f8b4d2d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e340dee View commit details
    Browse the repository at this point in the history
  3. Update zip file backend

    terrorfisch committed May 15, 2018
    Configuration menu
    Copy the full SHA
    87f061a View commit details
    Browse the repository at this point in the history

Commits on May 23, 2018

  1. Configuration menu
    Copy the full SHA
    a82f25a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6eac5c8 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    cda7cc1 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    23576e6 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2018

  1. Added missing type annotations.

    lumip committed May 30, 2018
    Configuration menu
    Copy the full SHA
    fa33b51 View commit details
    Browse the repository at this point in the history
  2. Re-added old ExtendedJSONEncoder (and renamed new ones) to enable smo…

    …oth transition.
    
    New classes are now called JSONSerializableDecoder/-Encoder .
    lumip committed May 30, 2018
    Configuration menu
    Copy the full SHA
    492d58c View commit details
    Browse the repository at this point in the history
  3. Re-added old Serializer tests, made new serialization routines compat…

    …ible with old for transition period and marked old routines as deprecated.
    lumip committed May 30, 2018
    Configuration menu
    Copy the full SHA
    6c61170 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    10b81d0 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2018

  1. Adapted remaining PulseTemplate classes to new serialization routines.

    All now implement get_serialization_data and deserialize for old and new
    serialization routines for a transition period.
    lumip committed Jun 7, 2018
    Configuration menu
    Copy the full SHA
    901cbad View commit details
    Browse the repository at this point in the history
  2. Fixes for get_serialization_data routines and corresponding tests.

    Methods did not call Serializable::get_serialization_data() and thus did not inculde #type and #identifier keys in the returned dictionaries. Tests did not cover this.
    Moved identifier_name and type_identifier_name from JSONSerializableEncoder/-Decoder to Serializable.
    
    Signed-off-by: Lukas Prediger <lukas.prediger@rwth-aachen.de>
    lumip committed Jun 7, 2018
    Configuration menu
    Copy the full SHA
    6e42954 View commit details
    Browse the repository at this point in the history
  3. qctoolkit.serialization.PulseStorage: Fixes to __getitem__.

    Implementation was inconsistent with _temporary_storage usage in other places.
    lumip committed Jun 7, 2018
    Configuration menu
    Copy the full SHA
    65f568c View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2018

  1. Configuration menu
    Copy the full SHA
    01dab0d View commit details
    Browse the repository at this point in the history
  2. Changes to (Nested)DummySerializable.

    DummySerializable now behaves as follows: Takes any number of keyword-arguments and stores them as class attributes. get_serialization_data() returns a dictionary with (key, value) pairs for all attributes given this way. This is compatible with previous usage (where only a "data" attribute of type Any was given and get_serialization_data() returned a dictionary with only key "data" and whatever given value).
    Removed duplicated definition of DummySerializable further down the same file.
    NestedDummySerializable now is also compatible with new serialization
    routines and implements deserialize().
    lumip committed Jun 14, 2018
    Configuration menu
    Copy the full SHA
    014b4b8 View commit details
    Browse the repository at this point in the history
  3. Made SerializableTests an abstract base class.

    ..to adapt syntax/type to behavior.
    
    Slight change in DummySerializableTests.make_kwargs to extend testing of DummySerializable funcitonality a bit.
    lumip committed Jun 14, 2018
    Configuration menu
    Copy the full SHA
    5b8d6e2 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    ed93eae View commit details
    Browse the repository at this point in the history
  5. Added method to convert a single pulse from old to new serialization …

    …format.
    
    Pulse is read using old serialization routines from a source storage backend and destination storage backend to a new using the new routines.
    lumip committed Jun 14, 2018
    Configuration menu
    Copy the full SHA
    5afe318 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    a858158 View commit details
    Browse the repository at this point in the history
  7. Added list_contents() method to StorageBackend.

    Method should list all available serializables. No tests currently.
    lumip committed Jun 14, 2018
    Configuration menu
    Copy the full SHA
    cb79d11 View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2018

  1. Configuration menu
    Copy the full SHA
    17e0d1a View commit details
    Browse the repository at this point in the history

Commits on Jun 16, 2018

  1. Configuration menu
    Copy the full SHA
    95bbbd5 View commit details
    Browse the repository at this point in the history
  2. Added convert_pulses_in_storage method.

    convert_pulses_in_storage converts all pulses in a given source StorageBackend
    from the old serialization format into the new and stores them in a given
    destination StorageBackend.
    Both conversion methods now raise an exception if they would need to
    overwrite identifiers already existing in the destination backend.
    Commit also includes more extensive tests for both conversion methods as
    well as some style corretions for already existing tests.
    lumip committed Jun 16, 2018
    Configuration menu
    Copy the full SHA
    408429e View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    8a8e167 View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2018

  1. Configuration menu
    Copy the full SHA
    e3f334d View commit details
    Browse the repository at this point in the history
  2. PulseStorage can be conveniently replace default pulse registration.

    Added get_default_pulse_registration() method to query current default pulse registration.
    Added method PulseStorage.as_default_registry(), a context manager method that causes the PulseStorage object to replace the current default pulse registration for the duration of a "with:" block.
    Added method PulseStorage.set_to_default_registry() which permanently sets the PulseStorage object as the default pulse registration.
    lumip committed Jun 17, 2018
    Configuration menu
    Copy the full SHA
    789309a View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2018

  1. All serialization tests now subclasses of tests.serialization_tests.S…

    …erializableTests.
    
    Adapted tests.pulses.sequencing_dummies.DummyPulseTemplate to have sensible serialization semantics so it can by higher order PTs during tests.
    lumip committed Jun 21, 2018
    Configuration menu
    Copy the full SHA
    13e652c View commit details
    Browse the repository at this point in the history
  2. PulseStorage now produces ordered and beautified JSON.

    Closes #270. However, test_flush_destroy_object (testing that PulseStorage flushes on destruction) inexplicably fails now. Probably some timing issues with garbage collection. Refer to issue #281 .
    lumip committed Jun 21, 2018
    Configuration menu
    Copy the full SHA
    6ab3563 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2018

  1. PulseStorage now immediately writes new pulses to storage backend.

    This makes method flush() obsolete, so it was removed.
    lumip committed Jun 28, 2018
    Configuration menu
    Copy the full SHA
    7316cec View commit details
    Browse the repository at this point in the history
  2. Safer dealing with duplicate ids when storing pulses.

    JSONEncoder previously did not check whether subpulses are identical to already known pulses with the same id but just replaced them with a reference to the already known subpulse in the serialization. This could lead to unexpected behavior. JSONEncoder now raises a RuntimeError instead in this case.
    PulseStorage adopts a transaction model and takes care that in case of such an exception, a pulse is not partially serialized, i.e., if a pulse serialization aborts due to a duplicated subpulse identifier, no changes to the storage is made (that is, new subpulses that were already succesfully serialized will be discarded if an error is encountered for another subpulse).
    lumip committed Jun 28, 2018
    Configuration menu
    Copy the full SHA
    eca1d0e View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2018

  1. Merge pull request #266 from qutech/issues/250_serialization_backward…

    …_compatability
    
    backward-compatible implementations of get_serialization_data and deserialize
    terrorfisch committed Jul 5, 2018
    Configuration menu
    Copy the full SHA
    6995a0a View commit details
    Browse the repository at this point in the history
  2. Rename pulse registration to registry and add it as a keyword argumen…

    …t to each init method.
    
    Trigger garbage collection on each pulse initialization.
    terrorfisch committed Jul 5, 2018
    Configuration menu
    Copy the full SHA
    a5c0074 View commit details
    Browse the repository at this point in the history
  3. Fixed DummySerializable and tests.

    DummySerializable's constructor now also passes on the registry argument.
    Fixed all failing tests.
    lumip committed Jul 5, 2018
    Configuration menu
    Copy the full SHA
    c799fcd View commit details
    Browse the repository at this point in the history
  4. Restrict garbage collection to the case where there is a registered s…

    …erializable with the same name
    terrorfisch committed Jul 5, 2018
    Configuration menu
    Copy the full SHA
    ea0c8e4 View commit details
    Browse the repository at this point in the history
  5. Merge pull request #288 from qutech/issues/276_registry_forwarding

    Fix (global) pulse registry
    lumip committed Jul 5, 2018
    Configuration menu
    Copy the full SHA
    0354e18 View commit details
    Browse the repository at this point in the history
  6. Fixed variable naming as to not overwrite builtins.

    Signed-off-by: Lukas Prediger <lukas.prediger@rwth-aachen.de>
    lumip committed Jul 5, 2018
    Configuration menu
    Copy the full SHA
    266d55d View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    36ddb50 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    1ea0878 View commit details
    Browse the repository at this point in the history
  9. Fixed failing tests.

    NestedDummySerializable's constructor now passed the registry argument through to superclass.
    
    Signed-off-by: Lukas Prediger <lukas.prediger@rwth-aachen.de>
    lumip committed Jul 5, 2018
    Configuration menu
    Copy the full SHA
    441b95b View commit details
    Browse the repository at this point in the history
  10. Merge pull request #285 from qutech/issues/281_pulse_storage_flushing

    Fix issues 271&281
    lumip committed Jul 5, 2018
    Configuration menu
    Copy the full SHA
    eee6037 View commit details
    Browse the repository at this point in the history
  11. Allow default registry to be None.

    Signed-off-by: Lukas Prediger <lukas.prediger@rwth-aachen.de>
    lumip committed Jul 5, 2018
    Configuration menu
    Copy the full SHA
    da57af8 View commit details
    Browse the repository at this point in the history
  12. PulseStorage checks for duplicate id now considers storage backend. A…

    …dded __delitem__() to PulseStorage.
    
    __delitem__() has no test routines yet.
    Closes #273.
    lumip committed Jul 5, 2018
    Configuration menu
    Copy the full SHA
    dcfdd33 View commit details
    Browse the repository at this point in the history
  13. Merge branch 'issues/250_serialization' into issues/270_beautified_js…

    …on_output
    
    Signed-off-by: Lukas Prediger <lukas.prediger@rwth-aachen.de>
    lumip committed Jul 5, 2018
    Configuration menu
    Copy the full SHA
    01aa987 View commit details
    Browse the repository at this point in the history
  14. Merge branch 'issues/250_serialization' into issues/275_pulse_storage…

    …_context_manager
    
    Signed-off-by: Lukas Prediger <lukas.prediger@rwth-aachen.de>
    lumip committed Jul 5, 2018
    Configuration menu
    Copy the full SHA
    b49b943 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    bbfeabe View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2018

  1. Configuration menu
    Copy the full SHA
    345a676 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    183e940 View commit details
    Browse the repository at this point in the history
  3. Merge branch 'master' into issues/250_serialization

    # Conflicts:
    #	tests/pulses/table_pulse_template_tests.py
    terrorfisch committed Jul 6, 2018
    Configuration menu
    Copy the full SHA
    ada2754 View commit details
    Browse the repository at this point in the history
  4. Add test for __delitem__

    terrorfisch committed Jul 6, 2018
    Configuration menu
    Copy the full SHA
    1d92720 View commit details
    Browse the repository at this point in the history
  5. Merge pull request #293 from qutech/issues/289_pulse_storage_delitem_…

    …test
    
    Add test for __delitem__
    terrorfisch committed Jul 6, 2018
    Configuration menu
    Copy the full SHA
    4ab908d View commit details
    Browse the repository at this point in the history
  6. Increase test coverage

    terrorfisch committed Jul 6, 2018
    Configuration menu
    Copy the full SHA
    80b7d90 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2018

  1. Merge pull request #295 from qutech/issues/294_test_coverage

    Increase test coverage
    terrorfisch committed Jul 9, 2018
    Configuration menu
    Copy the full SHA
    bb757a0 View commit details
    Browse the repository at this point in the history
  2. Merge remote-tracking branch 'remotes/origin/issues/250_serialization…

    …' into issues/250_serialization_conversion_script
    lumip committed Jul 9, 2018
    Configuration menu
    Copy the full SHA
    e739a49 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    b7dc1e7 View commit details
    Browse the repository at this point in the history
  4. Fix: MappingPulseTemplate had no implementation for new serialization…

    … routines.
    
    Also added tests (these were also missing for the old serialization routines!).
    lumip committed Jul 9, 2018
    Configuration menu
    Copy the full SHA
    52fbbc5 View commit details
    Browse the repository at this point in the history
  5. Ensured that no occurrence of get_serialization_data/deserialize requ…

    …ires a Serializer object as non-optional positional argument anymore.
    lumip committed Jul 9, 2018
    Configuration menu
    Copy the full SHA
    248b777 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2018

  1. Merge pull request #297 from qutech/issues/250_fix_mapping_pt

    Fixes for new serialization
    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    ac78907 View commit details
    Browse the repository at this point in the history
  2. Added methods set_default_pulse_registry() and new_default_pulse_regi…

    …stry().
    
    set_default_pulse_registry() allows to set a new user-specified registry as the global default pulse registry.
    new_default_pulse_registry() creates a new, empty global default pulse registry (as a WeakValueDictionary).
    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    9cb1190 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    29b6064 View commit details
    Browse the repository at this point in the history
  4. Test to demonstrate issue #301.

    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    27ae2cb View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    07d6e71 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    648ddb8 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    bbc41b4 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    e90cc1e View commit details
    Browse the repository at this point in the history
  9. Added tests for new methods.

    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    c8a106d View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    1d10fde View commit details
    Browse the repository at this point in the history
  11. Added test_conversion() to SerializableTests

    Ensures that the conversion using convert_pulses_in_storage() results in an equal PulseTemplate.
    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    06df6ed View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    bc8bc0e View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    2b4624a View commit details
    Browse the repository at this point in the history
  14. Renamed abstract assert_equal_instance -> assert_equal_instance_excep…

    …t_id in SerializableTests (and added assert_equal_instance building on it)
    
    and made sure that all implementations adhere to that
    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    dfc72ef View commit details
    Browse the repository at this point in the history
  15. Added renamed() method to Serializable.

    Signed-off-by: Lukas Prediger <lukas.prediger@rwth-aachen.de>
    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    f2cd50d View commit details
    Browse the repository at this point in the history
  16. PulseStorage now raises a ValueError when a Serializable/PT is stored…

    … under a different than its own identifier.
    
    Signed-off-by: Lukas Prediger <lukas.prediger@rwth-aachen.de>
    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    bb935b8 View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    18634f7 View commit details
    Browse the repository at this point in the history
  18. Generalizing type annotations for set/get_default_pulse_registry() an…

    …d Serializable.__init__()
    
    Signed-off-by: Lukas Prediger <lukas.prediger@rwth-aachen.de>
    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    0bca178 View commit details
    Browse the repository at this point in the history
  19. Merge pull request #305 from qutech/issues/301_pulse_storage_as_defau…

    …lt_issue
    
    Issues/301 pulse storage as default issue
    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    9ef8a0f View commit details
    Browse the repository at this point in the history
  20. Test to ensure that registration of Serializables does not occur befo…

    …re they serialize correctly.
    
    Signed-off-by: Lukas Prediger <lukas.prediger@rwth-aachen.de>
    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    8573607 View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    87749df View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    d70a29b View commit details
    Browse the repository at this point in the history
  23. Fix #298 : PulseTemplates(/Serializables) are only registered in the …

    …global registry after they are completely initialized.
    
    Serializable now does not perform registration in __init__ but provides a method _register() which MUST be called by leaf-level subclasses. (This is probably also not a good solution in the long term as it effectively prevents further subclassing from those classes but it works for now).
    
    Signed-off-by: Lukas Prediger <lukas.prediger@rwth-aachen.de>
    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    5c4b212 View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    45042a5 View commit details
    Browse the repository at this point in the history
  25. PulseStorage now "implements" MutableMapping.

    Implementations of __len__ and __iter__ curently raise NotImplementedError s.
    
    Signed-off-by: Lukas Prediger <lukas.prediger@rwth-aachen.de>
    lumip committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    d763147 View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    68b2ac5 View commit details
    Browse the repository at this point in the history
  27. Merge pull request #300 from qutech/issues/299_default_registry_none

    Issues/299 default registry none
    terrorfisch committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    fe9c3b7 View commit details
    Browse the repository at this point in the history
  28. Merge pull request #304 from qutech/issues/250_serialization_conversi…

    …on_script
    
    Serialization conversion script
    terrorfisch committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    5a5b5d7 View commit details
    Browse the repository at this point in the history
  29. Configuration menu
    Copy the full SHA
    81b7891 View commit details
    Browse the repository at this point in the history
  30. Configuration menu
    Copy the full SHA
    5faafcc View commit details
    Browse the repository at this point in the history
  31. Merge pull request #306 from qutech/issues/272_PulseStorage_id_incons…

    …istencies
    
    Issues/272 pulse storage id inconsistencies
    terrorfisch committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    62d18ad View commit details
    Browse the repository at this point in the history
  32. Configuration menu
    Copy the full SHA
    f638d24 View commit details
    Browse the repository at this point in the history
  33. Merge pull request #307 from qutech/issues/298_early_pulse_registration

    Issues/298 early pulse registration
    terrorfisch committed Jul 12, 2018
    Configuration menu
    Copy the full SHA
    b796ab1 View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2018

  1. Merge branch 'issues/250_serialization' into issues/308_consistent_ty…

    …pe_annotations_for_registry
    lumip committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    6f08c7d View commit details
    Browse the repository at this point in the history
  2. Introduced PulseRegistryType and changed type annotations for pulse r…

    …egistry occurrences accordingly.
    lumip committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    a8732f9 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    c6ab296 View commit details
    Browse the repository at this point in the history
  4. Merge pull request #310 from qutech/issues/308_consistent_type_annota…

    …tions_for_registry
    
    Issues/308 consistent type annotations for registry
    terrorfisch committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    42b888d View commit details
    Browse the repository at this point in the history
  5. Added test to ensure that PulseStorage duplicate id check is consiste…

    …nt over several instantiations.
    lumip committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    ae21951 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    2925c89 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    9548742 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2018

  1. Configuration menu
    Copy the full SHA
    0d1f9df View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2018

  1. Merge pull request #311 from qutech/issues/250_one_more_test

    Increased test coverage for serialization
    lumip committed Jul 17, 2018
    Configuration menu
    Copy the full SHA
    47338ee View commit details
    Browse the repository at this point in the history