Skip to content

feat(core): compress ValidMeasurementResults when serializing#769

Merged
AlessandroPomponio merged 9 commits into
mainfrom
ap_709_minimize_measurement_result
Apr 1, 2026
Merged

feat(core): compress ValidMeasurementResults when serializing#769
AlessandroPomponio merged 9 commits into
mainfrom
ap_709_minimize_measurement_result

Conversation

@AlessandroPomponio
Copy link
Copy Markdown
Member

@AlessandroPomponio AlessandroPomponio commented Mar 31, 2026

Summary

This PR implements a compressed serialization format for ValidMeasurementResult to eliminate redundant ExperimentReference data. Instead of repeating the same ExperimentReference in every measurement's property, it's now stored once at the top level, significantly reducing serialized size. The implementation maintains full backward compatibility with the old format.

Resolves #709

Files Changed

📄 orchestrator/schema/result.py

Added custom serialization and deserialization methods to ValidMeasurementResult class:

  • serialize_to_custom_format(): Custom serializer that extracts the common ExperimentReference (validated to be identical across all measurements) and stores it at the top level, removing it from individual ObservedPropertyValue entries. Properly handles all Pydantic serialization settings like exclude_none, exclude_defaults, etc.
  • deserialize_from_custom_format(): Custom validator that handles both old (redundant) and new (compressed) formats, ensuring backward compatibility. Detects the format and reconstructs measurements appropriately.
  • Removed outdated docstring comment about accessing experiment reference through measurements.

📄 tests/schema/test_measurement_result.py

Added comprehensive test coverage for the new serialization format:

  • test_compressed_serialization_format(): Verifies the new format structure with experimentReference at top level
  • test_old_format_deserialization(): Ensures backward compatibility by testing deserialization of old format data
  • test_new_format_deserialization(): Validates correct deserialization of the new compressed format
  • test_serialization_deserialization_roundtrip(): Tests full roundtrip to ensure data integrity
  • test_compression_achieved(): Quantifies compression benefits by comparing serialized sizes
  • test_json_serialization_roundtrip(): Verifies JSON string serialization/deserialization works correctly with the new format

Signed-off-by: Alessandro Pomponio <alessandro.pomponio1@ibm.com>
@AlessandroPomponio
Copy link
Copy Markdown
Member Author

I confirmed the minimization is happening by running a new operation using https://ibm.github.io/ado/examples/no-priors-characterization/

I was then able to do ado show entities with both the latest operation and one that was run before this change

Signed-off-by: Alessandro Pomponio <alessandro.pomponio1@ibm.com>
Comment thread orchestrator/schema/result.py Outdated
Comment thread orchestrator/schema/result.py Outdated
Comment thread orchestrator/schema/result.py Outdated
Comment thread orchestrator/schema/result.py Outdated
Comment thread orchestrator/schema/result.py Outdated
Comment thread orchestrator/schema/result.py Outdated
Signed-off-by: Alessandro Pomponio <alessandro.pomponio1@ibm.com>
Signed-off-by: Alessandro Pomponio <alessandro.pomponio1@ibm.com>
Signed-off-by: Alessandro Pomponio <alessandro.pomponio1@ibm.com>
Comment thread orchestrator/schema/result.py
Comment thread orchestrator/schema/result.py Outdated
Signed-off-by: Alessandro Pomponio <alessandro.pomponio1@ibm.com>
Signed-off-by: Alessandro Pomponio <alessandro.pomponio1@ibm.com>
Signed-off-by: Alessandro Pomponio <alessandro.pomponio1@ibm.com>
Comment thread orchestrator/schema/result.py Outdated
Co-Authored-By: Michael Johnston <66301584+michael-johnston@users.noreply.github.com>
Signed-off-by: Alessandro Pomponio <alessandro.pomponio1@ibm.com>
Copy link
Copy Markdown
Member

@michael-johnston michael-johnston left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice - looks clean.

@AlessandroPomponio AlessandroPomponio added this pull request to the merge queue Apr 1, 2026
Merged via the queue into main with commit aba9ab6 Apr 1, 2026
19 checks passed
@AlessandroPomponio AlessandroPomponio deleted the ap_709_minimize_measurement_result branch April 1, 2026 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: custom serialize to compress measurement result

2 participants