fix(tests): environment-agnostic cassette normalization and sorting#1521
Merged
fix(tests): environment-agnostic cassette normalization and sorting#1521
Conversation
VCR cassette normalization improvements to eliminate noisy diffs between local Docker and staging recordings: - Add _sort_known_arrays() that sorts only specific non-deterministic API response arrays (referenceProperties, edges, workspaceDataFilterColumns/References, userGroups) during YAML serialization. Unlike blanket sorting, this preserves server order for arrays that tests assert on. - Fix XML group sorting in bytes branch: _sort_xml_groups() was only called for str bodies, skipping bytes bodies from compressed responses. Now applied in both branches of custom_before_request() and custom_before_response(). - Add _stabilize_hash_placeholders() to rename session-global numeric hash indices (EXECUTION_NORMALIZED_3) to format-based names (EXECUTION_NORMALIZED_CSV, EXPORT_NORMALIZED_XLSX) for export cassettes, with per-cassette re-indexing for non-export hashes. - Add firstname/lastname/email to demo_user.json fixture so staging user profile matches local Docker bootstrap. risk: low
Re-recorded all VCR cassettes against local Docker to pick up the new normalization: targeted JSON array sorting for referenceProperties, edges, workspaceDataFilterColumns/References, and userGroups. risk: low
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1521 +/- ##
=======================================
Coverage 78.18% 78.18%
=======================================
Files 228 228
Lines 14930 14930
=======================================
Hits 11673 11673
Misses 3257 3257 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
jaceksan
approved these changes
Apr 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When VCR cassettes are recorded locally (Docker Compose) vs on staging, the resulting YAML files differ in ways that have nothing to do with actual API changes — causing noisy, hard-to-review PRs.
Root causes identified:
referenceProperties,edges,workspaceDataFilterReferences, etc.) in non-deterministic orderdemouser has profile fields (email, name) locally but not on stagingChanges
Targeted JSON array sorting
Sorts only arrays under 5 specific keys known to be non-deterministic:
referenceProperties,edges,workspaceDataFilterColumns,workspaceDataFilterReferences,userGroups. Applied only during YAML serialization — does not affect VCR replay behavior or test assertions.XML group sorting fix
_sort_xml_groups()was only called forstrresponse bodies. Compressed responses arrive asbytesand were skipped entirely. Now applied in both branches ofcustom_before_request()andcustom_before_response().Export hash stabilization
Replaces session-global numeric indices (
EXECUTION_NORMALIZED_3) with format-based names (EXECUTION_NORMALIZED_CSV,EXPORT_NORMALIZED_XLSX). Each cassette gets self-contained, deterministic placeholders regardless of test execution order.Demo user profile fields
Added
firstname,lastname,emailtodemo_user.jsonsomake load-stagingproduces the same user profile as the local Docker bootstrap.Test plan
OVERWRITE— all tests passOVERWRITE— all tests pass (190 SDK + 66 pandas)make clean-staging+make load-stagingsucceeds and demo user has profile fieldsrisk: low