Add package block for content-aware JSON config diffing and patching #259
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Package split implements diffing and patching of JSON-like structures, where the boundaries between logical objects are defined by schema.
Arrays can be diffed if they are arrays of maps. The splits must identify a key for the array, to track element identity.
Add test for Patch unmarshalling.
There was a bug where if a nested field/element was deleted, then the top-level field containing it would be deleted instead. Now a field/element is only deleted if it matches the final path element.
To make testing easier, output Patches from Diff sorted by their Path.
Perform a round-trip JSON encoding to convert arbitrary Go types into the working format for Diff and ApplyPatches. ApplyPatch changed into ApplyPatches - because of the JSON encode/decode above, the operation is now more expensive and no longer in-place. Therefore, allow providing multiple patches to apply at the same time.
Due to a shadowed variable, nil was being incorrectly returned as Patch.Value. Add more tests to improve coverage. Add testable examples to demonstrate usage of the package.
|
@mattnathan I've separated this out from the original PR (#256) as discussed |
We decided it's a less confusing name, overall. Rename type Split to Block. Rename fields and parameters accordingly.
Add new string-based format for representing paths in Patch. Old array-based format is still supported for parsing in JSON, but output will be in the new string-based format.
|
@mattnathan I've added the new string-based XMLPath-like path format as discussed. |
Use the Path wrapper type in Patch, so that the new string JSON serialisation format is used.
Make Path.MarshalJSON a value-receiver method, so Patch.Path is serialised correctly. Change ExampleDiff to show new path format.
mattnathan
requested changes
Aug 23, 2024
The slices package doesn't exist yet in Go 1.20 (added in Go 1.21). Use golang.org/x/exp/slices which contains the required functions.
Creates a sort order for Paths. Remove sorting of patches from Diff - document that the user can do it using ComparePaths if required. Sort patches in TestDiff, ExampleDiff so we have deterministic output to check against.
move compareAny (block.go) -> comparePrimitives (path.go) to better reflect its purpose and usage site. diffMap: rename variables to match type names diffPages -> diffBlockValues: rename to match type name. Adjust local variable names too. prefixPatches, fieldPathSegments: use Path instead of []PathSegment fieldTree -> blockTree: rename type to better reflect what it is - a tree-based representation of a []Block. Also add an example of the transformation performed by buildBlockTree. splitMap: document purpose in more detail, with example Block: reorder fields for better grouping. Changes based on review feedback (#259).
Allow parsing paths that start with an array access. Allow multiple contiguous array subscripts.
hexaglow
changed the title
mattnathan
requested changes
Aug 29, 2024
… sub-block maps.Clone(nil) returns nil, which can't be written to. In splitMap, we require a writeable map so add special-case handling. Add regression test which can detect this issue.
If a patch references a sub-field (or sub-array, at any nesting depth) of an array element that doesn't exist, the patch failed to apply. In applyPatch, when referencing an array element that doesn't exist, create a placeholder with the right key so we have something to apply the patch to. This could be triggered with patches from Diff, if the patch to the sub-field came before the patch creating the array element.
Use global ignoreSentinal object in Ignore.MarshalJSON In equalKeys, second loop could only return false if the first loop had, so it's redundant. Comparing length and the keys from a to b is sufficient.
mattnathan
approved these changes
Aug 29, 2024
hexaglow
added a commit
that referenced
this pull request
Aug 29, 2024
move compareAny (block.go) -> comparePrimitives (path.go) to better reflect its purpose and usage site. diffMap: rename variables to match type names diffPages -> diffBlockValues: rename to match type name. Adjust local variable names too. prefixPatches, fieldPathSegments: use Path instead of []PathSegment fieldTree -> blockTree: rename type to better reflect what it is - a tree-based representation of a []Block. Also add an example of the transformation performed by buildBlockTree. splitMap: document purpose in more detail, with example Block: reorder fields for better grouping. Changes based on review feedback (#259).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New package
blockcan be used for diffing JSON-like config structures. FunctionDiffproduces a slice ofPatches, which can be applied to the original A structure to transform it into the B structure.Difftakes a list ofBlocks, which define the boundaries withing the structure between changes.By designing the
Blocks to match the logical structure of the config, the user can apply patches to different config structures than were originally used to produce theDiff, with limited chance of conflict.