NEW FEATURE: diff2: A better "diff" mechanism#1852
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds an experimental diffing algorithm called "diff2" which will some day replace the original "diff" algorithm. For now this feature is hidden behind a flag. Once it is stable, it will become the default and the original will be removed.
The feature is hidden behind the flag
--diff2(or-diff2when running integration tests).Diff2 is a complete rewrite from scratch. The goals are (1) make
IGNORE*()and related features more reliable, (2) make it easier to write providers by providing a more simple API, (3) add a general mechanism (UNMANAGED()) which replaces theIGNORE*()functions (theIGNORE*()functions will still exist, they'll just callUNMANAGED())diff2 is significantly less complex than the original diff. How did we achieve this? The old system worked with the data as it exists in the
models.Recordsstructure, which is difficult to work with. diff2 starts by gathering all the DNS records into a structure that is more friendly to the work that needs to be done. Now the differencing algorithm becomes easier to write. In other words, instead of solving a hard problem, we first re-arrange the data so that the original problem is easier.diff2 is significantly easier for the providers to use. The old system returned a list of adds, changes and deletes and expected the provider to re-arrange the data into groups that matched how the API expects updates to be done. The new system provides 4 entry-points covering the 4 types of APIs we've seen in the wild: ByRecord, ByRSet, ByLabel and ByZone. They all return a list of "instructions" that simply need to be translated into API calls.
Timeline:
Since the old
pkg/diffandpkg/diff2can co-exist, there's no rush to convert all providers. However, I don't want to delay too much. I will probably convert most providers myself or at least send draft PRs to provider-owners. There will be at least 1 release with the--diff2flag defaulting off, so that all users can test the new system before it becomes the default. After `diff2 becomes the default, the old code will disappear shortly after. I hope to have this completed by February or March of 2023. Maybe sooner? We'll see!