Prune and preserve necessary updates in generate_appcast #2218
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We preserve a batch of updates for each branch point (defined by an update's {majorVersion, minSystemVersion, maxSystemVersion, channel}). The latest for each branch point is used for generating delta updates.
We prune older update items that are no longer needed in the feed or needed for generating delta updates from newer items. We move pruned update files into a separate hidden directory, which is later garbage collected after a couple weeks based on each file's last modification date.
To insert new update items that are older than the latest update in the feed, --version flag needs to be used.
The overall goal is to have the feed only keep a small window of updates to serve to the user. This corrects behavior by examining the existing update feeds first and collecting information to determine all available branch points that needs to be preserved. This will also ease future processing generate_appcast needs to do and ensure the cache's directory doesn't blow up by pruning old updates.
Fixes #1719
Misc Checklist
Only bug fixes to regressions or security fixes are being backported to the 1.x (master) branch now. If you believe your change is significant enough to backport, please also create a separate pull request against the master branch.
Testing
I tested and verified my change by using one or multiple of these methods:
Testing with large corpus of updates with generate_appcast.
Tested creating new updates incrementally, with a new channel or major update requirement.
macOS version tested: 12.5 (21G72)