Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prune and preserve necessary updates in generate_appcast #2218

Merged
merged 2 commits into from
Aug 8, 2022

Conversation

zorgiepoo
Copy link
Member

@zorgiepoo zorgiepoo commented Aug 1, 2022

We preserve a batch of updates for each branch point (defined by an update's {majorVersion, minSystemVersion, maxSystemVersion, channel}). The latest for each branch point is used for generating delta updates.

We prune older update items that are no longer needed in the feed or needed for generating delta updates from newer items. We move pruned update files into a separate hidden directory, which is later garbage collected after a couple weeks based on each file's last modification date.

To insert new update items that are older than the latest update in the feed, --version flag needs to be used.

The overall goal is to have the feed only keep a small window of updates to serve to the user. This corrects behavior by examining the existing update feeds first and collecting information to determine all available branch points that needs to be preserved. This will also ease future processing generate_appcast needs to do and ensure the cache's directory doesn't blow up by pruning old updates.

Fixes #1719

Misc Checklist

  • My change requires a documentation update on Sparkle's website repository
  • My change requires changes to generate_appcast, generate_keys, or sign_update

Only bug fixes to regressions or security fixes are being backported to the 1.x (master) branch now. If you believe your change is significant enough to backport, please also create a separate pull request against the master branch.

Testing

I tested and verified my change by using one or multiple of these methods:

  • Sparkle Test App
  • Unit Tests
  • My own app
  • Other (please specify)

Testing with large corpus of updates with generate_appcast.
Tested creating new updates incrementally, with a new channel or major update requirement.

macOS version tested: 12.5 (21G72)

We (roughly) preserve a batch of updates for each branch point (defined by an update's {majorVersion, minSystemVersion, maxSystemVersion, channel}). The latest for each branch point is used for generating delta updates.

We prune older update items that are no longer needed in the feed or needed for generating delta updates from newer items. We move pruned update files into a separate hidden directory, which is later garbage collected after a couple weeks based on each file's last modification date.

To insert new update items that are older than the latest update in the feed, --version flag needs to be used.

The overall goal is to have the feed only keep a small window of updates to serve to the user. This will also ease future processing generate_appcast needs to do and ensure the cache's directory doesn't blow up by pruning old updates.
@zorgiepoo zorgiepoo merged commit 37ef179 into 2.x Aug 8, 2022
@zorgiepoo zorgiepoo deleted the generate-appcast-pruning branch August 8, 2022 02:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

generate_appcast seems to completely overwhelm my system when there are a lot of packages
1 participant