Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Only one retention cycle in progress at a time, #31785 #31786

Merged
merged 2 commits into from
Dec 14, 2022

Conversation

patriknw
Copy link
Member

@patriknw patriknw commented Dec 9, 2022

On top of #31784 but otherwise ready for review.

  • Keep track of retention steps, including detailed debug logging.
  • If retention is in progress when a new retention is triggered it will skip that retention cycle.
  • Had to include that as mutable state in BehaviorSetup rather than in the RunningState because the background tasks of deleting events and snapshots can't easily update the RunningState.
  • Also changed so that snapshot deletion always use 0 as lower sequence number.
    • This is important because otherwise some snapshots may be left behind when retention is skipped. That could probably already happen in failure scenarios previously.

The retention process for SnapshotCountRetentionCriteria looks like this:

  1. Save snapshot after persisting events when shouldSnapshotAfterPersist returned SnapshotWithRetention.
  2. Delete events (when deleteEventsOnSnapshot=true), runs in background.
  3. Delete snapshots (when isOnlyOneSnapshot=false), runs in background.

References #31785

Copy link
Member

@johanandren johanandren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic LGTM, some readability nitpicks

Base automatically changed from wip-retention-snapshots-patriknw to main December 9, 2022 16:33
* Keep track of retention steps, including detailed debug logging.
* If retention is in progress when a new retention is triggered it will
  skip that retention cycle.
* Had to include that as mutable state in BehaviorSetup rather than in the
  RunningState because the background tasks of deleting events and snapshots
  can't easily update the RunningState.
* Also changed so that snapshot deletion always use 0 as lower sequence number.
  * This is important because otherwise some snapshots may be left behind when
    retention is skipped. That could probably already happen in failure scenarios
    previously.

The retention process for SnapshotCountRetentionCriteria looks like this:
1. Save snapshot after persisting events when shouldSnapshotAfterPersist returned SnapshotWithRetention.
2. Delete events (when deleteEventsOnSnapshot=true), runs in background.
3. Delete snapshots (when isOnlyOneSnapshot=false), runs in background.
@patriknw patriknw force-pushed the wip-31785-retention-inprogress-patriknw branch from cfc7de3 to 91dcee7 Compare December 9, 2022 16:38
@patriknw patriknw marked this pull request as ready for review December 14, 2022 11:18
@patriknw patriknw merged commit 57b750a into main Dec 14, 2022
@patriknw patriknw deleted the wip-31785-retention-inprogress-patriknw branch December 14, 2022 11:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants