Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

archival: Delete replaced segments during spillover GC #16163

Conversation

Lazin
Copy link
Contributor

@Lazin Lazin commented Jan 19, 2024

Previously, if the spillover was enable we weren running normal GC during the housekeeping run. Normal housekeeping is responsible for removing segments evicted due to retention (which is not happening if spillover region is not empty). But it also removes replaced segments. So essentially, we have to run both types of GC on every spillover housekeeping run.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.3.x
  • v23.2.x
  • v23.1.x

Release Notes

Bug Fixes

  • Fix tiered-storage housekeeping problem that may cause replaced segments to pile up if the spillover is enabled.

Previously, if the spillover was enable we weren running normal GC
during the housekeeping run. Normal housekeeping is responsible for
removing segments evicted due to retention (which is not happening if
spillover region is not empty). But it also removes replaced segments.
So essentially, we have to run both types of GC on every spillover
housekeeping run.
@Lazin Lazin added sev/high loss of availability, pathological performance degradation, recoverable corruption area/cloud-storage Shadow indexing subsystem labels Jan 19, 2024
@Lazin Lazin requested review from andijcr and abhijat January 19, 2024 10:30
@piyushredpanda piyushredpanda merged commit 24f31d3 into redpanda-data:dev Jan 19, 2024
19 checks passed
@vbotbuildovich
Copy link
Collaborator

/backport v23.3.x

@vbotbuildovich
Copy link
Collaborator

/backport v23.2.x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/cloud-storage Shadow indexing subsystem area/redpanda sev/high loss of availability, pathological performance degradation, recoverable corruption
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants