Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[v23.1.x] cloud_storage: move eviction under remote_partition #9758

Merged
merged 1 commit into from
Apr 1, 2023
Merged

[v23.1.x] cloud_storage: move eviction under remote_partition #9758

merged 1 commit into from
Apr 1, 2023

Conversation

vbotbuildovich
Copy link
Collaborator

Backport of PR #9590

@andrwng @vbotbuildovich
cloud_storage: move eviction under remote_partition
77243d2 
Previously each remote_partition would wait for an eviction barrier to
pass through the eviction loop, ensuring all segments are destructed
before stopping the partition. Each segment references members of the
remote_partition, so it's important the shutdown sequence stops the
segments before destructing the remote_partition. At the same time,
having each partition wait for another set of partitions to finish
flushing can result in a slow shutdown.

This commit moves the eviction loop into the remote_partition, allowing
partition shutdown to entirely avoid waiting for any other partition to
shut down, while still ensuring that each underlying segment is
destructed after the remote_partition.

Without this commit, I witnessed the period of partition shutdown in a
heavily loaded server take 30 minutes. With this commit I see a
similarly shaped shutdown taking 10 seconds.

Related redpanda-data#9569

(cherry picked from commit 03587d8)
@vbotbuildovich vbotbuildovich added this to the v23.1.x-next milestone Mar 31, 2023
@vbotbuildovich vbotbuildovich added the kind/backport PRs targeting a stable branch label Mar 31, 2023
@andrwng andrwng marked this pull request as ready for review April 1, 2023 02:28
andrwng
andrwng approved these changes Apr 1, 2023
View reviewed changes
Copy link
Contributor

@andrwng andrwng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean backport

@andrwng andrwng merged commit 66e350f into redpanda-data:v23.1.x Apr 1, 2023
@RafalKorepta RafalKorepta modified the milestones: v23.1.x-next, v23.1.5 Apr 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrwng andrwng andrwng approved these changes

@Lazin Lazin Awaiting requested review from Lazin

@jcsp jcsp Awaiting requested review from jcsp

Assignees
No one assigned
Labels
area/redpanda kind/backport PRs targeting a stable branch
Projects
None yet
Milestone
v23.1.5
Development

Successfully merging this pull request may close these issues.
3 participants
@vbotbuildovich @andrwng @RafalKorepta