Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ebs br: serial execution of volume backup gc #5452

Merged
merged 5 commits into from Dec 19, 2023

Conversation

BornChanger
Copy link
Contributor

@BornChanger BornChanger commented Dec 11, 2023

What problem does this PR solve?

Closes #5451

What is changed and how does it work?

This PR make the EBS snapshot volume backup GC execute serially. In the single volumebackup, delete snapshots in batches.

Code changes

  • Has Go code change
  • Has CI related scripts change

Tests

  • Unit test
  • E2E test
  • Manual test
  • No code

Side effects

  • Breaking backward compatibility
  • Other side effects:

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Release Notes

Please refer to Release Notes Language Style Guide before writing the release note.


@ti-chi-bot ti-chi-bot bot requested a review from shonge December 11, 2023 15:31
@ti-chi-bot ti-chi-bot bot added the size/L label Dec 11, 2023
Signed-off-by: BornChanger <dawn_catcher@126.com>
@BornChanger
Copy link
Contributor Author

/retest

@BornChanger
Copy link
Contributor Author

/cc @WangLe1321 @csuzhangxc

@codecov-commenter
Copy link

codecov-commenter commented Dec 11, 2023

Codecov Report

Merging #5452 (dedb25b) into master (34b37bb) will increase coverage by 0.00%.
The diff coverage is 28.57%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #5452   +/-   ##
=======================================
  Coverage   61.57%   61.58%           
=======================================
  Files         230      230           
  Lines       29156    29166   +10     
=======================================
+ Hits        17954    17963    +9     
+ Misses       9433     9431    -2     
- Partials     1769     1772    +3     
Flag Coverage Δ
unittest 61.58% <28.57%> (+<0.01%) ⬆️

Signed-off-by: BornChanger <dawn_catcher@126.com>
@BornChanger
Copy link
Contributor Author

/test-pull-e2e-kind

Signed-off-by: BornChanger <dawn_catcher@126.com>
@BornChanger
Copy link
Contributor Author

/run-pull-e2e-kind-br

@ti-chi-bot ti-chi-bot bot added the lgtm label Dec 19, 2023
Copy link
Contributor

ti-chi-bot bot commented Dec 19, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: WangLe1321

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the approved label Dec 19, 2023
@ti-chi-bot ti-chi-bot bot removed the lgtm label Dec 19, 2023
Copy link
Contributor

ti-chi-bot bot commented Dec 19, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-12-19 04:19:01.696908278 +0000 UTC m=+934632.734135197: ☑️ agreed by WangLe1321.
  • 2023-12-19 05:57:39.186258018 +0000 UTC m=+940550.223484967: ✖️🔁 reset by ti-chi-bot[bot].

Copy link
Contributor

ti-chi-bot bot commented Dec 19, 2023

New changes are detected. LGTM label has been removed.

@csuzhangxc csuzhangxc merged commit d6e665e into pingcap:master Dec 19, 2023
5 of 6 checks passed
@csuzhangxc
Copy link
Member

/cherry-pick release-1.5

@ti-chi-bot
Copy link
Member

@csuzhangxc: new pull request created to branch release-1.5: #5473.

In response to this:

/cherry-pick release-1.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

csuzhangxc pushed a commit that referenced this pull request Dec 19, 2023
Signed-off-by: BornChanger <dawn_catcher@126.com>
Co-authored-by: BornChanger <dawn_catcher@126.com>
Comment on lines +151 to +155
eg, _ := errgroup.WithContext(context.Background())
workerPool := NewWorkerPool(e.concurrency, "delete snapshots")
for volID := range snapIDMap {
snapID := snapIDMap[volID]
eg.Go(func() error {
workerPool.ApplyOnErrorGroup(eg, func() error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the batch size. How many snapshots will be deleted concurrently.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its 3 -

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ebs br: serialize deletion of volumebackups and snapshots in backup
6 participants