Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

br: ensure pvc names sequential for ebs restore #4888

Merged
merged 7 commits into from Feb 24, 2023
Merged

br: ensure pvc names sequential for ebs restore #4888

merged 7 commits into from Feb 24, 2023

Conversation

WangLe1321
Copy link
Contributor

@WangLe1321 WangLe1321 commented Feb 17, 2023 •
edited by fengou1

What problem does this PR solve?

Closes #4884

What is changed and how does it work?

Code changes

  • Has Go code change
  • Has CI related scripts change

Tests

  • Unit test
  • E2E test
  • Manual test

precondition.
according to https://docs.pingcap.com/zh/tidb-in-kubernetes/stable/advanced-statefulset, upgrade tide-operator with asts.

case#1

  • create a cluster with 6 nodes
  • scale out the cluster to 9 nodes
  • scale in the cluster to 6 nodes, and set delete-slots [3,4,5]
  • keep released existing pv/pvcs and then do the backup
  • restore the backup
    4KPXRP1DmX

case#2

  • create a cluster with 6 nodes

  • scale out the cluster to 9 nodes

  • scale in the cluster to 6 nodes, and set delete-slots [3,4,5]

  • remove released existing pv/pvcs and then do the backup

  • restore the backup
    UVmA3RdkK2

  • No code

Side effects

  • Breaking backward compatibility
  • Other side effects:

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Release Notes

Please refer to Release Notes Language Style Guide before writing the release note.

@WangLe1321
br: ensure pvc names sequential for ebs restore
4697fad 
Signed-off-by: WangLe1321 <wangle1321@163.com>
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Feb 17, 2023 •
edited

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • WizardXiao
  • fengou1

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@fengou1 fengou1 self-requested a review February 17, 2023 08:31
@codecov-commenter
Copy link

codecov-commenter commented Feb 17, 2023 •
edited

Codecov Report

Merging #4888 (51886e0) into master (c874bb8) will increase coverage by 1.10%.
The diff coverage is 72.05%.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #4888      +/-   ##
==========================================
+ Coverage   59.39%   60.49%   +1.10%     
==========================================
  Files         227      231       +4     
  Lines       25793    28944    +3151     
==========================================
+ Hits        15319    17511    +2192     
- Misses       9017     9879     +862     
- Partials     1457     1554      +97
Flag Coverage Δ
e2e 25.43% <0.00%> (?)
unittest 59.43% <79.03%> (+0.04%) ⬆️

@WangLe1321
br: remove unused pvs and pvcs for ebs restore
1507e3e 
Signed-off-by: WangLe1321 <wangle1321@163.com>
@WangLe1321
br: remove pv without volume id
7107b49 
Signed-off-by: WangLe1321 <wangle1321@163.com>
@WangLe1321
br: add comments
d4022e5 
Signed-off-by: WangLe1321 <wangle1321@163.com>
@WangLe1321
Copy link
Contributor Author

/run pull-e2e-kind

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind pull-e2e-kind-across-kubernetes pull-e2e-kind-basic pull-e2e-kind-br pull-e2e-kind-serial pull-e2e-kind-tikv-scale-simultaneously

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind pull-e2e-kind-across-kubernetes pull-e2e-kind-serial

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind-across-kubernetes

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind pull-e2e-kind-across-kubernetes pull-e2e-kind-serial

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind-tngm

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind-across-kubernetes

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind pull-e2e-kind-across-kubernetes pull-e2e-kind-br pull-e2e-kind-serial pull-e2e-kind-tikv-scale-simultaneously pull-e2e-kind-tngm

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind-tikv-scale-simultaneously

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind

1 similar comment
@fengou1
Copy link
Contributor

fengou1 commented Feb 22, 2023

/test pull-e2e-kind

@fengou1
Copy link
Contributor

fengou1 commented Feb 23, 2023

/test pull-e2e-kind

1 similar comment
@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind-across-kubernetes pull-e2e-kind-basic pull-e2e-kind-tikv-scale-simultaneously

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind pull-e2e-kind-br pull-e2e-kind-serial

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind

2 similar comments
@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind

@fengou1
Copy link
Contributor

fengou1 commented Feb 23, 2023

/test pull-e2e-kind

@fengou1
Copy link
Contributor

fengou1 commented Feb 23, 2023

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 51886e0

@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind

3 similar comments
@WangLe1321
Copy link
Contributor Author

/test pull-e2e-kind

@fengou1
Copy link
Contributor

fengou1 commented Feb 23, 2023

/test pull-e2e-kind

@fengou1
Copy link
Contributor

fengou1 commented Feb 24, 2023

/test pull-e2e-kind

@csuzhangxc csuzhangxc merged commit 136086c into pingcap:master Feb 24, 2023
WizardXiao pushed a commit to WizardXiao/tidb-operator that referenced this pull request Mar 10, 2023
@WangLe1321 @WizardXiao
br: ensure pvc names sequential for ebs restore (pingcap#4888)
fcda371
@WizardXiao WizardXiao mentioned this pull request Mar 10, 2023
Merged
10 tasks
WizardXiao added a commit that referenced this pull request Mar 11, 2023
@WizardXiao @fengou1 @WangLe1321
dp: cherry pick dp prs to release-1.4 (#4929)
7f4e089 
* feat: support tiflash backup and restore during volume snapshot (#4812)

* feat: calc the backup size from snapshot storage usage (#4819)

* fix backup failed when pod was auto restarted by k8s (#4883)

* init code for test

* just clean before backup data

* delete test code

* import pingcap/errors

* add check version

* remove test code

* add running status check

* add restart condition to clarify logic

* fix status update

* fix ut

* br: ensure pvc names sequential for ebs restore (#4888)

* BR: Restart backup when backup job/pod unexpected failed by k8s (#4895)

* init code for test

* just clean before backup data

* delete test code

* import pingcap/errors

* add check version

* remove test code

* add running status check

* add restart condition to clarify logic

* fix status update

* fix ut

* init code

* update crd reference

* fix miss update retry count

* add retry limit as constant

* init runnable code

* refine main controller logic

* add some note

* address some comments

* init e2e test code

* add e2e env to extend backup time

* add e2e env for test

* fix complie

* just test kill pod

* refine logic

* use pkill to kill pod

* fix reconcile

* add kill pod log

* add more log

* add more log

* try kill pod only

* wait and kill running backup pod

* add wait for pod failed

* fix wait pod running

* use killall backup to kill pod

* use pkill -9 backup

* kill pod until pod is failed

* add ps to debug

* connect commands by semicolon

* kill pod by signal 15

* use panic simulate kill pod

* test all kill pod test

* remove useless log

* add original reason of job or pod failure

* rename BackupRetryFailed to BackupRetryTheFailed

* BR: Auto truncate log backup in backup schedule (#4904)

* init schedule log backup code

* add run log backup code

* update api

* refine some nodes

* refine cacluate logic

* add ut

* fix make check

* add log backup test

* refine code

* fix notes

* refine function names

* fix conflict

* fix: add a new check for encryption during the volume snapshot restore (#4914)

* br: volume-snapshot may lead to a panic when there is no block change between two snapshot (#4922)

* br: refine BackoffRetryPolicy time format (#4925)

* refine BackoffRetryPolicy time format

* fix some ut

---------

Co-authored-by: fengou1 <85682690+fengou1@users.noreply.github.com>
Co-authored-by: WangLe1321 <wangle1321@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fengou1 fengou1 fengou1 approved these changes

@WizardXiao WizardXiao WizardXiao approved these changes

@handlerww handlerww Awaiting requested review from handlerww

@liubog2008 liubog2008 Awaiting requested review from liubog2008

Assignees

@WangLe1321 WangLe1321

Labels
area/backup status/can-merge status/LGT2
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

br: volume-snapshot can not restore when backup a scale-in cluster
6 participants
@WangLe1321 @ti-chi-bot @codecov-commenter @fengou1 @WizardXiao @csuzhangxc