Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BR: Auto truncate log backup in backup schedule #4904

Merged
merged 14 commits into from
Mar 7, 2023

Conversation

WizardXiao
Copy link
Contributor

@WizardXiao WizardXiao commented Feb 24, 2023

What problem does this PR solve?

What is changed and how does it work?

  1. add logBackupTemplate to backup schedule, it can manage log backup, include start, auto truncate, delete
  2. add logBackup to backup schedule's status
  3. truncate log backup accroding to maxReservedTime like snapshot backup

Code changes

  • Has Go code change
  • Has CI related scripts change

Tests

  • Unit test
  • E2E test
  • Manual test
  • No code
  1. create backup schedule with log backup:
    BlTLqhn6oY
  2. before backup exceed the time limit
    4790c6a1-8f92-4caf-992b-7e4aef52c9a7
  3. after backup exceed the time limit
    c2906676-1292-42b3-9315-3c701ab9ab00
  4. log backup was auto truncated
    bc33165d-f0d6-4bc3-9503-f77682cf28d8

Side effects

  • Breaking backward compatibility
  • Other side effects:

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation

Release Notes

Please refer to Release Notes Language Style Guide before writing the release note.

Auto truncate log backup in backup schedule

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Feb 24, 2023

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • WangLe1321
  • grovecai

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@WizardXiao WizardXiao changed the title [WIP] Auto truncate log backup [WIP] Auto truncate log backup in backup schedule Feb 24, 2023
@codecov-commenter
Copy link

codecov-commenter commented Feb 24, 2023

Codecov Report

Merging #4904 (40a2043) into master (98237ee) will increase coverage by 1.14%.
The diff coverage is 59.22%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4904      +/-   ##
==========================================
+ Coverage   59.06%   60.21%   +1.14%     
==========================================
  Files         227      231       +4     
  Lines       26063    29387    +3324     
==========================================
+ Hits        15393    17694    +2301     
- Misses       9204    10116     +912     
- Partials     1466     1577     +111     
Flag Coverage Δ
e2e 25.05% <0.00%> (?)
unittest 59.13% <65.59%> (+0.07%) ⬆️

@WizardXiao WizardXiao changed the title [WIP] Auto truncate log backup in backup schedule Auto truncate log backup in backup schedule Feb 28, 2023
@WizardXiao WizardXiao changed the title Auto truncate log backup in backup schedule BR: Auto truncate log backup in backup schedule Feb 28, 2023
@WizardXiao
Copy link
Contributor Author

/test pull-e2e-kind-br

@WizardXiao
Copy link
Contributor Author

/test pull-e2e-kind-serial pull-e2e-kind-across-kubernetes

pkg/apis/pingcap/v1alpha1/backup_schedule.go Show resolved Hide resolved
pkg/apis/util/config/config.go Outdated Show resolved Hide resolved
pkg/apis/util/config/config.go Outdated Show resolved Hide resolved
Copy link
Contributor

@grovecai grovecai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@WizardXiao
Copy link
Contributor Author

/test pull-e2e-kind

1 similar comment
@WizardXiao
Copy link
Contributor Author

/test pull-e2e-kind

@WizardXiao WizardXiao merged commit c92ff56 into pingcap:master Mar 7, 2023
WizardXiao added a commit to WizardXiao/tidb-operator that referenced this pull request Mar 10, 2023
* init schedule log backup code

* add run log backup code

* update api

* refine some nodes

* refine cacluate logic

* add ut

* fix make check

* add log backup test

* refine code

* fix notes

* refine function names

* fix conflict
WizardXiao added a commit that referenced this pull request Mar 11, 2023
* feat: support tiflash backup and restore during volume snapshot (#4812)

* feat: calc the backup size from snapshot storage usage (#4819)

* fix backup failed when pod was auto restarted by k8s (#4883)

* init code for test

* just clean before backup data

* delete test code

* import pingcap/errors

* add check version

* remove test code

* add running status check

* add restart condition to clarify logic

* fix status update

* fix ut

* br: ensure pvc names sequential for ebs restore (#4888)

* BR: Restart backup when backup job/pod unexpected failed by k8s (#4895)

* init code for test

* just clean before backup data

* delete test code

* import pingcap/errors

* add check version

* remove test code

* add running status check

* add restart condition to clarify logic

* fix status update

* fix ut

* init code

* update crd reference

* fix miss update retry count

* add retry limit as constant

* init runnable code

* refine main controller logic

* add some note

* address some comments

* init e2e test code

* add e2e env to extend backup time

* add e2e env for test

* fix complie

* just test kill pod

* refine logic

* use pkill to kill pod

* fix reconcile

* add kill pod log

* add more log

* add more log

* try kill pod only

* wait and kill running backup pod

* add wait for pod failed

* fix wait pod running

* use killall backup to kill pod

* use pkill -9 backup

* kill pod until pod is failed

* add ps to debug

* connect commands by semicolon

* kill pod by signal 15

* use panic simulate kill pod

* test all kill pod test

* remove useless log

* add original reason of job or pod failure

* rename BackupRetryFailed to BackupRetryTheFailed

* BR: Auto truncate log backup in backup schedule (#4904)

* init schedule log backup code

* add run log backup code

* update api

* refine some nodes

* refine cacluate logic

* add ut

* fix make check

* add log backup test

* refine code

* fix notes

* refine function names

* fix conflict

* fix: add a new check for encryption during the volume snapshot restore (#4914)

* br: volume-snapshot may lead to a panic when there is no block change between two snapshot (#4922)

* br: refine BackoffRetryPolicy time format (#4925)

* refine BackoffRetryPolicy time format

* fix some ut

---------

Co-authored-by: fengou1 <85682690+fengou1@users.noreply.github.com>
Co-authored-by: WangLe1321 <wangle1321@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants