Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refine gc strategy of pump (#646) #663

Merged
merged 4 commits into from Jul 13, 2019
Merged

refine gc strategy of pump (#646) #663

merged 4 commits into from Jul 13, 2019

Conversation

IANTHEREAL
Copy link
Collaborator

@IANTHEREAL IANTHEREAL commented Jul 2, 2019 •
edited

What problem does this PR solve?

most users forget to mark drainer offline using binlogctl after they stop drainer, It would block gc of pump, and then the pump will easily fill up the disk capacity because there are too many binlogs.

Regardless of ease of use or design, this design is unreasonable. There are two main reasons:

  • when the disk space of the pump is full, the pump will be unavailable, which will have a great impact on the data and TiDB instance.
  • drainer is a relatively separate tool and should not have too much impact on core system formation

What is changed and how it works?

  • pump only keeps binlog events whose ts <= current time - gc(days)
  • issue a critical metrics while some drainer checkpoint is old than gc ts of pump，it means binlog that drainer need is purged

Check List

Tests

  • Unit test

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation
  • Need to update the tidb-ansible repository
  • Need to be included in the release note

@IANTHEREAL
Copy link
Collaborator Author

/run-all-tests

@IANTHEREAL
Copy link
Collaborator Author

/run-all-tests

@ericsyh ericsyh requested a review from july2993 July 2, 2019 16:14
@suzaku
Copy link
Contributor

suzaku commented Jul 3, 2019

/run-integration-tests

@mahjonp
Copy link

mahjonp commented Jul 3, 2019

/run-all-tests tidb=release-2.1 tikv=release-2.1 pd=release-2.1

@suzaku
Copy link
Contributor

suzaku commented Jul 9, 2019

/run-all-tests tidb=release-2.1 tikv=release-2.1 pd=release-2.1

suzaku
suzaku reviewed Jul 9, 2019
View reviewed changes
Copy link
Contributor

@suzaku suzaku left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@IANTHEREAL
Copy link
Collaborator Author

@july2993 PTAL

july2993
july2993 approved these changes Jul 12, 2019
View reviewed changes
Copy link
Contributor

@july2993 july2993 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@IANTHEREAL IANTHEREAL merged commit 150b982 into pingcap:release-2.1 Jul 13, 2019
@IANTHEREAL IANTHEREAL deleted the cp branch July 13, 2019 04:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@suzaku suzaku suzaku left review comments

@july2993 july2993 july2993 approved these changes

Assignees
No one assigned
Labels
status/LGT2 type/cherry-pick
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@IANTHEREAL @suzaku @mahjonp @july2993