Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: check gc safe time first when iterate the history ddl jobs (#18380) #18386

Merged
merged 4 commits into from
Jul 15, 2020

Conversation

ti-srebot
Copy link
Contributor

cherry-pick #18380 to release-3.0


Signed-off-by: crazycs520 crazycs520@gmail.com

What problem does this PR solve?

When requesting the /tiflash/replica HTTP API, it will iterate the history DDL job to find the dropped table that has not been GC.

Before this PR, if the TiDB cluster has not done the drop table or drop database DDL jobs, the /tiflash/replica HTTP API will iterate all DDL jobs, it would be very slow when there are many history DDL jobs.

How to avoid this problem before this PR?

You can simply create a random table and then drop it, then after the GC safe time passed, then request the /tiflash/replica HTTP API will be normal speed.

use test;
create table random_t (a int);
drop table random_t;

What is changed and how it works?

check gc safe time first when iterate the history ddl jobs.

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Manual test (add detailed scripts or steps below)
1.  execute a lot of DDL jobs (such 1W) except for `drop table/drop database`
2. wait after passed the GC safe point.
3. time curl http://{$TIDB_ADDR}/tiflash/replica 

Side effects

  • No

Release note

  • Fix the issue of request /tiflash/replica HTTP API was slow when there are many history DDL jobs in TiDB cluster.

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor Author

/run-all-tests

@crazycs520
Copy link
Contributor

/run-all-tests

@qw4990 qw4990 modified the milestones: v3.0.16, v3.0.17 Jul 10, 2020
Copy link
Contributor

@tangenta tangenta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot
Copy link
Contributor Author

@tangenta,Thanks for your review. However, LGTM is restricted to Reviewers or higher roles.See the corresponding SIG page for more information. Related SIGs: execution(slack).

Copy link
Contributor

@AilinKid AilinKid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot
Copy link
Contributor Author

@AilinKid,Thanks for your review. However, LGTM is restricted to Reviewers or higher roles.See the corresponding SIG page for more information. Related SIGs: execution(slack).

@crazycs520
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor Author

@AilinKid,Thanks for your review. However, LGTM is restricted to Reviewers or higher roles.See the corresponding SIG page for more information. Related SIGs: execution(slack).

@crazycs520
Copy link
Contributor

/run-unit-test

@SunRunAway SunRunAway merged commit 338a0bd into pingcap:release-3.0 Jul 15, 2020
@SunRunAway SunRunAway deleted the release-3.0-c13e4ab1be3e branch July 15, 2020 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution type/enhancement The issue or PR belongs to an enhancement. type/3.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants