Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: implement IterReverse for tikvSnapshot and use desc scan to get latest N ddl history jobs. #10152

Merged
merged 31 commits into from
May 31, 2019

Conversation

crazycs520
Copy link
Contributor

@crazycs520 crazycs520 commented Apr 15, 2019

What problem does this PR solve?

Currently admin show ddl jobs get all ddl history jobs, but we only need the latest N ddl history jobs. So we can use desc scan to get latest N ddl history jobs.

What is changed and how it works?

  1. Implement IterReverse for tikvSnapshot.
  2. Use IterReverse to get latest ddl history jobs for admin show ddl jobs

Compare
There are 17083 ddl history jobs.

mysql root@localhost:test> admin show ddl jobs 100000
17083 rows in set
Time: 2.323s

Before spend 0.5s.

mysql root@localhost:test> admin show ddl jobs 10
+--------+---------+------------+--------------+--------------+-----------+----------+-----------+-----------------------------------+--------+
| JOB_ID | DB_NAME | TABLE_NAME | JOB_TYPE     | SCHEMA_STATE | SCHEMA_ID | TABLE_ID | ROW_COUNT | START_TIME                        | STATE  |
+--------+---------+------------+--------------+--------------+-----------+----------+-----------+-----------------------------------+--------+
| 34165  | test23  | t2317      | create table | public       | 15586     | 34164    | 0         | 2019-04-15 19:44:45.005 +0800 CST | synced |
| 34163  | test22  | t2340      | create table | public       | 15456     | 34162    | 0         | 2019-04-15 19:44:44.955 +0800 CST | synced |
| 34161  | test20  | t2421      | create table | public       | 15238     | 34160    | 0         | 2019-04-15 19:44:44.854 +0800 CST | synced |
| 34159  | test21  | t2382      | create table | public       | 15310     | 34158    | 0         | 2019-04-15 19:44:44.754 +0800 CST | synced |
| 34157  | test23  | t2316      | create table | public       | 15586     | 34156    | 0         | 2019-04-15 19:44:44.654 +0800 CST | synced |
| 34155  | test22  | t2339      | create table | public       | 15456     | 34154    | 0         | 2019-04-15 19:44:44.458 +0800 CST | synced |
| 34153  | test20  | t2420      | create table | public       | 15238     | 34152    | 0         | 2019-04-15 19:44:43.854 +0800 CST | synced |
| 34151  | test21  | t2381      | create table | public       | 15310     | 34150    | 0         | 2019-04-15 19:44:43.706 +0800 CST | synced |
| 34149  | test23  | t2315      | create table | public       | 15586     | 34148    | 0         | 2019-04-15 19:44:43.654 +0800 CST | synced |
| 34147  | test22  | t2338      | create table | public       | 15456     | 34146    | 0         | 2019-04-15 19:44:43.506 +0800 CST | synced |
+--------+---------+------------+--------------+--------------+-----------+----------+-----------+-----------------------------------+--------+
10 rows in set
Time: 0.517s

In This PR spend 0.01s.

mysql root@localhost:test> admin show ddl jobs
+--------+---------+------------+--------------+--------------+-----------+----------+-----------+-----------------------------------+--------+
| JOB_ID | DB_NAME | TABLE_NAME | JOB_TYPE     | SCHEMA_STATE | SCHEMA_ID | TABLE_ID | ROW_COUNT | START_TIME                        | STATE  |
+--------+---------+------------+--------------+--------------+-----------+----------+-----------+-----------------------------------+--------+
| 34165  | test23  | t2317      | create table | public       | 15586     | 34164    | 0         | 2019-04-15 19:44:45.005 +0800 CST | synced |
| 34163  | test22  | t2340      | create table | public       | 15456     | 34162    | 0         | 2019-04-15 19:44:44.955 +0800 CST | synced |
| 34161  | test20  | t2421      | create table | public       | 15238     | 34160    | 0         | 2019-04-15 19:44:44.854 +0800 CST | synced |
| 34159  | test21  | t2382      | create table | public       | 15310     | 34158    | 0         | 2019-04-15 19:44:44.754 +0800 CST | synced |
| 34157  | test23  | t2316      | create table | public       | 15586     | 34156    | 0         | 2019-04-15 19:44:44.654 +0800 CST | synced |
| 34155  | test22  | t2339      | create table | public       | 15456     | 34154    | 0         | 2019-04-15 19:44:44.458 +0800 CST | synced |
| 34153  | test20  | t2420      | create table | public       | 15238     | 34152    | 0         | 2019-04-15 19:44:43.854 +0800 CST | synced |
| 34151  | test21  | t2381      | create table | public       | 15310     | 34150    | 0         | 2019-04-15 19:44:43.706 +0800 CST | synced |
| 34149  | test23  | t2315      | create table | public       | 15586     | 34148    | 0         | 2019-04-15 19:44:43.654 +0800 CST | synced |
| 34147  | test22  | t2338      | create table | public       | 15456     | 34146    | 0         | 2019-04-15 19:44:43.506 +0800 CST | synced |
+--------+---------+------------+--------------+--------------+-----------+----------+-----------+-----------------------------------+--------+
10 rows in set
Time: 0.010s

Check List

Tests

  • Unit test

Code changes

Side effects

Related changes

@crazycs520
Copy link
Contributor Author

@MyonKeminta @tiancaiamao @winkyao PTAL

meta/meta.go Outdated Show resolved Hide resolved
store/mockstore/mocktikv/rpc.go Show resolved Hide resolved
store/tikv/scan.go Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented May 24, 2019

Codecov Report

Merging #10152 into master will increase coverage by 0.0224%.
The diff coverage is 86.4077%.

@@               Coverage Diff                @@
##             master     #10152        +/-   ##
================================================
+ Coverage   78.2253%   78.2477%   +0.0224%     
================================================
  Files           413        413                
  Lines         87542      87623        +81     
================================================
+ Hits          68480      68563        +83     
+ Misses        13933      13922        -11     
- Partials       5129       5138         +9

@crazycs520
Copy link
Contributor Author

@winkyao PTAL again.

store/mockstore/mocktikv/rpc.go Outdated Show resolved Hide resolved
store/mockstore/mocktikv/cluster.go Outdated Show resolved Hide resolved
Copy link
Contributor

@zimulala zimulala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zimulala zimulala added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels May 30, 2019
@crazycs520
Copy link
Contributor Author

/run-all-tests

@crazycs520
Copy link
Contributor Author

/rebuild

@crazycs520
Copy link
Contributor Author

/run-all-tests

Copy link
Contributor

@winkyao winkyao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@winkyao winkyao merged commit 8599fee into pingcap:master May 31, 2019
crazycs520 added a commit to crazycs520/tidb that referenced this pull request Jun 4, 2019
zz-jason pushed a commit that referenced this pull request Jun 4, 2019
crazycs520 added a commit to crazycs520/tidb that referenced this pull request Aug 20, 2019
@you06 you06 added the sig/sql-infra SIG: SQL Infra label Mar 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/sql-infra SIG: SQL Infra status/LGT2 Indicates that a PR has LGTM 2. type/enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants