Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner,executor: support `admin show slow` command #7785

Merged
merged 8 commits into from Sep 26, 2018

Conversation

Projects
None yet
5 participants
@tiancaiamao
Copy link
Contributor

tiancaiamao commented Sep 26, 2018

What problem does this PR solve?

After this PR, together with #7688 and #7692, we can use admin show slow now.

What is changed and how it works?

Build ShowSlow plan and ShowSlowExec executor.

It works like this:

mysql> admin show slow top 3;
+--------------------------------+----------------------------+----------+------------------------------------------------------------------------------------------------------+------+---------+--------------------+----------------+------+-----------+-----------+----------+
| SQL                            | START                      | DURATION | DETAILS                                                                                              | SUCC | CONN_ID | TRANSACTION_TS     | USER           | DB   | TABLE_IDS | INDEX_IDS | INTERNAL |
+--------------------------------+----------------------------+----------+------------------------------------------------------------------------------------------------------+------+---------+--------------------+----------------+------+-----------+-----------+----------+
| select count(*) from t limit 5 | 2018-09-26 16:09:21.101256 | 00:00:09 | process_time:1m8.032s wait_time:31.62s request_count:14 total_keys:20000014 processed_keys:20000000  |    1 |       2 | 403164197311217665 | root@127.0.0.1 | test | [32]      |           |        1 |
| select count(*) from t         | 2018-09-26 16:09:11.269202 | 00:00:08 | process_time:1m2.451s wait_time:29.444s request_count:14 total_keys:20000014 processed_keys:20000000 |    1 |       1 | 403164194729099265 | root@127.0.0.1 | test | [32]      |           |        1 |
+--------------------------------+----------------------------+----------+------------------------------------------------------------------------------------------------------+------+---------+--------------------+----------------+------+-----------+-----------+----------+
2 rows in set (0.01 sec)

Check List

Tests

  • Unit test to cover the basic statement.

As the result fields always change, it's hard to put it into unit test.
I manually check the result to assure that works.

Related changes

  • Need to update the documentation
  • Need to be included in the release note

@tiancaiamao tiancaiamao changed the title support `admin show slow` command planner,executor: support `admin show slow` command Sep 26, 2018

@tiancaiamao

This comment has been minimized.

Copy link
Contributor Author

tiancaiamao commented Sep 26, 2018

@winkyao

This comment has been minimized.

Copy link
Member

winkyao commented Sep 26, 2018

Please fix ci.


dom := domain.GetDomain(e.ctx)
slowQueries := dom.ShowSlowQuery(e.ShowSlow)
for _, slow := range slowQueries {

This comment has been minimized.

Copy link
@zz-jason

zz-jason Sep 26, 2018

Member

the number of rows appended to chk should be smaller than max_chunk_size

This comment has been minimized.

Copy link
@tiancaiamao

tiancaiamao Sep 26, 2018

Author Contributor

So if the result is larger than max_chunk_size, it has to be split into multiple batches?

This comment has been minimized.

Copy link
@zz-jason

This comment has been minimized.

Copy link
@zz-jason

zz-jason Sep 26, 2018

Member

we have to use a cursor to record how many slow queries have been returned to the client.

@@ -746,6 +750,23 @@ func buildShowDDLJobQueriesFields() *expression.Schema {
return schema
}

func buildShowSlowSchema() *expression.Schema {
schema := expression.NewSchema(make([]*expression.Column, 0, 11)...)
schema.Append(buildColumn("", "SQL", mysql.TypeVarchar, 256))

This comment has been minimized.

Copy link
@coocood

coocood Sep 26, 2018

Member

length is too small.

This comment has been minimized.

Copy link
@tiancaiamao

tiancaiamao Sep 26, 2018

Author Contributor

How about 1024?

This comment has been minimized.

Copy link
@winkyao

winkyao Sep 26, 2018

Member

1024 still too small.

schema.Append(buildColumn("", "TRANSACTION_TS", mysql.TypeLonglong, 4))
schema.Append(buildColumn("", "USER", mysql.TypeVarchar, 32))
schema.Append(buildColumn("", "DB", mysql.TypeVarchar, 64))
schema.Append(buildColumn("", "TABLE_IDS", mysql.TypeVarchar, 64))

This comment has been minimized.

Copy link
@coocood

coocood Sep 26, 2018

Member

length is too small.

schema.Append(buildColumn("", "USER", mysql.TypeVarchar, 32))
schema.Append(buildColumn("", "DB", mysql.TypeVarchar, 64))
schema.Append(buildColumn("", "TABLE_IDS", mysql.TypeVarchar, 64))
schema.Append(buildColumn("", "INDEX_IDS", mysql.TypeVarchar, 64))

This comment has been minimized.

Copy link
@coocood

coocood Sep 26, 2018

Member

length is too small.

schema.Append(buildColumn("", "DETAILS", mysql.TypeVarchar, 256))
schema.Append(buildColumn("", "SUCC", mysql.TypeTiny, 2))
schema.Append(buildColumn("", "CONN_ID", mysql.TypeLonglong, 4))
schema.Append(buildColumn("", "TRANSACTION_TS", mysql.TypeLonglong, 4))

This comment has been minimized.

Copy link
@coocood

coocood Sep 26, 2018

Member

length is too small

schema.Append(buildColumn("", "DURATION", mysql.TypeDuration, 64))
schema.Append(buildColumn("", "DETAILS", mysql.TypeVarchar, 256))
schema.Append(buildColumn("", "SUCC", mysql.TypeTiny, 2))
schema.Append(buildColumn("", "CONN_ID", mysql.TypeLonglong, 4))

This comment has been minimized.

Copy link
@coocood

coocood Sep 26, 2018

Member

length is too small.

func buildShowSlowSchema() *expression.Schema {
schema := expression.NewSchema(make([]*expression.Column, 0, 11)...)
schema.Append(buildColumn("", "SQL", mysql.TypeVarchar, 256))
schema.Append(buildColumn("", "START", mysql.TypeTimestamp, 64))

This comment has been minimized.

Copy link
@coocood

coocood Sep 26, 2018

Member

length is too big

This comment has been minimized.

Copy link
@tiancaiamao

tiancaiamao Sep 26, 2018

Author Contributor

32?

schema := expression.NewSchema(make([]*expression.Column, 0, 11)...)
schema.Append(buildColumn("", "SQL", mysql.TypeVarchar, 256))
schema.Append(buildColumn("", "START", mysql.TypeTimestamp, 64))
schema.Append(buildColumn("", "DURATION", mysql.TypeDuration, 64))

This comment has been minimized.

Copy link
@coocood

coocood Sep 26, 2018

Member

length is too big

@@ -381,6 +381,12 @@ func (a *ExecStmt) logSlowQuery(txnTS uint64, succ bool) {
if user != nil {
userString = user.String()
}
if len(tableIDs) > 10 {

This comment has been minimized.

Copy link
@winkyao

winkyao Sep 26, 2018

Member

what about if len(tableIDs) > len("table_ids:") ?

This comment has been minimized.

Copy link
@tiancaiamao

tiancaiamao Sep 26, 2018

Author Contributor

There is a comment for it already.

tk.MustQuery(`select sleep(1)`)

// Collecting slow queries is asynchronous, wait a while to ensure it's done.
time.Sleep(5 * time.Millisecond)

This comment has been minimized.

Copy link
@winkyao

winkyao Sep 26, 2018

Member

Is 5ms enough?

This comment has been minimized.

Copy link
@coocood

coocood Sep 26, 2018

Member

Maybe add a retry to make the test stable?

tk.MustExec("use test")
tk.MustExec(`drop table if exists t`)
tk.MustExec(`create table t(a bigint)`)
tk.MustQuery(`select sleep(1)`)

This comment has been minimized.

Copy link
@winkyao

winkyao Sep 26, 2018

Member

We can't assume sleep 1s is a slow query, we need to use cfg.Log.SlowThreshold?

@zz-jason
Copy link
Member

zz-jason left a comment

LGTM

@coocood
Copy link
Member

coocood left a comment

LGTM

@coocood

This comment has been minimized.

Copy link
Member

coocood commented Sep 26, 2018

/run-all-tests

@tiancaiamao tiancaiamao merged commit 081920d into pingcap:master Sep 26, 2018

4 checks passed

ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
jenkins-ci-tidb/build Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details

@tiancaiamao tiancaiamao deleted the tiancaiamao:show-log branch Sep 26, 2018

dbjoa added a commit to cloud-pi/tidb that referenced this pull request Oct 1, 2018

@morgo

This comment has been minimized.

Copy link
Member

morgo commented Oct 5, 2018

I just started playing with this today :-)

@tiancaiamao Is it possible to change the duration to be in ms (unformatted)? 1 second is quite a long time for OLTP, and a query could still be considered slow but show a 00:00:00 duration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.