Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

domain,executor: store topN slow query in domain #7646

Merged
merged 13 commits into from Sep 12, 2018

Conversation

@tiancaiamao
Copy link
Contributor

tiancaiamao commented Sep 8, 2018

What problem does this PR solve?

Store topN slow query in domain, so later we can retrieve it later.

What is changed and how it works?

logSlowQuery in session will send a copy to domain, the domain maintains a heap to store the
topN recent slow queries.

Check List

Tests

  • Unit test
@shenli

This comment has been minimized.

Copy link
Member

shenli commented Sep 9, 2018

Why put it in the domain?

@winkyao

This comment has been minimized.

Copy link
Member

winkyao commented Sep 10, 2018

If put it in domain, it may lead to cycle import?

@tiancaiamao tiancaiamao removed the status/WIP label Sep 10, 2018
@tiancaiamao

This comment has been minimized.

Copy link
Contributor Author

tiancaiamao commented Sep 10, 2018

There is no cycle import. executor imports domain, domain doesn't imports executor. @winkyao

So if we don't put it in domain, where should we put it? @shenli

@@ -471,6 +514,7 @@ func NewDomain(store kv.Storage, ddlLease time.Duration, statsLease time.Duratio
sysSessionPool: pools.NewResourcePool(factory, capacity, capacity, resourceIdleTimeout),
statsLease: statsLease,
infoHandle: infoschema.NewHandle(store),
slowQuery: newTopNSlowQuery(30, time.Hour*24*7),

This comment has been minimized.

Copy link
@coocood

coocood Sep 10, 2018

Member

This should be configurable.

This comment has been minimized.

Copy link
@tiancaiamao

tiancaiamao Sep 10, 2018

Author Contributor

I agree, but we can do it in another PR.

}
}

func (q *topNSlowQuery) shiftUp(end int) {

This comment has been minimized.

Copy link
@coocood

coocood Sep 10, 2018

Member

siftUp?

@coocood

This comment has been minimized.

Copy link
Member

coocood commented Sep 10, 2018

This is not a performance hot spot, we can just use heap in the standard library.

tiancaiamao added 2 commits Sep 10, 2018
@tiancaiamao

This comment has been minimized.

Copy link
Contributor Author

tiancaiamao commented Sep 10, 2018


// topNSlowQuery maintains a heap to store recent slow queries.
// N = 30, recent = 7 days by default.
type topNSlowQuery struct {

This comment has been minimized.

Copy link
@coocood

coocood Sep 10, 2018

Member

There are multiple query entries, so I think topNSlowQueries is better.

close(q.ch)
}

func (q *topNSlowQuery) Push(info *slowQueryInfo) {

This comment has been minimized.

Copy link
@coocood

coocood Sep 10, 2018

Member

Why not implement the Heap interface in this type?
For the name confliction, we can change this Push to Add or Append.

@tiancaiamao

This comment has been minimized.

Copy link
Contributor Author

tiancaiamao commented Sep 10, 2018

PTAL @coocood

@@ -371,6 +372,13 @@ func (a *ExecStmt) logSlowQuery(txnTS uint64, succ bool) {
logutil.SlowQueryLogger.Warnf(
"[SLOW_QUERY] %vcost_time:%v %s succ:%v con:%v user:%s txn_start_ts:%v database:%v %v%vsql:%v",
internal, costTime, sessVars.StmtCtx.GetExecDetails(), succ, connID, user, txnTS, currentDB, tableIDs, indexIDs, sql)
if !sessVars.InRestrictedSQL {

This comment has been minimized.

Copy link
@winkyao

winkyao Sep 11, 2018

Member

Just log general sql? I prefer to keep two heap to log the general sql and internal sql.

This comment has been minimized.

Copy link
@tiancaiamao

tiancaiamao Sep 11, 2018

Author Contributor

Internal SQL are always the same, it will not give us too much information.

}

func (do *Domain) topNSlowQueryLoop() {
defer do.wg.Done()

This comment has been minimized.

Copy link
@winkyao

winkyao Sep 11, 2018

Member

recover this goroutine.

}

// Rebuild the heap.
q.data = q.data[:idx]

This comment has been minimized.

Copy link
@winkyao

winkyao Sep 11, 2018

Member

You must use a lock to protect q.data, as long as you need to read the slice later.

This comment has been minimized.

Copy link
@coocood

coocood Sep 11, 2018

Member

We can do reading in the same goroutine.

This comment has been minimized.

Copy link
@tiancaiamao

tiancaiamao Sep 11, 2018

Author Contributor

I'll use copy on read, and they will be in one goroutine, no lock. @winkyao

q.data = append(q.data, x.(*slowQueryInfo))
}

func (q *topNSlowQueries) Pop() interface{} {

This comment has been minimized.

Copy link
@winkyao

winkyao Sep 11, 2018

Member

Pop can only return the minimum duration query, how can we implement topn, for example, the n is 30, and I wanna get top 3 query. and how can we just peek the heap?

This comment has been minimized.

Copy link
@winkyao

winkyao Sep 11, 2018

Member

Maybe a b-tree is better?

This comment has been minimized.

Copy link
@coocood

coocood Sep 11, 2018

Member

The read operation is not implemented in this PR.
Reading doesn't need to call Pop()

This comment has been minimized.

Copy link
@tiancaiamao

tiancaiamao Sep 11, 2018

Author Contributor

Read will not be a frequent operation, so we just copy on read.
Get top 3 query is easy, copy the origin heap, Pop Pop Pop.

@tiancaiamao

This comment has been minimized.

Copy link
Contributor Author

tiancaiamao commented Sep 11, 2018

@coocood

This comment has been minimized.

Copy link
Member

coocood commented Sep 11, 2018

LGTM

Copy link
Member

winkyao left a comment

LGTM

Copy link
Member

zhexuany left a comment

LGTM

@@ -329,6 +331,48 @@ func (do *Domain) Reload() error {
return nil
}

// LogTopNSlowQuery keeps topN recent slow queries in domain.
func (do *Domain) LogTopNSlowQuery(sql string, start time.Time, duration time.Duration,

This comment has been minimized.

Copy link
@zz-jason

zz-jason Sep 11, 2018

Member

This function takes so many parameters, which makes it hard to read and maintain, could you extract a struct to store all the parameters and pass the struct as the parameter instead?

This comment has been minimized.

Copy link
@lysu

lysu Sep 11, 2018

Member

maybe slowQueryInfo is 🐶

This comment has been minimized.

Copy link
@zz-jason

zz-jason Sep 11, 2018

Member

how about exporting slowQueryInfo and use slowQueryInfo instead?

}

func (h *slowQueryHeap) Len() int { return len(h.data) }
func (h *slowQueryHeap) Less(i, j int) bool { return h.data[i].duration < h.data[j].duration }

This comment has been minimized.

Copy link
@zz-jason

zz-jason Sep 11, 2018

Member

should it be h.data[i].duration > h.data[j].duration?

This comment has been minimized.

Copy link
@tiancaiamao

tiancaiamao Sep 11, 2018

Author Contributor

It's Less ... @zz-jason

This comment has been minimized.

Copy link
@zz-jason

zz-jason Sep 11, 2018

Member

use heap[len(heap)-1] to store the slowest query?

This comment has been minimized.

Copy link
@coocood

coocood Sep 11, 2018

Member

heap[0] is the fastest slow query, heap[len(heap)-1] may not be the slowest query.

Copy link
Member

zz-jason left a comment

has unresolved comments

@tiancaiamao tiancaiamao dismissed stale reviews from zhexuany and winkyao via d88c674 Sep 11, 2018
@tiancaiamao tiancaiamao force-pushed the tiancaiamao:topn-slow-query branch from c9db110 to d88c674 Sep 11, 2018
@tiancaiamao

This comment has been minimized.

Copy link
Contributor Author

tiancaiamao commented Sep 11, 2018

PTAL @zz-jason

@zz-jason

This comment has been minimized.

Copy link
Member

zz-jason commented Sep 11, 2018

LGTM

@zz-jason

This comment has been minimized.

Copy link
Member

zz-jason commented Sep 11, 2018

/run-all-tests

1 similar comment
@zimulala

This comment has been minimized.

Copy link
Member

zimulala commented Sep 11, 2018

/run-all-tests

@tiancaiamao

This comment has been minimized.

Copy link
Contributor Author

tiancaiamao commented Sep 11, 2018

/run-all-tests

@zz-jason

This comment has been minimized.

Copy link
Member

zz-jason commented Sep 12, 2018

/run-all-tests

@tiancaiamao tiancaiamao merged commit 6604e33 into pingcap:master Sep 12, 2018
11 checks passed
11 checks passed
ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
jenkins-ci-tidb/build Jenkins job succeeded.
Details
jenkins-ci-tidb/common-test Jenkins job succeeded.
Details
jenkins-ci-tidb/integration-common-test Jenkins job succeeded.
Details
jenkins-ci-tidb/integration-compatibility-test Jenkins job succeeded.
Details
jenkins-ci-tidb/integration-ddl-test Jenkins job succeeded.
Details
jenkins-ci-tidb/mybatis-test Jenkins job succeeded.
Details
jenkins-ci-tidb/sqllogic-test Jenkins job succeeded.
Details
jenkins-ci-tidb/unit-test Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details
@tiancaiamao tiancaiamao deleted the tiancaiamao:topn-slow-query branch Sep 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.