Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

planner: Do not pushdown unsupported function to tiflash in TopN operator #19363

Merged
merged 2 commits into from Aug 25, 2020
Merged

planner: Do not pushdown unsupported function to tiflash in TopN operator #19363

merged 2 commits into from Aug 25, 2020

Conversation

windtalker
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #19359

Problem Summary:
As the title says.

What is changed and how it works?

Proposal: xxx

What's Changed:

Get correct store type from cop task in PhysicalTopN.canPushDown

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Integration test
  • Manual test (add detailed scripts or steps below)

Side effects

Release note

  • No release note

@windtalker windtalker requested a review from a team as a code owner August 21, 2020 08:26
@windtalker windtalker requested review from lzmhhh123 and removed request for a team August 21, 2020 08:26
lzmhhh123
lzmhhh123 reviewed Aug 21, 2020
View reviewed changes
Copy link
Member

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 21, 2020
SunRunAway
SunRunAway reviewed Aug 24, 2020
View reviewed changes
Copy link
Contributor

@SunRunAway SunRunAway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot removed the status/LGT1 Indicates that a PR has LGTM 1. label Aug 24, 2020
@ti-srebot ti-srebot added the status/LGT2 Indicates that a PR has LGTM 2. label Aug 24, 2020
@SunRunAway SunRunAway added the sig/planner SIG: Planner label Aug 24, 2020
@SunRunAway
Copy link
Contributor

@windtalker Please refine the PR title.

@zz-jason
Copy link
Member

@windtalker please follow the PR title format to refine the PR title:

Thank you for contributing to TiDB!

PR Title Format:

  1. pkg [, pkg2, pkg3]: what's changed
  2. *: what's changed

@zz-jason
Copy link
Member

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 25, 2020
@ti-srebot
Copy link
Contributor

Your auto merge job has been accepted, waiting for:

  • 19139
  • 19379

@SunRunAway SunRunAway changed the title Do not pushdown unsupported function to tiflash in TopN operator planner: Do not pushdown unsupported function to tiflash in TopN operator Aug 25, 2020
@SunRunAway SunRunAway closed this Aug 25, 2020
@SunRunAway SunRunAway reopened this Aug 25, 2020
@SunRunAway
Copy link
Contributor

/merge

@ti-srebot
Copy link
Contributor

@SunRunAway Oops! auto merge is restricted to Committers of the SIG.See the corresponding SIG page for more information. Related SIGs: planner(slack).

@ti-srebot
Copy link
Contributor

/run-all-tests

@zz-jason
Copy link
Member

/merge

@ti-srebot
Copy link
Contributor

Your auto merge job has been accepted, waiting for:

  • 19441
  • 19308
  • 18697
  • 19154
  • 19020
  • 19139
  • 19379

@ti-srebot
Copy link
Contributor

@windtalker merge failed.

@lzmhhh123
Copy link
Member

Data race at:

[2020-08-25T09:13:17.444Z] WARNING: DATA RACE

[2020-08-25T09:13:17.444Z] Read at 0x00c06769eca0 by goroutine 1045:

[2020-08-25T09:13:17.444Z]   github.com/pingcap/tidb/executor.ResetContextOfStmt()

[2020-08-25T09:13:17.444Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/executor.go:1528 +0x4b0

[2020-08-25T09:13:17.444Z]   github.com/pingcap/tidb/session.(*session).ExecuteStmt()

[2020-08-25T09:13:17.444Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:1140 +0x1f1

[2020-08-25T09:13:17.444Z]   github.com/pingcap/tidb/session.(*session).Execute()

[2020-08-25T09:13:17.444Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:1081 +0x3ee

[2020-08-25T09:13:17.444Z]   github.com/pingcap/tidb/session.execRestrictedSQL()

[2020-08-25T09:13:17.444Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:835 +0x15c

[2020-08-25T09:13:17.444Z]   github.com/pingcap/tidb/session.(*session).ExecRestrictedSQLWithContext()

[2020-08-25T09:13:17.444Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:774 +0x25b

[2020-08-25T09:13:17.444Z]   github.com/pingcap/tidb/session.(*session).ExecRestrictedSQL()

[2020-08-25T09:13:17.444Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:744 +0x92

[2020-08-25T09:13:17.444Z]   github.com/pingcap/tidb/bindinfo.(*BindHandle).Update()

[2020-08-25T09:13:17.444Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/bindinfo/handle.go:137 +0x16c

[2020-08-25T09:13:17.444Z]   github.com/pingcap/tidb/domain.(*Domain).globalBindHandleWorkerLoop.func1()

[2020-08-25T09:13:17.444Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/domain/domain.go:938 +0x254

[2020-08-25T09:13:17.444Z] 

[2020-08-25T09:13:17.444Z] Previous write at 0x00c06769eca0 by goroutine 266:

[2020-08-25T09:13:17.444Z]   github.com/pingcap/tidb/executor_test.(*testSuiteJoinSerial).TestIssue18070.func1()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/join_test.go:2084 +0x70

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/executor_test.(*testSuiteJoinSerial).TestIssue18070()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/join_test.go:2103 +0x61a

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/executor_test.(*testSuiteJoinSerial).TestIssue18070()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/join_test.go:2089 +0x25f

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/executor_test.(*testSuiteJoinSerial).TestIssue18070()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/join_test.go:2088 +0x228

[2020-08-25T09:13:17.445Z]   runtime.call32()

[2020-08-25T09:13:17.445Z]       /usr/local/go/src/runtime/asm_amd64.s:539 +0x3a

[2020-08-25T09:13:17.445Z]   reflect.Value.Call()

[2020-08-25T09:13:17.445Z]       /usr/local/go/src/reflect/value.go:321 +0xd3

[2020-08-25T09:13:17.445Z]   github.com/pingcap/check.(*suiteRunner).forkTest.func1()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:850 +0x9aa

[2020-08-25T09:13:17.445Z]   github.com/pingcap/check.(*suiteRunner).forkCall.func1()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:739 +0x113

[2020-08-25T09:13:17.445Z] 

[2020-08-25T09:13:17.445Z] Goroutine 1045 (running) created at:

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/domain.(*Domain).globalBindHandleWorkerLoop()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/domain/domain.go:925 +0x6c

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/domain.(*Domain).LoadBindInfoLoop()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/domain/domain.go:918 +0x17a

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.BootstrapSession()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:1819 +0x388

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1233 +0x433

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1231 +0x406

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1229 +0x3d9

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1227 +0x3ac

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1225 +0x37f

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1223 +0x352

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1221 +0x325

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1219 +0x2f8

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1217 +0x2cb

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1215 +0x29e

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.445Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1213 +0x271

[2020-08-25T09:13:17.445Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1211 +0x244

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1209 +0x217

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1207 +0x1ea

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1205 +0x1bd

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1204 +0x190

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1203 +0x163

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1202 +0x136

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1200 +0x109

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1198 +0xdc

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/session.doDDLWorks()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:1196 +0x5b

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/session.bootstrap()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/bootstrap.go:338 +0x1a1

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/session.runInBootstrapSession()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:1885 +0xe2

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/session.BootstrapSession()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:1779 +0xb69

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/executor_test.(*baseTestSuite).SetUpSuite()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/executor_test.go:172 +0x27d

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/executor_test.(*testSuiteJoinSerial).SetUpSuite()

[2020-08-25T09:13:17.446Z]       <autogenerated>:1 +0x63

[2020-08-25T09:13:17.446Z]   runtime.call32()

[2020-08-25T09:13:17.446Z]       /usr/local/go/src/runtime/asm_amd64.s:539 +0x3a

[2020-08-25T09:13:17.446Z]   reflect.Value.Call()

[2020-08-25T09:13:17.446Z]       /usr/local/go/src/reflect/value.go:321 +0xd3

[2020-08-25T09:13:17.446Z]   github.com/pingcap/check.(*suiteRunner).runFixture.func1()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:799 +0x195

[2020-08-25T09:13:17.446Z]   github.com/pingcap/check.(*suiteRunner).forkCall.func1()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:739 +0x113

[2020-08-25T09:13:17.446Z] 

[2020-08-25T09:13:17.446Z] Goroutine 266 (finished) created at:

[2020-08-25T09:13:17.446Z]   github.com/pingcap/check.(*suiteRunner).forkCall()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:734 +0x4a3

[2020-08-25T09:13:17.446Z]   github.com/pingcap/check.(*suiteRunner).forkTest()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:832 +0x1b9

[2020-08-25T09:13:17.446Z]   github.com/pingcap/check.(*suiteRunner).doRun()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:666 +0x13a

[2020-08-25T09:13:17.446Z]   github.com/pingcap/check.(*suiteRunner).run()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:696 +0x13f

[2020-08-25T09:13:17.446Z]   github.com/pingcap/check.Run()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/run.go:150 +0x5a

[2020-08-25T09:13:17.446Z]   github.com/pingcap/check.RunAll()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/run.go:136 +0x906

[2020-08-25T09:13:17.446Z]   github.com/pingcap/check.TestingT()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/run.go:99 +0x751

[2020-08-25T09:13:17.446Z]   github.com/pingcap/tidb/executor_test.TestT()

[2020-08-25T09:13:17.446Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/executor_test.go:96 +0x263

[2020-08-25T09:13:17.446Z]   testing.tRunner()

[2020-08-25T09:13:17.447Z]       /usr/local/go/src/testing/testing.go:909 +0x199

@XuHuaiyu PTAL.

@lzmhhh123 lzmhhh123 merged commit e52cb37 into pingcap:master Aug 25, 2020
windtalker added a commit to windtalker/tidb that referenced this pull request Aug 31, 2020
@windtalker @ti-srebot
planner: Do not pushdown unsupported function to tiflash in TopN oper…
380a2d4 
…ator (pingcap#19363)

Co-authored-by: ti-srebot <66930949+ti-srebot@users.noreply.github.com>
@elvizlai
Copy link
Contributor

elvizlai commented Sep 1, 2020

seems not released with v4.0.5

@windtalker windtalker mentioned this pull request Sep 1, 2020
Merged
@windtalker windtalker deleted the fix_topn_to_tiflash branch December 28, 2020 04:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SunRunAway SunRunAway SunRunAway left review comments

@lzmhhh123 lzmhhh123 lzmhhh123 left review comments

@ti-srebot ti-srebot ti-srebot approved these changes

Assignees
No one assigned
Labels
sig/planner SIG: Planner status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Query on TiFlash throw CastStringAsInt is not supported error
6 participants
@windtalker @SunRunAway @zz-jason @ti-srebot @lzmhhh123 @elvizlai