Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

planner: Log warnings when agg function can not be pushdown in explain statement #25553

Merged
merged 14 commits into from Jun 24, 2021
Merged

planner: Log warnings when agg function can not be pushdown in explain statement #25553

merged 14 commits into from Jun 24, 2021

Conversation

windtalker
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Problem Summary:
After #22713, if an expression can not be pushdown to storage layer, it will log a warning if current sql is an explain statement, however, for aggregation function agg(expr) if it can not be pushdown to storage layer becuase expr is not supported, no warning is logged because current implementation call AggFuncToPBExpr directly to check if the agg function can be pushed to storage layer.

What is changed and how it works?

Proposal: xxx

What's Changed:

Refine CheckAggCanPushCop and log warnings when agg function can not be pushdown in explain statement.
How it Works:

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM
  • Breaking backward compatibility

Release note

  • Log warnings when agg function can not be pushdown in explain statement

@windtalker windtalker requested a review from a team as a code owner June 18, 2021 04:29
@windtalker windtalker requested review from time-and-fate and removed request for a team June 18, 2021 04:29
@ti-chi-bot ti-chi-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jun 18, 2021
@windtalker windtalker changed the title Log warnings when agg function can not be pushdown in explain statement planner: Log warnings when agg function can not be pushdown in explain statement Jun 18, 2021
@sre-bot
Copy link
Contributor

sre-bot commented Jun 18, 2021

Please follow PR Title Format:

  • pkg [, pkg2, pkg3]: what's changed

Or if the count of mainly changed packages are more than 3, use

  • *: what's changed

@windtalker windtalker requested a review from a team as a code owner June 21, 2021 09:06
@windtalker windtalker requested review from XuHuaiyu and removed request for a team June 21, 2021 09:06
@windtalker
Copy link
Contributor Author

/label needs-cherry-pick-5.0

@windtalker
Copy link
Contributor Author

/label needs-cherry-pick-5.1

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 21, 2021
@github-actions github-actions bot added the sig/execution SIG execution label Jun 21, 2021
@fzhedu
Copy link
Contributor

fzhedu commented Jun 22, 2021

/run-all-tests

fzhedu
fzhedu reviewed Jun 22, 2021
View reviewed changes
executor/show_test.go
Comment on lines +152 to +153
tk.MustExec("explain select max(date_add(value, interval 1 day)) from show_warnings_expr_pushdown group by a")
c.Assert(tk.Se.GetSessionVars().StmtCtx.WarningCount(), Equals, uint16(2))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if the unsupported function occur in group by?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and distinct?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add tests about groupBy and distinct.

@ti-chi-bot ti-chi-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 24, 2021
@ti-chi-bot ti-chi-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 24, 2021
fzhedu
fzhedu approved these changes Jun 24, 2021
View reviewed changes
Copy link
Contributor

@fzhedu fzhedu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/LGTM

@ti-chi-bot
Copy link
Member

@fzhedu: Please use GitHub review feature instead of /lgtm [cancel] when you want to submit review to the pull request.
For how to use GitHub review feature, see also this document provided by GitHub.

For the reason we drop support to the commands, see also this page.
This reply is being used as a temporary reply during the migration of review process and will be removed on July 1st.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • fzhedu
  • hanfei1991

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 24, 2021
@fzhedu
Copy link
Contributor

fzhedu commented Jun 24, 2021

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 23e3fcc

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 24, 2021
@ti-chi-bot ti-chi-bot merged commit 27489d4 into pingcap:master Jun 24, 2021
ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Jun 24, 2021
@windtalker @ti-srebot
cherry pick pingcap#25553 to release-5.0
a162e06 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot ti-srebot mentioned this pull request Jun 24, 2021
Merged
@ti-srebot
Copy link
Contributor

cherry pick to release-5.0 in PR #25736

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Jun 24, 2021
@windtalker @ti-srebot
cherry pick pingcap#25553 to release-5.1
d196c2e 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot ti-srebot mentioned this pull request Jun 24, 2021
Merged
@ti-srebot
Copy link
Contributor

cherry pick to release-5.1 in PR #25737

@tisonkun tisonkun mentioned this pull request Jun 24, 2021
Closed
jiajunhuang pushed a commit to jiajunhuang/tidb that referenced this pull request Jun 28, 2021
@windtalker @jiajunhuang
planner: Log warnings when agg function can not be pushdown in explai…
e1d4760 
…n statement (pingcap#25553)
ti-chi-bot pushed a commit that referenced this pull request Jul 20, 2021
@ti-srebot
planner: Log warnings when agg function can not be pushdown in explai…
0192259 
…n statement (#25553) (#25736)
LittleFall pushed a commit to LittleFall/tidb that referenced this pull request Jul 20, 2021
@windtalker @LittleFall
planner: Log warnings when agg function can not be pushdown in explai…
13f9ba8 
…n statement (pingcap#25553)
@LittleFall LittleFall mentioned this pull request Jul 20, 2021
Closed
10 tasks
LittleFall pushed a commit to LittleFall/tidb that referenced this pull request Jul 20, 2021
@windtalker @LittleFall
planner: Log warnings when agg function can not be pushdown in explai…
d3d2f52 
…n statement (pingcap#25553)
@LittleFall LittleFall mentioned this pull request Jul 20, 2021
Merged
10 tasks
ti-chi-bot pushed a commit that referenced this pull request Jul 22, 2021
@ti-srebot
planner: Log warnings when agg function can not be pushdown in explai…
28ebe81 
…n statement (#25553) (#25737)
@windtalker windtalker deleted the agg_not_pushdown_warnings branch November 16, 2021 01:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fzhedu fzhedu fzhedu approved these changes

@hanfei1991 hanfei1991 hanfei1991 approved these changes

@time-and-fate time-and-fate Awaiting requested review from time-and-fate

@XuHuaiyu XuHuaiyu Awaiting requested review from XuHuaiyu

Assignees
No one assigned
Labels
needs-cherry-pick-release-5.0 needs-cherry-pick-release-5.1 sig/execution SIG execution size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@windtalker @sre-bot @fzhedu @CLAassistant @ti-chi-bot @ti-srebot @hanfei1991