Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KYLIN-4432 duplicated queries with sytax error take unexpect long time when lazy query enabled #1167

Merged
merged 4 commits into from
May 22, 2020

Conversation

xiacongling
Copy link

Proposed changes

This PR fix the bug described in KYLIN-4432 that duplicated queries with sytax error take unexpect long time when lazy query enabled.

Types of changes

What types of changes does your code introduce to Kylin?
Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

  • I have create an issue on Kylin's jira, and have described the bug/feature there in detail
  • Commit messages in my PR start with the related jira ID, like "KYLIN-0000 Make Kylin project open-source"
  • Compiling and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • If this change need a document change, I will prepare another pr against the document branch
  • Any dependent changes have been merged

Further comments

N/A

@coveralls
Copy link

coveralls commented Mar 24, 2020

@zhangayqian
Copy link
Contributor

Verified.
In kylin 3.0.1,KYLIN-4432 could be reproduce.
image
image

Add this PR will fix.

image

@kyotoYaho
Copy link
Contributor

kyotoYaho commented Mar 27, 2020

LGTM. How about extract evicting the dummy response logic to the finally block?

@xiacongling
Copy link
Author

@kyotoYaho Thanks for the suggestion. I updated the PR with a finally block.

@codecov-io
Copy link

codecov-io commented Mar 31, 2020

Codecov Report

Merging #1167 into master will decrease coverage by 0.05%.
The diff coverage is 0.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master    #1167      +/-   ##
============================================
- Coverage     25.06%   25.01%   -0.06%     
- Complexity     6250     6253       +3     
============================================
  Files          1446     1449       +3     
  Lines         88309    88502     +193     
  Branches      12358    12374      +16     
============================================
  Hits          22136    22136              
- Misses        64003    64189     +186     
- Partials       2170     2177       +7     
Impacted Files Coverage Δ Complexity Δ
...va/org/apache/kylin/rest/service/QueryService.java 0.00% <0.00%> (ø) 0.00 <0.00> (ø)
...org/apache/kylin/rest/util/QueryRequestLimits.java 35.71% <0.00%> (-4.77%) 5.00% <0.00%> (-1.00%)
...va/org/apache/kylin/cube/common/SegmentPruner.java 58.46% <0.00%> (-4.04%) 26.00% <0.00%> (+1.00%) ⬇️
...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java 69.30% <0.00%> (-1.83%) 7.00% <0.00%> (ø%)
...he/kylin/engine/mr/common/MapReduceExecutable.java 12.91% <0.00%> (-0.42%) 6.00% <0.00%> (ø%)
...ain/java/org/apache/kylin/engine/mr/CubingJob.java 1.53% <0.00%> (-0.04%) 2.00% <0.00%> (ø%)
...n/java/org/apache/kylin/tool/DiagnosisInfoCLI.java 0.00% <0.00%> (ø) 0.00% <0.00%> (ø%)
...ava/org/apache/kylin/tool/JobDiagnosisInfoCLI.java 0.00% <0.00%> (ø) 0.00% <0.00%> (ø%)
.../java/org/apache/kylin/common/util/HadoopUtil.java 14.56% <0.00%> (ø) 10.00% <0.00%> (ø%)
.../java/org/apache/kylin/metrics/MetricsManager.java 0.00% <0.00%> (ø) 0.00% <0.00%> (ø%)
... and 26 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 734a997...7034a04. Read the comment docs.

@kyotoYaho
Copy link
Contributor

Hi xiacongling, the previous one without finally block is better. I'm sorry I mislead you.

@xiacongling
Copy link
Author

Never mind, @kyotoYaho ! I think the problem is that conditions are lost in finally block. We can either rebuild the conditions or use the cached value.

  • The former solution will add complicated condition checks in the final block. These checks are difficult to understand and test, and may be buggy in the future when features are addd or improvements are made.
  • The latter will visit the cache one more time and will reduce the query performance.

Using "try .. finally" is more elegant, but some code refactoring is needed to keep it clean, I think. To get out of the dilemma, we can treat this PR as a quick bug-fix, and discuss the refactoring in another issue.

I will revert the commit. Thanks for the review!

Copy link
Contributor

@shaofengshi shaofengshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 LGTM

@shaofengshi shaofengshi merged commit 6342bc5 into apache:master May 22, 2020
hit-lacus pushed a commit to hit-lacus/kylin that referenced this pull request Jun 4, 2020
…e when lazy query enabled (apache#1167)

* KYLIN-4432 duplicated queries with sytax error take unexpect long time when lazy query enabled
zhangayqian pushed a commit to zhangayqian/kylin that referenced this pull request Jun 11, 2021
…e when lazy query enabled (apache#1167)

* KYLIN-4432 duplicated queries with sytax error take unexpect long time when lazy query enabled

(cherry picked from commit 6342bc5)
zhangayqian pushed a commit to zhangayqian/kylin that referenced this pull request Jun 11, 2021
…e when lazy query enabled (apache#1167)

* KYLIN-4432 duplicated queries with sytax error take unexpect long time when lazy query enabled

(cherry picked from commit 6342bc5)
hit-lacus pushed a commit that referenced this pull request Jun 27, 2021
…e when lazy query enabled (#1167)

* KYLIN-4432 duplicated queries with sytax error take unexpect long time when lazy query enabled

(cherry picked from commit 6342bc5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants