PHOENIX-4594: Perform binary search on guideposts during query compilation #347
+346
−40
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@karanmehta93 Can you also review this? |
binshi-bing
changed the title
binshi-bing
changed the title
…instead of linear search for the first guide post in the first region of the targeted scan ranges. In details:
1. The aglorithm continuously decode and load guide posts in batches (moving window). For each moving window, firstly compare the searching key with the last element to see whether the searching key is in the current window. If it isn't, perform binary search in the window; otherwise, move to the next window and repeat the above steps until it finds the start key of the first region in the scan ranges or its insertion position
There are two reasons to use moving windows to load guide posts in batches.
a. Firstly, we don't want to load all guide posts in the memory to increase memory footprint.
b. Secondly, we don't want to decode and load the guide posts beyond the targed scan ranges to increase the system overhead.
2. Added config parameter STATS_GUIDEPOST_MOVING_WINDOW_SIZE to denote the size of moving window used for loading guid posts.
| if (firstRegionStartKey.getLength() > 0 && this.gpsMovingWindowSize > 0) { | ||
| // Continuously decode and load guide posts in batches (moving window). For each moving window, | ||
| // firstly compare the searching key with the last element to see whether the searching key is | ||
| // in the current window. If it isn't, perform binary search in the window; otherwise, move to |
twdsilva
reviewed
Oct 1, 2018
| assertEquals((Long) 6L, info.getEstimatedRows()); | ||
| assertEquals((Long) 460L, info.getEstimatedBytes()); | ||
| // TODO: the original code before this change will hit the following assertion. Need to investigate it. | ||
| // assertTrue(info.getEstimateInfoTs() > 0); |
There was a problem hiding this comment.
Why does this assert fail now?
There was a problem hiding this comment.
As the "TODO" in the comment said, this is a new test case I wanted to add in this change, but it failed. So I reverted all the changes except this new test case, but it still failed, which means there is problem in the current code base without any of my change. I commented out the assertion here, but I opened another JIRA PHOENIX-4914 to track the original problem in the code base.
| "CREATE TABLE " + tableName + " (k INTEGER PRIMARY KEY, a bigint, b bigint)" | ||
| + " GUIDE_POSTS_WIDTH=" + guidePostWidth | ||
| + ", USE_STATS_FOR_PARALLELIZATION=true" + " SPLIT ON (102, 105, 108)"; | ||
| conn.createStatement().execute(ddl); |
There was a problem hiding this comment.
refactor the common init code in initDataAndStats and reuse it here.
There was a problem hiding this comment.
I'll hold this change as discussed in PHOENIX-4594.
jpisaac
pushed a commit
to jpisaac/phoenix
that referenced
this pull request
Oct 1, 2020
PHOENIX-5592 MapReduce job to asynchronously delete rows where the VI…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…instead of linear search for the first guide post in the first region of the targeted scan ranges. In details: