-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[refactor] IncrementalIndex extension preparations #12122
[refactor] IncrementalIndex extension preparations #12122
Conversation
…reparations # Conflicts: # processing/src/main/java/org/apache/druid/segment/incremental/IncrementalIndex.java # processing/src/main/java/org/apache/druid/segment/incremental/OnheapIncrementalIndex.java
This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If you think that's incorrect or this pull request should instead be reviewed, please simply write any comment. Even if closed, you can still revive the PR at any time or discuss it on the dev@druid.apache.org list. Thank you for your contributions. |
This pull request/issue has been closed due to lack of activity. If you think that |
Description
This PR refactors the code in preparation for new extensions for
IncrementalIndex
(i.e., #10001).It encapsulates common
IncrementalIndex
functionality inIncrementalIndex.java
as much as possible.In addition, it avoids exposing the dimensions storage outside of
IncrementalIndexRow
. That is, fetch each dimension directly without fetching the entire dimension array.The modifications to
IncrementalIndex
includes:getMetric
methods to acceptIncrementalIndexRow
instead ofint rowOffset
, becauseOakIncrementalIndex
does not keep a mapping from row index to an actual row.This does not affect the other implementations' performance because in all the cases these methods are used, the caller already had an
IncrementalIndexRow
object.CachingColumnSelectorFactory
toIncrementalIndex
The modifications to
IncrementalIndexRow
allow lazy evaluation of off-heap keys, without adding an overhead to the on-heap keys case. We add/modify the following methods:public Object[] getDims()
topublic Object getDim(int index)
public int getDimsLength()
public boolean isDimNull(int index)
public IndexedInts getIndexedDim(int index, @Nullable IndexedInts cachedIndexedInts)
getIndexedDim()
purpose is to support a lazy-evaluation version of an indexed ints dimension instead of the array of integers that is returned bygetDim()
.The modified implementation in
StringDimensionIndexer
, utilize this method and query using IndexedInts instead of the array. This allows our implementation (OakIncrementalIndex
) to use the lazy-evaluation approach, while the other implementations use the existing integer array, without performance degradation.Key changed/added classes in this commit
IncrementalIndex
IncrementalIndexRow
OnheapIncrementalIndex
StringDimensionIndexer
This PR has: