Improve TopN row number / rank performance #16753
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pettyjamesm
force-pushed
the
improve-grouped-topn-row-number
branch
from
7fe5a33
to
4f6d7f7
Compare
pettyjamesm
force-pushed
the
improve-grouped-topn-row-number
branch
from
784b656
to
2f37c7b
Compare
pettyjamesm
force-pushed
the
improve-grouped-topn-row-number
branch
2 times, most recently
from
e510847
to
8bcb55e
Compare
pettyjamesm
changed the title
pettyjamesm
force-pushed
the
improve-grouped-topn-row-number
branch
2 times, most recently
from
d79351f
to
42225ce
Compare
Adds a new benchmark method that only benchmarks processPage without attempting to produce the operator output since that is a separate operation. Also changes the benchmark to insert the same page more than once to exercise situations where no new rows necessarily need to be added to the TopN heap.
Changes include: - returning created objects directly from IdRegistry instead their id - using a CMOV friendly compaction scheme in RowReferencePageManager - making classes final for better JIT inlining opportunities
Avoids inserting and accounting for new Pages in GroupedTopNBuilder until we identify the first row that would need to be inserted. When no new rows are inserted, a significant amount of overhead can be avoided entirely which in some situation is common enough to produce appreciable throughput improvements.
pettyjamesm
force-pushed
the
improve-grouped-topn-row-number
branch
from
42225ce
to
2040fba
Compare
|
@erichwang / @martint - build is green and the PR is ready for review when you get a chance |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Avoids inserting and accounting for new Pages in
GroupedTopN{RowNumber, Rank}until we identify the first row that would need to be inserted. When no new rows are inserted, a significant amount of overhead can be avoided entirely which can be common enough to yield appreciable throughput improvements.Benchmark Results:
Release notes
( ) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
(x) Release notes are required, with the following suggested text: