Modularize batch queue in transparent decompression #6363
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
erimatnor
commented
Nov 30, 2023
| @@ -0,0 +1,79 @@ | |||
| /* | |||
There was a problem hiding this comment.
Contents moved from exec.h
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #6363 +/- ##
===========================================
- Coverage 86.92% 59.44% -27.49%
===========================================
Files 250 251 +1
Lines 58155 50844 -7311
Branches 12970 12847 -123
===========================================
- Hits 50550 30222 -20328
- Misses 5205 16134 +10929
- Partials 2400 4488 +2088 ☔ View full report in Codecov by Sentry. |
erimatnor
force-pushed
the
refactor-batch-queue
branch
from
1de7a82
to
b29c5f8
Compare
akuzm
reviewed
Nov 30, 2023
akuzm
reviewed
Nov 30, 2023
erimatnor
force-pushed
the
refactor-batch-queue
branch
3 times, most recently
from
2520c25
to
36881f6
Compare
akuzm
reviewed
Nov 30, 2023
erimatnor
force-pushed
the
refactor-batch-queue
branch
from
36881f6
to
98a0e08
Compare
akuzm
reviewed
Dec 1, 2023
akuzm
approved these changes
Dec 1, 2023
|
tsbench looks good, there are even some speedups, although not sure why |
erimatnor
force-pushed
the
refactor-batch-queue
branch
from
98a0e08
to
ae44237
Compare
mkindahl
approved these changes
Dec 5, 2023
erimatnor
force-pushed
the
refactor-batch-queue
branch
2 times, most recently
from
fd3565e
to
7b852d6
Compare
The batch queue implementation, which is part of transparent decompression, is refactored to be its own self-contained module. Previously, the batch queue functionality consisted of functions that operated on the DecompressChunkState node instead of its own distinct object. The two batch queue types, heap and fifo, are now modularized in their separate source files with a common interface exposed in `batch_queue.h`. The state related to the two batch queue implementations is moved from DecompressChunkState into the respective modules depending on type. For example, the heap state and related initialization code is moved to the heap-based batch queue. The DecompressContext is also moved to its own header file so that other modules need not include everything related to DecompressChunkState.
Rename DecompressChunkColumnDescription to CompressionColumnDescription to reflect that this type is decoupled from DecompressChunkState.
erimatnor
force-pushed
the
refactor-batch-queue
branch
from
7b852d6
to
6d40139
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The batch queue implementation, which is part of transparent decompression, is refactored to be its own self-contained module. Previously, the batch queue functionality consisted of functions that operated on the DecompressChunkState node instead of its own distinct object.
The two batch queue types, heap and fifo, are now modularized in their separate source files with a common interface exposed in
batch_queue.h. The state related to the two batch queue implementations is moved from DecompressChunkState into the respective modules depending on type. For example, the heap state and related initialization code is moved to the heap-based batch queue.The DecompressContext is also moved to its own header file so that other modules need not include everything related to DecompressChunkState.
In a separate commit, the DecompressChunkColumnDescription is renamed to CompressionColumnDescription to reflect that this type is decoupled from DecompressChunkState.
Disable-check: force-changelog-file
Disable-check: commit-count