feat: Caches dict Huffman tree upon attach#320
Merged
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
hellobertrand
force-pushed
the
perf/dict-tree-at-attach
branch
2 times, most recently
from
9430950 to
3aa5c24
Compare
Introduces a "tree-at-attach" mechanism for dictionary-based PivCo Huffman. The Huffman tree, canonical codes, and unpacked code lengths are now pre-built once when a dictionary is attached to the compression context. This avoids redundant computation (unpacking lengths and rebuilding the tree) for every block, improving performance for dictionary literal sections during encoding and decoding.
Instead of storing a pointer to caller-owned raw code lengths and building the tree on demand, `zxc_cctx_attach_dict_huf` now immediately unpacks the lengths and constructs the PivCo tree and canonical codes into the context. This "tree-at-attach" strategy eliminates redundant computation during per-block encoding/decoding and clarifies memory ownership, as the input `lengths` are only required to be valid during the attachment call.
Applies the standard function suffixing macro to the dictionary Huffman size calculation function. This ensures it adheres to the consistent naming convention and enables specialized, potentially optimized, variants for the 'tree-at-attach' performance strategy.
Removes the `static` keyword from `zxc_pivco_tree_t tree` declarations within Huffman dictionary test cases. This ensures each test invocation uses a fresh, stack-allocated tree, preventing unintended state sharing between test runs. This adjustment is necessary to align the test setup with the 'tree-at-attach' strategy, where the dictionary's Huffman tree is explicitly managed by the compression context.
The dictionary Huffman state (PivCo tree, canonical codes, and code lengths) is now encapsulated in a `zxc_dict_huf_state_t` struct. This struct is allocated within the compression context's workspace, but only when a dictionary is attached. This refactoring reduces the base `zxc_cctx_t` struct size for contexts that do not use a dictionary, improving memory efficiency by avoiding the unconditional allocation of dictionary-specific Huffman data. It reinforces the "tree-at-attach" strategy by centralizing and conditionally allocating this pre-built state.
hellobertrand
force-pushed
the
perf/dict-tree-at-attach
branch
from
3aa5c24 to
a99e1cc
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Optimizes dictionary literal Huffman processing by moving the tree and code generation out of the per-block hot path.
This change significantly improves performance by:
zxc_dict_huf_state_tstructure within the compression context (zxc_cctx_t) to store the dictionary's PivCo Huffman tree, canonical codes, and unpacked code lengths.zxc_cctx_attach_dict_hufto build this dictionary Huffman state once during context attachment, rather than rebuilding it for every block.This reduces redundant computation, lowering CPU overhead during compression and decompression of dictionary-referenced literal sections.