fix: Improves robustness and refines memory management#321
Merged
Conversation
…vements - Adds plausibility checks for `zxc_get_decompressed_size` to prevent oversized allocations from forged footers. - Enforces stricter validation for Huffman single-symbol tables, rejecting invalid code lengths. - Introduces `ZXC_ERROR_BAD_LEVEL` for out-of-range compression levels and incompatible level raises on static contexts. - Optimizes dictionary Huffman decoding by precomputing decode-side tables once at context attach. Refines documentation for in-place decompression margins and PivCo Huffman flat-root rules.
- Ensures `ZXC_ERROR_BAD_LEVEL` is consistently exposed and mapped across Go, Node.js, Python, and Rust wrappers, improving error reporting for invalid compression levels. - Updates the Go `Decompress` function to rely on C-side footer plausibility checks, simplifying Go's validation logic and mapping `ZXC_ERROR_DST_TOO_SMALL` to `ErrInvalidData` in specific contexts. - Optimizes WASM stream processing by avoiding `Uint8Array` concatenation when only a single chunk is present. - Refactors Python compression level validation into a reusable helper function.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


This PR enhances the robustness of decompression, particularly for in-place and dictionary-based operations, and optimizes memory usage by deferring certain allocations. It also refines error handling for compression levels and clarifies various aspects of the API and format documentation.
Key Changes
Robustness & Error Handling
ZXC_ERROR_BAD_LEVELto cleanly reject compression level raises that exceed a static context's pre-allocated workspace capacity.zxc_get_decompressed_sizeand in-place decompression functions, preventing oversized allocations from potentially forged values.level=99) toZXC_LEVEL_ULTRAacross all compression entry points, ensuring consistent behavior without unexpected failures.ZXC_ERROR_DST_TOO_SMALLand leverages the C-side footer plausibility check for decompressed size retrieval.Memory Management
tok_buffer,pivco_scratch) for heap-allocated decompression contexts, reducing the upfront memory footprint for common decompression scenarios. Static contexts continue to provision this memory eagerly.zxc_cctx_attach_dict_huf), avoiding redundant per-block recomputation during dictionary-based decompression.API & Documentation Clarity
zxc_cstream_create,zxc_dstream_create), as the push-stream format does not support dictionary IDs, preventing the creation of undecodable archives.README.mdanddocs/API.mdto provide a more precise formula for calculating thezxc_decompress_inplace_boundand clarify that destination buffers for in-place and push-stream modes are treated as writable scratch beyond the final outputpos.docs/FORMAT.mdanddocs/WHITEPAPER.md.ZXC_ERROR_BAD_LEVELto the public API and all language bindings (Go, Node.js, Python, Rust).Test Coverage
zxc_cctx_level_raise_reinitto verify dynamic workspace re-initialization on level changes.test_static_ctx_level_raise_rejectedto confirm that static contexts correctly reject level raises beyond their allocated capacity.test_get_decompressed_sizewith a forged footer scenario to ensure robust handling of malicious input.test_huffman_single_symbol_validationto cover the new Huffman table validation.