Skip to content

fix: Improves robustness and refines memory management#321

Merged
hellobertrand merged 4 commits into
mainfrom
fix/v7-review
Jul 12, 2026
Merged

fix: Improves robustness and refines memory management#321
hellobertrand merged 4 commits into
mainfrom
fix/v7-review

Conversation

@hellobertrand

@hellobertrand hellobertrand commented Jul 12, 2026

Copy link
Copy Markdown
Owner

This PR enhances the robustness of decompression, particularly for in-place and dictionary-based operations, and optimizes memory usage by deferring certain allocations. It also refines error handling for compression levels and clarifies various aspects of the API and format documentation.

Key Changes

Robustness & Error Handling

  • Introduces ZXC_ERROR_BAD_LEVEL to cleanly reject compression level raises that exceed a static context's pre-allocated workspace capacity.
  • Adds a plausibility check for decompressed sizes reported in archive footers within zxc_get_decompressed_size and in-place decompression functions, preventing oversized allocations from potentially forged values.
  • Enforces format rules for degenerate Huffman single-symbol tables, rejecting invalid code lengths greater than one.
  • Clamps out-of-range compression levels (e.g., level=99) to ZXC_LEVEL_ULTRA across all compression entry points, ensuring consistent behavior without unexpected failures.
  • Refines error propagation in the Go wrapper for ZXC_ERROR_DST_TOO_SMALL and leverages the C-side footer plausibility check for decompressed size retrieval.

Memory Management

  • Defers the allocation of decode-side entropy scratch buffers (tok_buffer, pivco_scratch) for heap-allocated decompression contexts, reducing the upfront memory footprint for common decompression scenarios. Static contexts continue to provision this memory eagerly.
  • Precomputes Huffman decoder tables (skip flags and flat-root code-to-symbol tables) at dictionary attach time (zxc_cctx_attach_dict_huf), avoiding redundant per-block recomputation during dictionary-based decompression.

API & Documentation Clarity

  • Explicitly rejects dictionary options when creating push-stream contexts (zxc_cstream_create, zxc_dstream_create), as the push-stream format does not support dictionary IDs, preventing the creation of undecodable archives.
  • Updates README.md and docs/API.md to provide a more precise formula for calculating the zxc_decompress_inplace_bound and clarify that destination buffers for in-place and push-stream modes are treated as writable scratch beyond the final output pos.
  • Refines the description of Huffman format rules for flat-subtree fast paths in docs/FORMAT.md and docs/WHITEPAPER.md.
  • Adds ZXC_ERROR_BAD_LEVEL to the public API and all language bindings (Go, Node.js, Python, Rust).

Test Coverage

  • Adds new unit tests for zxc_cctx_level_raise_reinit to verify dynamic workspace re-initialization on level changes.
  • Introduces test_static_ctx_level_raise_rejected to confirm that static contexts correctly reject level raises beyond their allocated capacity.
  • Enhances test_get_decompressed_size with a forged footer scenario to ensure robust handling of malicious input.
  • Adds test_huffman_single_symbol_validation to cover the new Huffman table validation.

…vements

- Adds plausibility checks for `zxc_get_decompressed_size` to prevent oversized allocations from forged footers.
- Enforces stricter validation for Huffman single-symbol tables, rejecting invalid code lengths.
- Introduces `ZXC_ERROR_BAD_LEVEL` for out-of-range compression levels and incompatible level raises on static contexts.
- Optimizes dictionary Huffman decoding by precomputing decode-side tables once at context attach. Refines documentation for in-place decompression margins and PivCo Huffman flat-root rules.
- Ensures `ZXC_ERROR_BAD_LEVEL` is consistently exposed and mapped across Go, Node.js, Python, and Rust wrappers, improving error reporting for invalid compression levels.
- Updates the Go `Decompress` function to rely on C-side footer plausibility checks, simplifying Go's validation logic and mapping `ZXC_ERROR_DST_TOO_SMALL` to `ErrInvalidData` in specific contexts.
- Optimizes WASM stream processing by avoiding `Uint8Array` concatenation when only a single chunk is present.
- Refactors Python compression level validation into a reusable helper function.
@codecov

codecov Bot commented Jul 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.74468% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/lib/zxc_common.c 95.65% 2 Missing ⚠️
src/lib/zxc_dispatch.c 94.87% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
4.5% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

@hellobertrand hellobertrand changed the title Improves robustness and refines memory management misc: Improves robustness and refines memory management Jul 12, 2026
@hellobertrand hellobertrand changed the title misc: Improves robustness and refines memory management fix: Improves robustness and refines memory management Jul 12, 2026
@hellobertrand hellobertrand merged commit 1a37c72 into main Jul 12, 2026
83 of 84 checks passed
@hellobertrand hellobertrand deleted the fix/v7-review branch July 12, 2026 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant