Skip to content

v0.2.1

Choose a tag to compare

@Mnwa Mnwa released this 14 Oct 12:48
· 49 commits to main since this release
7807e75
  • Reuse the Huffman tree during depth selection. Optimal-depth trials
    restore leaf lengths from the original internal nodes instead of sorting
    and rebuilding the same tree for each depth. Table selection and encoded
    bytes remain identical to zstd 1.5.7.

  • Less work on incompressible blocks. After a failed Huffman attempt,
    raw literals that already exceed the block's compression threshold cause
    an early raw-block decision. This avoids copying a literals section and
    encoding sequences that would be discarded. Short output buffers retain
    their existing error behavior, and frames remain byte-identical to C.

  • Simpler fast/dfast candidate checks. Bounded reads from the searchable
    history reject stale indices and guard the comparison word with one range
    check. The search order and minimum-match specializations are unchanged.

  • Wider match comparisons. Fast, double-fast and lazy search share a
    word-first, 128-byte scan in the selected SIMD backend. On AVX2 it tests
    four vectors together, then resolves the first differing word. Short
    matches and bounded tails keep their exact bytewise semantics.

  • Less output initialization in cold compression. compress and
    compress_into reserve the same capacity but initialize the output of
    inputs larger than 128 KiB one block at a time, reusing unwritten room.
    The RLE check uses bulk slice comparison. Frames remain byte-identical
    to zstd 1.5.7; caller-owned slices keep their exact capacity semantics.

  • Faster empty and tiny inputs at levels 11-22. The optimal parser's
    position and match tables are sized per block on first use
    (min(block + 3, ZSTD_OPT_SIZE) entries) instead of being filled to
    4099 entries at every frame start. Frames are unchanged.

  • Updated compression benchmarks
    for the final working tree: 384 measurements, median speed relative to C
    1.000×, with lower mean latency in 96 of 192 dataset/level pairs. All
    frames remain byte-identical to zstd 1.5.7. Large text remains slower;
    the aggregate does not establish an overall lead.

  • Faster Decompressor::decompress and decompress_into. Raw and RLE
    blocks are appended in one pass instead of zeroing room and copying over
    it; frames that declare a content size are decoded into a vector cut at
    their end, so neither slack nor spare capacity is kept (a declared frame's
    vector now has exactly its length as capacity); capacity doubles, bounded
    by the frame, and a declared size is reserved up front up to 32 bytes per
    unread input byte. 1 MiB of random data decodes in 20.1 µs (was 43.8 µs;
    zstd 1.5.7 20.6 µs), 64 KiB in 1.01 µs (was 1.82 µs; zstd 1.79 µs). Errors
    are unchanged: a frame that overruns its declared size is re-decoded uncut
    to report the same ContentSizeMismatch.

  • New cold comparison against zstd 1.5.7 in benchmarks/comparison
    (its own workspace): mbrotli's eight datasets, including Alice in
    Wonderland (testdata/alice29.txt), at levels -5, -1 and 1-22, with
    byte-identity preflight, CSV export, and generated per-level pages,
    per-dataset charts and run reports in docs/benchmarks/, laid out as
    mbrotli's benchmark documentation. See docs/benchmarking.md.