Repository navigation
v0.1.0
First version: a safe-Rust Zstandard encoder byte-identical to zstd 1.5.7,
an independent RFC 8878 decoder, parallel compression, and a C ABI.
Compression
- Added
Compressorwithcompress,compress_into,compress_to_slice
andcompress_to_uninit. One-shot output is byte-identical to
ZSTD_compress2with the same parameters. - Added incremental sessions:
EncoderSession, andEncoderSessionOwned
withreinitandinto_compressor. A session matches
ZSTD_compressStream2driven with the same chunks, output capacities and
Continue/Flush/Endoperations, including the reference's direct
compression of a finalEndand the content size it then records. - Configuration in
EncoderConfig:- levels -131072..=22 (
Level), the explicitCompressionParams(window,
chain, hash and search logs, minimum match, target length,Strategy); - long distance matching (
LongDistanceMatching,LdmParams); - checksum and content-size flags, and a source size hint.
- Out-of-range values fail with
ConfigError::OutOfRange, which names the
Parameter.Compressor::default()uses level 3.
- levels -131072..=22 (
- Added
std::ioadapters (EncoderReader,EncoderWriter,FinishError)
and memory retention control (RetentionPolicy,trim,
retained_bytes). - The core ports zstd 1.5.7's encoder:
- all nine strategies with their extDict variants, the row match finder
and binary trees; - the optimal parser, including btultra2's first-block pass;
- long distance matching;
- the pre- and post-block splitters;
- FSE and Huffman table selection;
- streaming buffers and window wrap-around;
- index overflow correction.
- all nine strategies with their extDict variants, the row match finder
Decompression
- Added
Decompressorwithdecompress,decompress_intoand
decompress_to_slice;DecoderSessionandDecoderSessionOwned; the
DecoderReaderandDecoderWriteradapters. Concatenated and skippable
frames are supported. DecodeLimitsbound the window (default 2^27, up to 2^31), the input and
the output. Failures are typed (DecodeError,InvalidDataKind).- It is stricter than the reference in two places: Huffman streams must be
consumed exactly, and offsets beyond the window are rejected in every API.
Frames that need a dictionary fail withDictionaryUnsupported.
Parallel compression (std)
- Added
compressor::parallel:ParallelCompressor, withprepare_slicebatches of caller-run tasks
and streamingParallelEncoderSessions with aParallelTaskQueue;- segments that reach back into an overlap of the previous input;
- frames independent of the task count and execution order;
- panic-safe task completion.
- Parallel frames decode anywhere but are not byte-identical to the serial
encoder or to zstd's multithreaded mode.
Tooling
- Added
mzstd-ffi:mzstd_compress(byte-identical toZSTD_compress),
mzstd_decompress,mzstd_compress_bound, andinclude/mzstd.h. - Added
zstd-ffi, a test-only crate that builds the vendored zstd v1.5.7
and provides safe oracle wrappers and corpora. - Added AFL++ targets in
fuzz/afl:differential_c,output_capacity,
encoder_session,compressor_lifecycle,parallel,
parameter_parsing,decompress,decode_streamingand
decode_roundtrip. Each has committed regressions, a seed builder over
upstream's test data, and a campaign script. - Added Criterion benchmarks against zstd (
compress,decompress, and
mzstd-ffi'sone_shot), CI workflows and architecture specifications. - Measured with
cargo bench --bench compress -- 'compress/(text-1MiB|binary-1MiB)/'
andcargo bench --bench decompress -- 'decompress/(text-4MiB|binary-4MiB)/level-3'
(--warm-up-time 1 --measurement-time 3, Apple M5 Pro, rustc 1.99).
Output sizes are identical on both sides. Compression runs at 0.5–0.7×
zstd's speed (text 1 MiB, level 3: 418 vs 775 MiB/s; level 19: 3.8 vs
5.5 MiB/s). Decompression runs at 0.2–0.35× (text 4 MiB: 716 MiB/s vs
2.19 GiB/s). Neither codec has SIMD yet.