Skip to content

structured-world/structured-zstd

 
 

Repository files navigation

structured-zstd

Pure Rust zstd implementation — managed fork of ruzstd.

CI Crates.io docs.rs License: Apache-2.0

Managed Fork

This is a maintained fork of KillingSpark/zstd-rs (ruzstd) by Structured World Foundation. We maintain additional features and hardening for the CoordiNode database engine while contributing patches upstream.

Fork goals:

  • Dictionary compression improvements (critical for per-label trained dictionaries in LSM-tree)
  • Performance parity with C zstd for decompression (currently 1.4-3.5x slower)
  • No FFI — pure cargo build, no cmake/system libraries (ADR-013 compliance)

Upstream sync: We regularly rebase on upstream and contribute improvements back.

What is this

A pure Rust implementation of the Zstandard compression format, as defined in RFC 8878.

This crate contains a fully operational decompressor and a compressor that is usable but does not yet match the speed, ratio, or configurability of the original C library.

Current Status

Decompression

Complete RFC 8878 implementation. Performance: ~1.4-3.5x slower than C zstd depending on data compressibility.

Compression

  • Uncompressed blocks
  • Fastest (roughly level 1)
  • Default (roughly level 3)
  • Better (roughly level 7)
  • Best (roughly level 11)
  • Checksums
  • Dictionary compression

Dictionary Generation

When the dict_builder feature is enabled, the dictionary module can create raw content dictionaries. Within 0.2% of the official implementation on the github-users sample set.

Usage

Compression

use structured_zstd::encoding::{compress, compress_to_vec, CompressionLevel};

let data: &[u8] = b"hello world";
let compressed = compress_to_vec(data, CompressionLevel::Fastest);

Decompression

use structured_zstd::decoding::StreamingDecoder;
use structured_zstd::io::Read;

let compressed_data: Vec<u8> = vec![];
let mut source: &[u8] = &compressed_data;
let mut decoder = StreamingDecoder::new(&mut source).unwrap();

let mut result = Vec::new();
decoder.read_to_end(&mut result).unwrap();

Support the Project

USDT TRC-20 Donation QR Code

USDT (TRC-20): TFDsezHa1cBkoeZT5q2T49Wp66K8t2DmdA

License

Apache License 2.0

Contributions will be published under the same Apache 2.0 license.

About

Pure Rust zstd implementation (fork of ruzstd). Dictionary compression, no FFI. For coordinode-lsm-tree and CoordiNode.

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors

Languages

  • Rust 100.0%