Skip to content

v0.2.0 - Dual-Engine FAST Assembly Decompressors

Choose a tag to compare

@seb3773 seb3773 released this 19 Nov 16:45
· 6 commits to main since this release

Lightweight, asymmetric LZ compressor designed around an extremely simple, memory-free (0-alloc) decoder.

Dual-Engine FAST Assembly Decompressors (x86-64 SSE2 & ARM64 NEON)

  • x86-64 (lz0dec_fast_x86_64.s): Reaches 520 – 840 MB/s on text/code and up to 1.33 GB/s on repetitive structured files (e.g. kennedy.xls), with fully inlined subroutines.
  • ARM64 (lz0dec_fast_arm64.s): Pure leaf function (0 bytes on stack, pure register execution via x0..x17 and v0), delivering a 4× to 7× speedup over standard portable C.
  • Dual-Engine Architecture (FAST vs. SAFE): 16-byte SIMD vector wildcopies (movdqu / ldr q/str q) for bulk decompression paired with a sentinel bit-reader tail engine guaranteeing zero out-of-bounds reads/writes.
  • Segment Window & Branch-Free Elias Decoding: Groups control bits into 64-bit refill windows to eliminate branch mispredictions, decoding Elias Gamma lengths and offsets via direct bit detection (bsr/clz) and a 256-byte LUT with zero branches.

Choice of Decompression Engines

LZ0 provides 3 complementary decompression engines:

  • fast (default on x86-64 & ARM64): Maximum throughput SIMD/vector assembly (~1 GB/s, up to 1.33 GB/s).
  • compact: Tiny hand-crafted assembly engine (~300 bytes of code, 0 KB state/LUT, 250–700 MB/s).
  • min: Minimal standalone portable C routine (~75 lines, 0-malloc, clean fallback).

CLI Tools

  • lz0 / lz0.exe: Complete compressor and decompressor CLI.
  • lz0dec / lz0dec.exe: Standalone decompressor CLI designed for verification, engine comparison, and live throughput benchmarks.

Key Highlights

  • High Throughput: 10× to 100× faster decompression than adaptive entropy LZ compressors.
  • Strict Safety: Zero dynamic allocations (0-malloc), strict bounds checking on reads and writes, crash-proof on malformed inputs (240k+ guard-page tests passed, ASan/UBSan & Valgrind clean).
  • Specifications & Documentation: Normative bitstream format (LZ0_FORMAT.md), decoder procedure (LZ0_DECODER_SPEC.md), encoder details (LZ0_ENCODER.md), and benchmarks (BENCHMARKS.md).
  • MIT Licensed.

Packages

  • Linux x86_64: lz0-v0.2.0-linux-x86_64.tar.gz
  • Linux ARM64: lz0-v0.2.0-linux-arm64.tar.gz
  • Windows x86_64: lz0-v0.2.0-windows-x86_64.zip