Home

Ultra Fast LZMA2

uf-lzma2 is a fork of Conor McCarthy's fast-lzma2: an LZMA2 compression library built on a parallel radix match finder, with the LZMA2 codec by Igor Pavlov underneath. The library is renamed uf-lzma2 inside (UF2_* symbols, uf-lzma2.h, libuf-lzma2); the repository keeps the name ultra-fast-lzma2.

The one rule

Output stays standard LZMA2. Everything this fork adds is on the encoder side, in the framing, or in how the library is built; nothing changes the bitstream a decoder has to understand. In practice:

• Raw LZMA2 (UF2_p_omitProperties) is decoded byte for byte by the LZMA SDK's Lzma2Dec.
• .xz output (UF2_p_format = UF2_format_xz) opens in xz and 7-Zip.
• 7-Zip-zstd can keep integrating it the way it integrates fast-lzma2 today.

Where it stands

Measured by the lz6 bakeoff with lzbench, one thread, on Silesia per file (see Benchmarks):

codec	ratio	compress	decompress
uf-lzma2 L10	22.97%	3.3 MB/s	110.5 MB/s
xz L9	23.02%	2.5 MB/s	110.3 MB/s
brotli L11	23.75%	0.5 MB/s	358.6 MB/s
zstd L19	24.96%	2.7 MB/s	789.5 MB/s

The smallest output in that table, at the decode speed of xz. Decompression is the axis LZMA2 cannot win: its range coder decodes one adaptive bit at a time, and Findings records what was measured when looking for a way around that.

What the fork added

• ARM64 assembler LZMA decoder, ported from the LZMA SDK 26.03.
• The assembler decoder in lzbench: lzbench builds fast-lzma2 from its C files only, so every published fast-lzma2 decode figure is the C decoder. With the assembler one, same bytes: +36% to +48%.
• v1.1.0: fast-lzma2's dev branch, unreleased since 2019, landed with byte-identical output.
• Standard .xz output and input (Output formats), multi-block since 1.3.0 and decoded on several threads: Silesia at level 10 with 16 MiB blocks, 859 MB/s on 16 threads against 102 on one, for 23.36% instead of 22.97%.
• Streaming .xz since 1.4.0, both ways, and a fix for a crash inherited from fast-lzma2 in streaming decompression of a bad property byte.
• Level 11: level 10 plus a per-input search over lc/lp/pb, keeping the smallest output. AIT 53.80% → 52.64%, Silesia 22.97% → 22.88%, decompression within a few percent, compression about 3x slower. See Findings.
• bench/curve, a reproducible speed/ratio harness (Building).
• Fixes: a division by zero in the fuzzer, a public header guard shared with fast-lzma2.

Pages

• Benchmarks - the numbers, and how to read them
• Output formats - native, raw LZMA2, .xz
• Building - makefiles, cross-compiling, lzbench, the measuring harness
• Findings - what was measured, including the ideas it ruled out