Skip to content

v0.4.1

@codedeviate codedeviate tagged this 14 Sep 19:18
A 117-byte xz file could make the pure-Rust backend reserve **227 GiB**.
lzma_rust2's Index::parse calls try_reserve_exact on the record count
the file declares, BEFORE its caller compares that count with the blocks
actually decoded. A stream declaring zero blocks and a record-count
varint of 15,294,529,535 asks for 244,712,472,560 bytes.

Found by the project's own fuzzer, in a 120,000-run sweep.

It looked harmless at first and was not: the reservation is lazy, so a
system that overcommits grants it untouched and peak RSS never moves,
and the parse then fails on the first bad record — AFTER the allocation.
Under strict overcommit or a memory cgroup it fails instead.

DecodeOpts::memory_limit never reached it. Phase 1f's bound is a prefix
pre-flight that parses ONE block header; XzReader itself takes no limit.
That left two holes: the index count, and a dictionary declared in a
later block or later concatenated stream.

The fix uses XzStream::new_mem_limit, which existed in the crate all
along and was passed over in Phase 1f on the belief that it needed a
push-to-pull bridge Phase 1e had cancelled. That ruling was about a
different crate's Write-shaped API; XzStream::process is already a pull
API, so this is a loop rather than a bridge. It checks every block
header as parsed, and validates the index count before allocating.

The bomb is now exit 5 with no allocation. Genuine dictionary refusals
stay exit 6 — a bound on work is never reported as corruption.

One behaviour change: --memory-limit 64M now refuses an xz -9 stream at
exit 6 where it previously decoded. Preset 9 needs 65,576 KiB against a
65,536 KiB bound, and the old pre-flight was permitting a decode
measured at 74 MB peak — past the limit the caller set. The CLI default
is unaffected.

Known and not fixed here: lzip.rs has the same pre-flight blind spot for
a later member's dictionary, with LzipStream::new_mem_limit likewise
unused. Much smaller exposure, and its own change.
Assets 2
Loading