Skip to content

1.1.0 — format v8, addressable blocks

Choose a tag to compare

@drsamonline drsamonline released this 04 Nov 09:19
· 52 commits to main since this release

Format v8: every block's position is now in the trailer, so a block can be reached without walking the stream.

This release cannot read 1.0.0 archives, and 1.0.0 cannot read these. v7 streams are refused with bad-version, which is this project's usual clean break. There are no v7 archives in the wild to strand.

What is new

  • A block offset table in the trailer — one absolute u64 per block, behind a ZTOX tail marker so a reader finds the table by arithmetic from the end. The header is byte-identical to v7; only the trailer grew.
  • ziptastic info — the header, the block table, and every block's offset, size and mode. It verifies the table against the manifest before printing anything.
  • ziptastic d --block N — one block, decoded on its own.
  • ziptastic d --from-block N — a decode that resumes at a block boundary.
  • decode_blocks / decode_blocks_parallel in the library, for a caller that holds an index rather than a stream.
  • Streamed ZIPs are reached — a ZIP written to a pipe puts its sizes in a data descriptor, and the scan read only local headers, so every member was skipped. 320,424 bytes before, 183,286 after (−42.8%); the same ZIP inside a tar 321,197 → 183,454 (−44.0%). This was recorded as "a ZIP inside a plain tar is not reached" and the tar was never the obstacle.

Numbers, measured

  • Reading the index touches 0.76% of a 225-block, 1.38 MB archive, in 0.04 ms.
  • One block decodes in 4–11 ms against ~800 ms for the whole stream.
  • The table costs 8 × blocks + 4 bytes: 0.13% of that archive, 1.7% of a 687-byte one — recorded as a real cost, not rounded away.
  • Ratio and speed are unchanged from 1.0.0: 20–36% smaller than xz -9e on a realistic asset tree, at 0.3–0.6 MB/s.

What it does not do

  • Archive mode is still sequential. Its blocks may reference an earlier one, so --block on an archive-mode stream is a refusal by design, not a guess.
  • A transformed stream cannot be sliced — the inverse chain needs the whole stream.
  • The offset table is not a speed feature. An in-memory index-driven decode measured 4–6% faster and none of it was the table. FINDINGS.md §3c has the reasoning.

Checks

34 test suites, all green; 53 CLI assertions; scripts/regression-corpus.sh asserts the property each historical bug broke, not merely that the round trip works. Full record in docs/FINDINGS.md; the v8 design in docs/FORMAT_V8.md.

Assets

ziptastic-1.1.0-linux-x86_64 — Linux x86-64, statically linked against the vendored coders, built from this tag with the recipe in docs/BUILD.md. SHA256SUMS covers it. Source is GitHub's own archive of the tag; vendor/hcs/old also carries optional mirrors as submodules, none of which the build needs.