Skip to content

Changelog

YadeWira edited this page Sep 3, 2026 · 60 revisions

Changelog

Day-by-day (and sometimes hour-by-hour) history of how each codec reached its current status. See Component-Status for the current state and Reverse-Engineering-Notes for the general methodology.

Date Milestone
2026-04-24 lzpf arith primitives complete (BitReader, Huffman, RangeCoder, ReadCodeLengths); 2524 unit-test assertions
2026-04-24 DecodeArithBuffer (FUN_080a4ea0) validated against 3 real-data vectors
2026-04-24 DecodeLz77VariantA complete; 6/7 sample archives native
2026-04-24 Sliding-window dict + wrap-to-0 logic confirmed via GDB; last_lz_dest per-block reset fixed
2026-04-24 DecodeLz77VariantB (-cF, 24-bit hash) complete; 7/7 -cF samples native
2026-05-05 Hash table init bug fixed (0→3); window_capacity formula corrected ((p1+1)×64 KiB); 8/8 methods 100% native_strict
2026-05-07 Native CM decoder ported from nzdec_v0 reference (NZ_CM.cpp, 1100 LOC); all block modes decode natively. Stereo audio variant deferred.
2026-05-07 lzpf prefilter+arith mono path byte-exact (task #13): LpcPredictor fixed to 4-tap, 2-samples-per-iteration (factors update only on first of each pair — matching FUN_08095d90 SIMD path); Fletcher32 verified on ramp WAV fixture.
2026-05-07 lzhd native decoder complete (task #14): DecLZ PAQ context mixer + 12-bit range coder ported from nzdec_v0 NZ_LZ.cpp (680 LOC); byte-exact on 50 KB text fixture. C++ const-linkage bug fixed (extern const kLzModelLNext).
2026-05-07 lzpf prefilter stereo residual decoder (task #13b): DecodeResidualsStereo ported from FUN_0809bbf0 with binary-extracted tables; VLC magnitude + explicit sign-bit scheme. Speculative — no known fixture triggers this path with standard -cf.
2026-05-07 Parallel archive parser fixed (task #25): TryParseLegacyCnArchive now scans all NZ chunk records ((size<<4)|type varint + nibble-15 stream-ID extension), accumulates per-filename sizes across all per-stream type-1 chunks. -pN archives list and extract correctly.
2026-05-09 Archive writer codec chunk fix (task #26): RunAddNativeLegacyStream now pads codec payload to declared csize bytes; fixes chunk scanner misparse of own archives. encode_ok 5/8 → 8/8; all methods byte-exact round-trip.
2026-05-10 CM text-transform tt_flags=0x08 (dictionary): NzTextTransformDict ported from TransformText_1_Dictionary; tables extracted from linux32 binary; text files byte-exact under -cc.
2026-06-01 CM tt_flags=0x10 (word-list): corrected diagnosis — blocker is a bug in NzCmDecode, not the transform. GDB ground truth at 0x080a3340 → golden oracle; standalone harness reproduces a deterministic divergence at byte 26 bit 5; CM params ruled out by sweep. Transform decode tree (fcn.080a3340/a28a0/a1b60) mapped for porting once CM is fixed.
2026-06-02 lzpf stereo-split gap root-caused: missing inter-channel predictor FUN_08096e20 (2-stage cascaded sign-sign LMS 4-tap; ch2 predicts from ch1's reconstructed sample). Fully reverse-engineered, port pending. Confirmed mono audio fully native byte-exact; stereo currently bridges.
2026-06-02 CM -cc decode: established the documented CM engine/dispatcher addresses (0x0809e600/0x080a5c70/0x080aa850) do not execute for -cc decode (all breakpoints unhit; dispatcher no-ops with method byte 0). Real decode routine still unlocated; byte-26 bug reframed as a deterministic prediction/weight-update formula error (bits 0–208 match legacy).
2026-06-03 lzpf LZ77+arith path verified 100% native: exhaustive random/high-entropy corpus (200+ seeds × 5 sizes 4KB–512KB) all decode byte-exact natively. Task #24 closure confirmed: the hash_table init = 3 fix from 2026-05-05 plus the per-block last_lz_dest = -1 reset already cover all observable LZ77 dispatcher edge cases. Coverage estimate for -cf/-cF upgraded from ~95% → 100%.
2026-06-03 lzpf prefilter+arith stereo-split path complete (task #13b final): FUN_08096e20 (2-stage cascaded sign-sign LMS 4-tap, MMX path) ported as scalar equivalent. LmsObject struct (0x2070 bytes, 2 stages per object, 2 objects per block for ch1+ch2) added to include/lzpf_arith.h. ApplyLmsInterChannel performs in-place LMS on ch1/ch2 residual streams. Dispatcher in sfx_archive.cpp auto-detects stereo split from the prefilter header byte ((hdr>>1) % 3 != 0) and threads persistent LMS state across blocks. Verified byte-exact on synthetic correlated stereo WAV (stereo_lms.wav 64 KB) for both -cf and -cF. Coverage estimate for prefilter upgraded from ~90% → 100%.
2026-06-03 lzpf -cf/-cF multi-file content corruption bug found and fixed via defensive cross-check. Root cause: the native LZ77+arith path silently produced size-correct but content-wrong output for -cF multi-file archives with mixed random+repeat+zero files (no per-entry checksums to catch it). Reproducer: 3 random files (1KB/2KB/3KB) + 1 zero file (8KB) + 1 repeat file (8KB) compressed with -cF; native extraction corrupted the random files starting at byte ~759. Fix: full-buffer cross-check between native output and legacy extract-bridge output (added to RunLegacyCnExtractOrTest in sfx_archive.cpp); only fires when no per-entry checksums are present (i.e. when native could produce silent garbage). Skippable via NZ_DISABLE_LZPF_BRIDGE=1. Same pattern as the CM cross-check (HUECO B fix). Verified byte-exact on the regression fixture tests/fixtures/lzpf/regression_cF_multi.nz and across the 15-fixture × 7-method cross-comparison suite (105/105 byte-exact).
2026-06-04 lzpf variant B hash-table init bug root-caused via GDB trace and fixed via real port. The cross-check from ce232d2 was reverted (refactor(lzpf): remove universal cross-check). GDB trace on linux32/nz extracting the regression fixture showed that the legacy initializes the variant-B hash table to 3 (NOT 0 as previously assumed). C++ was initializing it to 0, causing silent corruption. Fix: change line 3013 to std::int32_t{3} for BOTH variants. Both -cf and -cF multi-file archives now decode byte-exact natively with NO legacy dependency.
2026-06-08 CM byte-26 divergence FIXED — root cause was a one-line port error in CM_Input_Bit: factors0_err used truncating (int32)factors[0] / 16 instead of the reference arithmetic shift factors[0] >> 4 (differs for negative factors[0], perturbing factors0_err_flt and flipping the factor[7] zeroing condition → wrong prediction from byte 26). Found via a per-bit next_probability differential trace against a compiled reference oracle (encode_su/nzdec_v0 NZ_CM.cpp). The earlier "architecturally divergent / unlocated decode routine" theory was disproven — the port is faithful; it was just the shift. nz_lzhd.cpp model_d LUT init had a sibling rows/cols-swapped bug (NzLzhdCreate always SIGSEGV'd) — fixed (12,1536)(1536,12).
2026-06-08 -cc fully native — ported the CM post-filter pipeline: param2 RLE (BwtRleExpander::DecodeU32src/nz_postfilter.cpp) and tt16 (TextTransformNumber::Decode, a number transform — NOT word-list as previously believed → src/nz_texttransform_num.cpp), wired into TryDecodeLegacyCm (CM→param2→param1→tt16→tt08→dece). -cc decodes byte-exact natively (no bridge) across source/numbers/dates/IPs/prose/markdown. Added NZ_NO_BRIDGE=1 (hard-fail instead of silent bridge) + tests/native_only_v2.sh (honest no-bridge measurement).
2026-06-08 -co/-cO (optimum) found to use DecLZ, not BWT. Small blocks (flat framing) decode natively; large blocks use the same virtual-stream LZ framing as -cd (reference also SIGSEGVs) and bridge. Discovered non-CM codecs apply kReorderAscii before tt08. -cd/-cD/-co/-cO large all share the one virtual-stream-framing blocker.
2026-06-08 -cc completed for non-audio inputs: (1) multi-chunk CMTryDecodeLegacyCm now loops over consecutive type-0 chunks (CM state persists; large -cc splits into several chunks, e.g. a 64 KB random file = 4 chunks); (2) stored blocksparam6==0 means the payload is the raw output verbatim (incompressible data; the reference rejects these with param6!=1); (3) param1 = AddBytesFilter ported (NzAddBytesFilter in nz_postfilter.cpp, delta filter, wired after param2). -cc byte-exact native on a 13/13 comprehensive sweep; native_only_v2 -cc 8/10→9/10 (only the stereo audio-CM variant remains).
2026-06-20 -cd large-file LZ window solved. The cross-chunk window is a single per-archive ring, not a fixed 64 KB — confirmed via GDB (obj+0x978). Sizes are not powers of two, so the ring helpers were switched from bitmask to modular wrap (nz_cd_tokens.cpp). Large files split output into 1 MB streams that match into each other through the ring, so the ring now persists across streams: new NzCdDecodeStream takes a caller-owned ring + ring_pos (in/out) + file-absolute output offset, and the dispatcher allocates it once. TryDecodeLegacyLzhd also self-verifies the decode against the stored per-file checksum (bridges on mismatch — no silent corruption). The earlier "vtable double-buffer window / fixed 64 KB ring" theory was superseded.
2026-06-21 -cn (store) multi-block reassembly → 10/10. A stored file large enough to split (> ~0x30000) is not a contiguous archive tail: NanoZip writes it as several blocks, each [varint (len<<4)|0][raw bytes], with a per-block checksum trailer (0x45/0x47/0x26 tag + checksum bytes, width per checksum_mode) between blocks; the last block has none. The native decoder previously assumed a single tail block (it scanned for one stream prefix tagged total<<4) and aborted with "Legacy stream prefix is not recognized." on multi-block files (e.g. a 215 KB source file → 196608 + 18566). New TryAssembleStoredBlocks walks the blocks from the first prefix (just past the per-file metadata), strips prefixes + trailers, and concatenates the raw payload; the walk must consume the archive exactly to EOF, which makes the start offset unambiguous. -cn native_only_v2 9/10→10/10, total 58→59/80; verified byte-exact on a size sweep (1 KB–1.5 MB, single + multi-block, random + compressible); suite 2553/2553, zero regression.
2026-06-21 -cF (lzpf B) stereo prefilter closed → native byte-exact. stereo_lms_cF.nz now decodes byte-exact to stereo_lms.wav (-cF native_only_v2 8/10→9/10, total ~58/80). GDB showed -cF (= nz_lzpf_large) uses the SAME FUN_080a4ea0 arith + the SAME stereo flow as -cf (NOT a vtable path); the only difference is the LPC filter order field FUN_08095d90 obj+0x1c08 = 8 for variant B vs 4 for variant A (shift obj+0x1c0a = 8 for both). Extended LpcPredictor to N taps: the ==8 path is 8-tap and adapts factors every sample; the <8 path is 4-tap and adapts every other sample. Tap count selected by is_variant_b (method_p0==2) in the dispatcher. -cf unchanged (4-tap).
2026-06-20 -cf (lzpf A) stereo prefilter closed end-to-end → native byte-exact. stereo_lms_cf.nz (both blocks) now decodes byte-exact to stereo_lms.wav (-cf native_only_v2 9/10→10/10, total ~57/80). Reverse-engineered the full FUN_080a5330 stereo flow via GDB stage-capture: residuals are PLANAR, decoded by two per-channel DecodeArithBuffer calls (each reads its own Huffman header; one n_elems call mis-located the side stream by ~1.4 KB); predictor-init = leading bit G (iStack_50078, LMS gate) + per-channel [active(1), order(3)]; per-channel LPC with two persistent predictors (pred/pred2); inter-channel LMS FUN_08096e20; FUN_080a50c0 reconstruction (ReconstructStereoSamples, L/R + mid/side). The old "interleaved split / raw-bytecode block-1" diagnosis was wrong — both blocks are stereo-prefilter. -cF (lzpf B vtable arith) stereo still bridges.
2026-06-20 -cd ring-size formula corrected → multi-stream fully native, no wrap. GDB on FUN_08099050 (obj+0x978) across text50/source.cpp/big_code/repeat_3M gives ring = 1/3/19/46 × 64 KB = round(total_output / 0x10000) · 0x10000 (min 64 KB), NOT the (method_p1+1)·0x10000 lzpf rule (which under-sized the ring for large files, e.g. 43×64 KB vs the real 46×64 KB for a 3 MB repeat file, forcing a wrap the binary never does). The encoder sizes the ring to hold the whole compact recon, so the cursor (obj+0x980) advances monotonically and never wraps. With the correct size, all multi-stream large files — 2–3.5 MB code/text/base64 and a 3 MB heavily-repetitive file — decode byte-exact natively (verified 6/6 on a varied size/compressibility sweep). -cd native_only_v2 9/10, suite 2553/2553, zero regression.
2026-06-22 Large single-container multi-stream -cf/-cF (1–8 MB) → native byte-exact. The lzpf output splits into ~1 MB compressor streams chained [stream_tag=(clen<<4)][data]; between streams sits the member's whole-output checksum record (tag 0x45 Fletcher32 / 0x47 crc32 / 0x26 crc16 + width). The dispatcher's lzpf chain loop now (1) consumes those checksum records to keep walking the chain, and (2) models the legacy sliding-window wrap (FUN_080b6bb0: when <32 KiB remain before the dict capacity, zero [cursor, cap+0x8000) and reset the cursor to 0). The dict capacity itself is not recoverable from the header — GDB on FUN_080b6bb0 (obj+4) shows the encoder picks one of (p1+1)·64 KiB, floor(total/128 KiB)·128 KiB, or ceil(total/128 KiB)·128 KiB by its threading — so the decoder tries each candidate and adopts the first whose decoded output matches the embedded whole-output checksum. A wrong capacity only changes bytes once a window wrap occurs, and that is exactly what the checksum rejects, so a mismatch bridges/refuses rather than emitting wrong bytes (no silent corruption). Verified byte-exact on a 1–8 MB sweep (-cf and -cF, incl. a file whose true dict is smaller than its output and genuinely wraps); the >~8 MB parallel-container layout (header flag 0x0f) still bridges. Suite 2553/2553, native_only_v2 total 59/80 unchanged (its corpus is ≤256 KB).
2026-06-24 Parallel multi-stream container (-cf/-cF, header flag 0x0f) → native byte-exact — the format used by the multi-threaded encoder for inputs >~8 MB (e.g. a multi-hundred-MB / GB .tar). The output is cut into N independent nz_lzpf streams, each serialised as a group of chunk records keyed by stream id: type-1 = slice output size, type-10 (u32 LE) = slice output offset, type-5/7/6 = slice Fletcher32/crc32/crc16, and one or more type-0 compressed chunks (concatenated — a slice >1 MB spans several). The dispatcher decodes each stream into its slice (verifying it against the slice checksum, which also selects the dict capacity), places it at its offset, and requires the slices to tile the whole output exactly — every byte is checksum-verified, so a wrong decode is rejected, never emitted. Also ported the raw-bytecode LZ77 block mode ((uvar9 & 2) set, (uvar9 & 1) clear — the opcode stream is the input bytes directly, no arith side stream; DecodeLz77VariantA/B now report bytecode-consumed so the caller advances the input cursor), closing -cF native_only_v2 9/10→10/10. Dict capacity is now probed at 64 KiB granularity (caps are N·64 KiB, not always N·128 KiB). Verified byte-exact on parallel containers up to 20 MB+ (-cf and -cF, text and random, multi-chunk slices). Remaining: side-block LZ77 on mixed-binary data (pre-existing, single- and multi-stream alike; refuses safely). Suite 2553/2553; native_only_v2 total 59→60/80.
2026-07-01 lzpf side-block LZ77 on mixed-binary data → fixed (the last -cf/-cF gap). Root-caused by GDB-diffing the decoder's hash_table against the legacy binary's own hash_table at the exact block boundary where a real-world tar (text + random content, ~9 MB, parallel container) first produced wrong bytes: legacy calls FUN_080b6d90 (sparse, every 101 bytes) or FUN_080b6cf0 (dense, every byte — selected by a bit in the literal block's header) to backfill hash_table after every literal block; the native port never did this, so any hash bucket touched only during a literal run kept its untouched init value (3) forever. Output stayed byte-correct for many blocks (nothing referenced those buckets yet) until an f6/f8/medium-match opcode finally looked one up and copied from window offset 3 (the dead left-pad zone) instead of the real prior occurrence — explaining the "sparse single-byte corruption, otherwise perfect" signature. Variant B (-cF) additionally zeroes its byte_buffer_8k side-table after every literal block (FUN_080b6c20). Isolated with a standalone harness that replays a captured-correct bytecode + window + hash_table snapshot through the LZ77 dispatcher alone, confirming the arith stage (DecodeArithBuffer) was already byte-exact and the bug was purely the missing backfill. A second, unrelated bug surfaced by the same repro was also fixed: the archive-metadata multi-block table scanner (a heuristic, self-documented as spoofable by noisy compressed bytes) was firing on parallel-container payloads and injecting a phantom filename entry that corrupted total_data_size; it's now skipped whenever size_accum (the per-stream parallel size accumulator) is already populated. Verified byte-exact on real mixed text+random tars (9 MB+, both -cf and -cF, single- and multi-stream) and a 15-seed pure-random sweep. Suite 2553/2553; native_only_v2 total 60/80 unchanged (its corpus doesn't exercise this path).
2026-07-02 -cd/-cD parallel multi-stream container → native byte-exact, mirroring the -cf/-cF parallel-container work above (same header flag 0x0f, same chunk-record scheme — the record-parsing loop is shared infrastructure, not lzpf-specific). Confirmed each parallel-encoder thread owns its own nz_cd instance: every stream gets a FRESH, independent ring (sized per-slice via round(slice_size/0x10000)·0x10000), unlike the single-container case where one ring is shared across the whole archive. A -cd stream's type-0 chunk records are each one complete, already-delimited raw DecLZ block (no concatenation needed, unlike lzpf's continuous bitstream) — decoded one chunk at a time via NzCdDecodeStream, threading the per-stream ring across chunks, verified against its own per-slice checksum before being tiled into the assembled output. Verified byte-exact on parallel containers up to 20 MB (repetitive text, pure random, and mixed text+random tars). Also fixed a latent crash (found via the same repro, in the pre-existing single-container path, not the new code): ReconstructRing/NzCdReconstruct never bounds-checked a match's offset before using it to index the ring/output buffer — malformed or format-mismatched bytecode (observed via -cD, whose distinct nz_lzhds format the current token decoder does not recognize and was misparsing as -cd tokens) could produce a garbage offset whose wrap arithmetic (wi + ring_size - offset) underflows to a huge uint32, indexing far out of bounds and crashing the process. Both functions now refuse (return 0, caught by the existing checksum-gate) on an out-of-range offset instead of computing it — a real fix beyond "return wrong bytes", since the failure mode was a SIGSEGV, not silent corruption. Suite 2553/2553; native_only_v2 total 60/80 unchanged (its corpus doesn't reach the >~8 MB parallel-container path or the crash trigger).
2026-07-03 -cD (nz_lzhds) literal model ported — the last major NanoZip 0.09a compression method closed. -cD shares its entire front end with -cd (same token assembler, column decoding, LZ match/rep-cache model, even the same recon loop — the literal handler is a vtable slot that's a raw-copy stub for -cd and this new model for -cD) and differs only in how literals are decoded: a per-context MTF rank table (256 × 64-byte records: 32-slot rank table + 32-byte presence bitmap, seeded to the identity permutation with bitmap bits 0–31 set, not all-zero — a fact only found via GDB ground truth, missing from the original architecture writeup) plus a 4-stage adaptive linear predictor, fed by an MSB-first bit reader + two-level Exp-Golomb integer decoder over a dedicated -cD-only bitstream whose location in the chunk layout (one raw length byte + that many raw bytes, between the token assembler's extra-bits blob and the literal column) was undocumented anywhere and had to be found by GDB-breaking at the direct call site and cross-checking against the reader-object's own base/end fields. Ported in new files nz_lzhds.{h,cpp} and wired into TryDecodeLegacyLzhd's single-container and parallel-container -cD paths. Landed byte-exact on the first implementation attempt (no debugging-iteration phase needed) once the chunk-layout question was resolved — verified against 3 GDB-captured golden vectors plus 6 real fixtures from 8 KB to 2.5 MB (crossing the 1 MB stream-split boundary), all byte-exact; the -cD safety gate was independently re-confirmed by feeding a known-unsupported stereo-audio -cD archive through the decoder and observing a clean refusal (exit 1, zero output files) rather than a crash or wrong bytes. native_only_v2 total 60→63/80, -cD row 6/10→9/10 (now matching -cd's own remaining gap — a shared rare CM/BWT sub-chunk case). Suite 2553/2553, zero regression.
2026-07-24 -co (nz_optimum1) single-container LZ/CM decode ported — three RE sessions of GDB tracing against the real linux32/nz binary finally found the real decode core. An earlier theory (large -co/-cO use an unreversed "virtual-stream" DecLZ framing) was disproven: DecLZ (a port of the community reference decoder, not the real binary) is simply never called during a real -co decode of any size. Live tracing found the actual per-block decoder is a dispatcher that mostly routes to a "subengine" — for -co this is FUN_0809e600, a distinct LZMA-shaped LZ77 engine (4 repeat-offset slots rep0–3, all init to 1; a cheap 1-bit "new distance vs. reuse a rep slot" selector with the opposite polarity from LZMA's own convention; unary/Elias-gamma-style length and distance coding empirically verified against 90+ and 27+ engineered ground-truth pairs respectively, zero exceptions) driven by a 4-context adaptive literal mixer that shares its low-level binary range-coder primitives (and one of its two per-bit tables, byte-for-byte) with nz_cm.cpp. A promising-looking detour — a pair of genuinely polymorphic "Huffman-shaped" objects found near the dispatcher — turned out to be unrelated dead ends for this codec (confirmed by exhaustive callee-list inspection: -co/-cO never call them at all); they belong to -cc's own, unrelated hash-bank engine. Ported in new files nz_optimum_lz.{h,cpp} + nz_optimum_lz_tables.{h,cpp}, validated byte-exact against 4 GDB-captured golden vectors (plain matches, small/large distances up to a ~300 KB span, and a high-entropy stress fixture with only 54 coincidental matches in 431 KB of otherwise-random output) before being wired into TryDecodeLegacyOptimum, checksum-gated like every other native codec in this project. A real port bug surfaced once wired against full real archives (not just isolated golden-vector blocks): the literal mixer's context-C seed sign-extended a history byte to int8_t before shifting, where the real binary zero-extends first (movzx/sar) — always 0 or 1, never −1 — found by GDB-diffing range-coder state position-by-position against the real binary on the exact failing archive. Also ports NzTextTransformRle (tt_flags & 0x20, an escape+run-length post-filter some -co archives need). Scope: -co single-container archives with decr_param==1 (LZ/CM) blocks only — decr_param==0 (BWT), one remaining text-transform bit (tt_flags & 0x02, the single highest-value remaining gap per fresh-fixture testing), the parallel container (needs no new RE, just wiring), and optimum2/-cO (FUN_080a5d90 — same backbone, GDB-verified byte-exact, but a materially richer 8-context literal engine plus a rolling LZP-style secondary predictor, fully RE'd but not yet ported) all still bridge safely. native_only_v2 -co row 3/10→7/10, total 63→67/80, suite 2553/2553, zero regression on any other codec.
2026-07-24 tt_flags & 0x02 (InsertLF) ported for -co and -cc — the highest-value remaining -co gap. An independent verify pass on the -co port above found that of 12 fresh, non-golden real-archive fixtures, only 5 decoded natively; 4 of the 7 failures shared one root cause unrelated to the LZ-core fix: ordinary real-world prose/text with moderate, non-uniform line-length repetition commonly sets tt_flags & 0x02, a text-transform bit this project's own narrower synthetic test corpus never happened to trigger. Ported NzTextTransformInsertLf, a near-verbatim transcription of the reference decoder's TransformText_3_InsertLF (a self-contained adaptive line-length model with its own small arithmetic decoder + lookup tables, decoding the tt2_data side-stream this project already parsed but had been discarding) — no binary RE needed, the reference source is directly available and complete for this one, unlike the -co/-cO LZ core. Wired into both TryDecodeLegacyCm (-cc) and TryDecodeLegacyOptimum (-co/-cO) at the correct point in the reference's bit-dispatch order. Found and fixed a real heap-corruption bug along the way: enabling bit 0x02 newly reaches tt_flags combinations that also set 0x08 (the dictionary transform) on real text, and NzTextTransformDict's CopyDictEntWithCase always writes a fixed 8/16 bytes per dict word regardless of the word's true length (by the reference's own design, which assumes caller-provided slack) — but the two sfx_archive.cpp call sites allocated their output buffers with zero slack, so a short word landing near the buffer tail could write past the allocation (glibc: "double free or corruption"). Root-caused with AddressSanitizer, fixed by padding both call sites' output buffers and the transform's internal scratch buffer. Verified byte-exact on real-prose fixtures (small/medium/large, both -co and -cc) that previously declined or crashed. native_only_v2 stays at 67/80 (its own corpus doesn't happen to set this bit — a good illustration of why fresh, adversarial fixture testing beyond the tracked suite matters), suite 2553/2553, zero regression.
2026-07-24 -co parallel multi-stream container (flag 0x0f, >~8 MB) → native byte-exact. Extends the single-container -co decoder to the same parallel-container envelope already solved for -cf/-cF and -cd/-cD (verbatim reuse of the generic varint chunk-record parsing loop — this project's own established "shared infrastructure, not codec-specific" pattern). First refactored TryDecodeLegacyOptimum's block-record decode loop into a reusable DecodeOptimumBlockSequence helper (pure refactor for the single-container path, re-verified byte-exact before proceeding) so both container shapes share one decode routine. Confirmed empirically, not assumed: a parallel stream's type-0 chunk begins directly with a block's payload_size — no leading stream_tag, the same framing -cd's parallel streams use — and, unlike -cd (one block per chunk), a single -co chunk can hold multiple back-to-back block records. Each stream gets a fresh NzOptimumLzDecoder using the same archive-wide window-size formula as the single-container path (NzOptimumLzWindowSizeFromP1), with every slice checksum-verified before being placed into the assembled output. decr_param==0 (BWT) blocks and -cO (nz_optimum2) continue to decline cleanly, unaffected by this change. Verified byte-exact on fresh parallel -co archives (15 MB text, forced multi-stream via -p4/-p8) against the legacy oracle. native_only_v2 stays at 67/80 (its corpus doesn't reach the >~8 MB parallel-container threshold), suite 2553/2553, zero regression.
2026-07-29 A real-world file-format sample sweep (56 files, tests/real_corpus_sweep.sh, new) found what native_only_v2.sh's small synthetic corpus (67/80) could not: a real correctness bug, and a big real-audio coverage gap. (1) TryDecodeLegacyCm (-cc) had zero checksum self-verification, unlike its siblings TryDecodeLegacyLzhd/TryDecodeLegacyOptimum (both already self-verify against the archive's stored checksum) — the caller's own cross-check against a bridge decode only runs when a bridge is actually available, so under NZ_NO_BRIDGE=1 (or simply no legacy binary present, this project's whole goal) a wrong CM decode was trusted outright. The sweep caught it concretely: a ~10 MB real file decoded through -cc with 394113 wrong bytes and was about to be written to disk as-is — block framing and total output size both checked out while the entropy-decoded content still diverged, so size alone was never a sufficient correctness signal here. Fixed by adding the same checksum-gate pattern the siblings already use; the underlying CM divergence on that specific archive is a separate, still-open bug, this fix's job was only to turn it into a clean decline instead of silent corruption. (2) -cf/-cF's stereo audio prefilter, previously declared done after validation against exactly one WAV fixture, turned out to be 5/9 on 9 varied real stereo/multichannel PCM WAVs (2/6 channels, 8/16/24-bit) — see the lzpf component-table entry above for the 4 root causes found and fixed (tail-remainder bytes, generic mid/side reconstruction, LMS shift bits, and LPC/LMS state reset across intervening literal blocks, the last one GDB-confirmed). Now 9/9. native_only_v2.sh unchanged at 67/80 throughout (its corpus is too narrow to exercise either bug) — both were found only because a real-world corpus was swept instead.
2026-07-30 -cO (nz_optimum2) single-container LZ/CM decode ported. -cO's real engine, FUN_080a5d90, was re-decompiled fresh via Ghidra (a prior session's capture was gone) and cross-checked against live GDB traces, the same methodology used for -co. Confirmed the backbone (range coder, rep0-3, dispatch bit, length/distance, bulk copy) is a byte-identical-formula scale-up of -co's own engine — several of its lookup tables (DAT_08173140/DAT_08173290/DAT_0813c640/DAT_081b37b0/DAT_08172900) are GDB-confirmed byte-identical to tables already embedded for -co/-cc (kLzModelLNext, kDivideLookup, kLzModelInterpolation) and reused directly rather than re-embedded. The literal-byte coder is genuinely richer than -co's: an 8-context mixer (vs -co's 4) plus a rolling 3-byte-hash LZP-style secondary predictor with no -co analog. Found and fixed 3 real port bugs via GDB ground-truth comparison (a wrong units-to-bytes conversion in the dispatch bit's second APM stage; a missing counter-based addressing scheme in rep-slot selection, where the port had wrongly copied -co's simpler linear scheme; a copy-paste wrong base address in the length decoder's extra-bits loop), plus a 4th, subtler divergence — correct on trivial cold-start input, wrong once mixer/weight state moved past it — that needed a second investigation round and a longer, real-world-shaped stress fixture to even surface, let alone fix. Ported in new files nz_optimum2_lz.{h,cpp} + nz_optimum2_lz_tables.{h,cpp}, validated against 2 golden vectors before being wired into TryDecodeLegacyOptimum for method_p0==6, checksum-gated identically to -co. Scope: -cO single-container archives with decr_param==1 blocks only — decr_param==0 (BWT) and the parallel container still bridge safely, matching -co's own remaining gaps. native_only_v2 -cO row 3/10→7/10, total 67→71/80. Independently re-swept the 56-file real-world corpus: 19/56 native, every decline clean (no crash, no wrong bytes), and every file the native binary actually emitted output for spot-checked byte-exact (12/12) against the legacy oracle. Suite 2553/2553, zero regression on any other codec.
2026-07-30 -cO parallel multi-stream container (flag 0x0f, >~8 MB) → native byte-exact. Extends the existing -co parallel-container branch to also cover -cO (method_p0==6) — the envelope is identical (same generic chunk-record loop, same per-stream block-record framing), only the per-block LZ/CM engine differs. DecodeOptimumBlockSequence was already templated on decoder type (the single-container path already used it for both -co and -cO), and the decoder type is fixed for the whole archive (not per-stream), so it's selected once via a small type-erased closure rather than duplicating the per-stream loop per type — no new RE, purely mechanical wiring now that a working -cO single-container engine exists to call per-stream. Verified byte-exact on a freshly generated, forced-parallel -cO text archive; a parallel random-data -cO archive correctly and safely declines (hits the still-unported decr_param==0/BWT path). native_only_v2 stays at 71/80 (its corpus doesn't reach the >~8 MB threshold), suite 2553/2553, zero regression.
2026-07-30 Fixed a latent word-boundary truncation bug in AddBytesFilter's BitReader (param1 post-filter) — found while wiring the same, supposedly-already-working filter into -co/-cO. BitReader::Initialize computed its fetch boundary as data + (size & ~3), rounding the side-stream length DOWN to a multiple of 4 bytes; the reference decoder uses the exact byte length with no rounding, so whenever a real param1 side-stream wasn't a multiple of 4 bytes, the reference still performs one more partial-word fetch to reach the final 1–3 trailing bytes — bits the decode genuinely needs, since nothing pads the stream to a u32 boundary. -cc (param1 was believed "done" since an earlier session) had never actually been exercised against a real side-stream of this exact shape by its own synthetic test corpus, so the bug went undetected until a real-world file-format corpus sweep hit it (e.g. real .doc files). Confirmed via reference-source diff (nzdec_v0/src/nz.h, line-for-line match except this one truncation) and ground-truth replay: the bug computes a corrupt offset that underflows to ~4 billion, which the existing bounds check correctly rejects — so this was always a coverage gap (clean decline), never a correctness risk. Fixed by tracking the exact unaligned end and zero-padding only the genuinely-missing trailing bytes, bounded strictly within the caller's buffer. The fix transparently improves -co/-cO too, since DecodeOptimumBlockSequence calls the identical function (wired into the optimum family in the same session, previously an unconditional decline). Real-corpus sweep (56 files): -cc 25→31/56, -co 18→20/56, -cO 19→21/56. native_only_v2 unchanged at 71/80 (doesn't trigger this bug), suite 2553/2553, zero regression.
2026-07-30 -co/-cO single-container "chain" mode (multiple stream_tag segments) → native byte-exact. TryDecodeLegacyOptimum parsed exactly one top-level stream_tag varint per single-container archive entry and declined if that segment's blocks didn't cover the entry's whole declared output size. Real files can have MORE THAN ONE such segment, directly concatenated with no separator or checksum record between them (unlike -cf/-cd's own chain mode, which has an inter-segment whole-output checksum) — found via a real 60 KB .doc file whose first segment covered only 10229 of 60416 declared output bytes; the very next byte was a second, independently valid stream_tag varint (low nibble 0, in-bounds stream_bytes). An earlier RE session's "chain mode doesn't exist for -co" conclusion was apparently reached from insufficient (large-synthetic-file-only) fixtures. Fixed by looping: keep reading stream_tag segments and decoding their block sequences into the same output buffer, via the same persistent per-entry decoder (its ring/dictionary window and probability tables carry over across segments, same as within one segment), until the entry's declared total output size is reached. The decoder type (-co's NzOptimumLzDecoder vs -cO's NzOptimum2LzDecoder) is selected once via a small closure, mirroring the parallel-container branch's existing pattern. Verified byte-exact on the real repro file (3 chain segments) for both -co and -cO. Real-corpus sweep (56 files): -co 20→26/56, -cO 21→27/56 (+6 each). native_only_v2 unchanged at 71/80 (its corpus doesn't happen to produce multi-segment archives), suite 2553/2553, zero regression. A separate, already-tracked param2 bug still correctly declines cleanly on the one corpus file that hits it, unaffected by this change.
2026-07-30 Fixed CM cross-chunk continuation bug (-cc) — the file that still declined after the chain-mode fix, initially misdiagnosed as a param2 bug. integrty.doc still failed under -cc; the failing function was param2's NzBwtRleDecodeU32 RLE expander, which is what an earlier session had assumed was buggy. Added NZOPT_TRACE_RLE-gated instrumentation and found param2's own logic was correct — line-for-line match of the reference, including its run_len > 30 sanity check correctly firing on genuinely-corrupt input (run_len=543, last_v=0xffffffff, a 543-word run of literal 0xff). Traced one level upstream: the archive has 3 records (CM chunk → param6==0 stored chunk → CM chunk, both CM chunks decr_param==0 so NzCmReset never fires between them), and the second CM chunk's own decode already degenerated into the 0xff run partway through — the CM decode was producing garbage before param2 ever saw it. Root cause: the CM model's rolling byte-window/hash context must observe every byte of the true output stream in order, including stored (non-CM-coded) bytes, to stay in sync with the true decompressed-output position for the next CM chunk; the port never fed stored bytes through the model. Validated with unusual rigor: built the community reference decoder (nzdec_v0) standalone, fed it the exact real payload bytes/CM config extracted from the real archive, and reproduced the identical corruption — ruling out a port/reference divergence — then confirmed the fix hypothesis against that same reproduction before touching the port. Fix: new NzCmFeedByte(cm, byte) (nz_cm.{h,cpp}) runs the same per-bit CM_Input_Bit update path CM_Decode uses, without consuming arithmetic-coded input; called from TryDecodeLegacyCm's stored-block branch for every stored byte before it's appended to output. Lesson recorded: the last function to fail and the function with the actual bug are not always the same function. Real-corpus sweep (56 files): -cc 31→32/56 (-co/-cO unaffected — the fix is specific to -cc's own stored-block branch). native_only_v2 unchanged at 71/80 (its corpus doesn't produce this decr_param==0 + stored-chunk shape), suite 2553/2553, zero regression. Verified byte-exact against the exact binary built from the pushed commit, not just the dev-tree build.
2026-08-03 decr_param==0 (BWT) decoding for -co/-cO → native byte-exact, in both shapes. BWT was the dominant real-world decline reason for both codecs; DecodeOptimumBlockSequence declined every such block outright. Two things were missing. The header layout: a non-CM decr_param==0 block carries param7 (only when param6 is set), a u32 inverse-BWT start position, and params 14/15 — none of which exist in the decr_param==1 (LZ) layout. Parsed now, and confirmed exact: on every traced block the read cursor lands precisely on the record boundary. The decode, which has two distinct shapes, both observed on real data. With param6==0 there is no entropy layer at all — the encoder found the BWT output incompressible and stored it raw, so there is no size18 field and the block expands to exactly its own payload size; only the inverse BWT runs (NzBwtUntransform, new files nz_bwt.{h,cpp}). With param6==1 the BWT output is wrapped in 256 independent per-leading-symbol buckets, each an arithmetic-coded move-to-front rank stream plus an optional byte-wise RLE expansion (NzBwtDecodeInput, with BwtUnpackInput/BwtIntModel/ReadSomeValue+2 tables/BwtRleExpander::Decode1/BackwardsByteStream ported alongside). The port deviates from the reference in three deliberate, safe ways: it always uses the general index-table form of the inverse BWT rather than the packed byte|index<<8 variant used below 16 MB (same permutation, but the packed form caps the addressable index at 2^24); it owns its scratch buffers instead of aliasing the index table onto the bytes past data (which in the reference silently requires every caller to over-allocate by 4*data_size+3) while still keeping the per-bucket 3*out_size stride that BwtUnpackMain genuinely depends on; and the run splat writes exactly num_rle bytes rather than rounding up to 4 and relying on the next run to overwrite the overshoot. Also range-checks bwt_start_pos and declines instead of reading out of bounds. Along the way this surfaced a compiler-visible latent bug inherited from the reference. Its rank-insertion loops read one element past their own uint8 P[256]: for (; new_c >= C[P[k + 8]] && k != 249; k += 8, new_c += 8). At k == 249 the left operand reads P[257] and the value is discarded, since the k check ends the loop on that same iteration — so it looks harmless, and is at -O0/-O1. At -O2 gcc is entitled to assume P[k+8] is in bounds, hence k <= 247, hence k != 249 is always true, and to delete the bound; the loop then runs away (k observed reaching 2313) and P[k] = last_rle writes far past P[255], corrupting the C[] array the compiler had placed immediately after P — surfacing much later as an absurd num_rle and a declined block. Testing the bound before the read removes the UB and restores the intended semantics. Found with ASAN (which flagged only the read, since at -O1 it does not exploit the UB) plus an A/B revert: with the reference form a real 1.5 MB BWT block fails, with the bound-first form it decodes byte-exact. The reference decoder is presumably miscompiled the same way at -O2, which may be part of why it is known to get some -co/-cO edges wrong. Adds NZOPT_TRACE_BWT-gated diagnostics at every rejection point. Shipped as 58c11b2 (raw-BWT shape) + 6b30a43 (entropy layer). native_only_v2.sh -co/-cO 7/10 → 9/10 each, TOTAL 71 → 75/80 (the remaining 5 are all one audio fixture). Real-world 52-file corpus, same corpus measured before and after: -co 29→32, -cO 27→31. Suite 2553/2553, test_optimum_lz 4/4, test_optimum2_lz 2/2, ASAN+UBSAN clean, zero regression, verified byte-exact against the binaries built from the pushed commits. Still declining cleanly: BWT param14/param15 (observed on a real .stw and .mdb), plus the already-tracked tt 0x04/0x01 bits and the dece exe-filter.
2026-08-03 decr_param==2 audio blocks → native (AudioPred ported), and the discovery that NanoZip has TWO audio bitstream families. Every audio-bearing archive under -cc/-co/-cO used to die before its first block trace, which turned out to be a header-shape bug rather than a codec gap: a decr_param==2 block reads a mode2_type byte, forces param6 to 1, reads size18 and then STOPS — no staged-checksum count and none of the param2/param1/param16/text-transform/dece fields — so the ordinary parser took mode2_type for param6 and walked into the following record. decr_param==3 shares the truncated shape; a 357-archive / 654-block sweep found it never occurs, and the reference's own fall-through for it reads uninitialised Header fields, so the port parses the shape and treats every post-filter as absent. New nz_audio.{h,cpp} ports the predictor: AudioPred with its 0x10000 output-chunk framing, AudioBitcountDecoder (arithmetic-coded magnitude classes with a zero-run escape, seeded from the shared kModelLutLookup), six LinearPredictors over a 512-stride dual history, and AudioStereoDecoder's cross-channel sign-LMS. The predictor is stateful across blocks and the reset rule is reproduced exactly (every non-audio block resets it; an audio block only when mode2_type is set), living per archive entry, shared across chain segments, fresh per parallel-container stream. Deliberate deviations, all commented in place: the BitReader advances a full 4 bytes per fetch even on a trailing partial word because BytesRead() feeds the chunk's input accounting (clamping it the way nz_postfilter.cpp's copy does would desynchronise the next chunk) while zero-filling rather than over-reading; the reference's 320 KB of stack scratch became members; and header_bytes is bounds-checked, bit-count classes are validated to 0..63 before indexing four const tables, and every int32 lane is written with explicit unsigned wraparound — not cosmetic, since UBSan showed the reference's signed overflow really does occur, and leaving that UB in place is precisely what let the optimiser delete a loop bound in the BWT port earlier the same day. Shipped a2c6861. -cO byte-exact on stereo and mono at 8/16/24-bit and on a 6 MB multi-block file exercising mode2_type=1 plus cross-block state; native_only_v2 -cO 9/10 → 10/10, TOTAL 75 → 76/80. -co/-cc still decline, and the differential says why: their output is bit-identical to the reference decoder's on the same payload (reference AudioPred built standalone and fed the exact bytes), so the port is faithful and the reference itself is wrong for the shapes those two emit — the discriminator is which linear predictors the encoder enables (-cO = {0,1,3}, -co adds 4, -cc adds 2+5), and sweeping the third pair's order over 13 values does not fix it, so the next step is GDB against linux32/nz rather than more guessing. Separately established: -cd/-cD audio is a different family altogether — chunk-header varint 0x0c, the same construct as the already-ported -cf/-cF audio block (0x04) with bit 3 set, selecting real-binary FUN_080a9ca0 instead of the ported FUN_080a5bb0; AudioPred is not the right decoder there. Suite 2553/2553, test_optimum_lz 4/4, test_optimum2_lz 2/2, ASAN+UBSAN clean on all three audio paths, zero regression, verified against the binary built from the pushed commit.
2026-08-31 BWT param14/param15 ported, and a window-synchronisation bug they exposed. Both are BWT-only follow-on transforms running after the inverse BWT and before the shared param2/param1/text-transform/dece chain, and both are LZ77 passes driven by their own arithmetic-coded side stream that locate matches by scanning the byte stream for a two-byte escape tag rather than coding literal/match flags. param14 (reference DecodeLZ_Param14, NZ_LZ.cpp:543): tag 0xfe 0xf1 then a selector byte (1 = the tag was literal data, 0 = a match), offset relative to the current output position, four repeat-offset slots. param15 (reference DecodeParam15, NZ.cpp:843): tag 0xfe 0xf0, and the match source is an absolute offset — four raw big-endian one's-complement bytes taken from the byte stream, not from the coder — into the whole accumulated output stream, so a match can reach back into earlier blocks; the caller splices the current block onto out_data to build that window and rolls it back afterwards. Neither reference function bounds-checks its offset (a corrupt stream walks off the front of its buffer); both ports decline instead. Naming trap recorded: this param14 is not nz_cd_tokens.cpp's NzCdParam14, the -cd char-class space-insertion text transform — same name in the original, completely different algorithm. The second half is the more valuable finding. The first real file to clear the new param14 gate still failed, and not on a BWT block — on a later, ordinary LZ block. In the original the LZ window is the shared accumulated-block buffer that every block writing into it advances (mem->data += size), and that includes a BWT block's post-param14/15 output; only audio blocks are exempt, because DecodeFromStream returns before touching the window for those. This port keeps a private ring that only DecodeBlock ever wrote, so a later LZ match reaching back into a BWT block's output read stale ring bytes and the block failed. Fixed by adding FeedWindow to both optimum decoders and calling it with the block's pre-post-filter bytes (what the window actually carries), never for audio blocks. Same class of bug as the CM stored-block desync fixed in 25d2f75 — worth remembering as a recurring shape: whenever a block bypasses the usual decoder, ask what shared state the original still advanced. One edge deliberately left documented rather than guessed: if a BWT block ever exceeded the ring capacity, the exact cursor the original would land on is unverified (no observed block does), and it would decline on checksum rather than emit wrong bytes. Shipped 296ee88. Measured on one identical 60-file real-world corpus, with a baseline binary built at caeb56d for honest attribution: -co 43→52, -cO 45→54 (+9 each, 72%→88%) — the largest single jump of any fix in this project, bigger than chain mode's +6/+6. Byte-exact on 12/12 targeted repros covering param14 alone, param14+param15 together, and the window-feed case. native_only_v2.sh unchanged at 76/80 — its synthetic corpus produces no param14 block at all, which is why this gap survived so long. Also learned: param14 is common, not rare (~20 of 220 sampled real files, across audio/music/image formats, not just the two documents that first surfaced it). Suite 2553/2553, test_optimum_lz 4/4, test_optimum2_lz 2/2, ASAN+UBSAN clean on every repro, verified against the binary built from the pushed commit.
2026-08-31 Text-transform bits 0x04 (HTML) and 0x01 (CR/CRLF) ported — the tt gap is closed — plus UB fixes in the paths this newly reached. 0x01 is TransformText_CR_to_CRLF (reference NZ_TextTransforms.cpp:402), applied LAST in the chain after 0x20/0x40: a state machine that re-expands a bare 10 into 13,10 or rewrites a 10 back to a lone 13. Its budget quirk is faithful but made safe — the reference starts the output budget at out_cap + 1 and only detects the overrun after writing that extra byte, so the port documents the requirement and the callers allocate one spare byte instead of overflowing. 0x04 is HtmlTransformer (reference NZ_TextTransforms.cpp:781): the encoder shortens a closing tag to a bare </ and the decoder rebuilds the name from a stack of opened tags, with <// escaping a literal </. The stack's lossiness is reproduced exactly (128 entries, names truncated to 16 bytes, a 4-entry recent-tag ring, and a fixed set of predefined tags never stacked because they never need closing); unlike the reference the name/length arrays are zeroed so a malformed stream cannot read indeterminate stack memory. Also wired 0x20 into -cc, which the optimum path already had — flagged in-code as not verified for -cc (no -cc archive in the 60-file corpus sets the bit, so there is no repro), but it is the identical codec-agnostic function and a wrong result declines on the entry checksum as today; both tt gates now accept the same mask. Getting these files to decode pushed execution into tt16 and the CM window matcher for the first time on this content, and ASAN/UBSAN immediately flagged pre-existing UB there — none of it in the new code: nz_texttransform_num.cpp shifted a negative int left in two places (now shifted as the unsigned bit pattern — bit-identical on two's complement, but no longer something the optimiser may act on, the same UB class that let gcc delete a loop bound in the BWT port), and nz_cm.cpp compared window bytes through unaligned uint16/uint32/uint64 casts in six places (now memcpy helpers — one instruction on x86, defined everywhere). Worth noting as a pattern: closing a coverage gap routinely exposes latent UB in code that was previously never reached on that content, so re-run ASAN after every gap fix, not just after touching the sanitised file. Shipped fe07c69. Measured on one identical 60-file real corpus with a baseline built at a3367ae: -co 52→55, -cO 54→57, -cc 51→54 (+3 each). Byte-exact on 9/9 repro combinations (3 files × 3 codecs). native_only_v2.sh unchanged at 76/80 — no synthetic fixture sets either bit. Suite 2553/2553, test_optimum_lz 4/4, test_optimum2_lz 2/2, ASAN+UBSAN clean on all 9 combinations, zero regression, verified against the binary built from the pushed commit.
2026-08-31 dece x86 exe-filter ported — the post-filter chain is now fully native — plus an out-of-bounds WRITE that fuzzing exposed in both optimum engines. dece is an x86 CALL/JMP address un-relativiser and the last step of the chain (reference DecodeFromStream: param2 → param1 → text transforms → dece). New nz_exefilter.{h,cpp} ports ExeFilter from NZ_x86.cpp: three adaptive model families (mode, offset, jump-recent) over the shared 12-bit range coder, an MTF cache of 3 recent call targets and 256 recent jump targets, and three separate input streams — the dece_data side vector (two backwards varints at its tail, arithmetic-coded stream in front), plus a raw tail carved off the block's own input holding the add-esp immediates and the big-endian 32-bit call targets. Both wiring sites also had to start capturing dece_data; they previously only skipped it. Position arithmetic: the reference works in truncated pointer values (base = (uint32)out_org - exe_base_, stored = offs + base - (uint32)out) — those absolute addresses cancel exactly, leaving only the output offset, and exe_base_ is dead state (assigned 0 in the constructor, and the reference builds a temporary filter per block). The port works in offsets: bit-identical, and unlike truncating a 64-bit pointer to uint32, well defined. The algebra also collapses the non-recent jump case to "stored == the model value". Ten reference hazards hardened, all reachable on malformed input: unchecked reads off both raw side streams; a one-byte peek past the input end; a bit-count that can reach 31 and index model_b_[1922] one past its own 1922 entries — an OOB read and write landing on model_c_; up to 7 bytes written past the output cap (the room check runs once per iteration, before a 4- or 7-byte write); unaligned uint32 stores that also violate strict aliasing; a 32-bit size_t overflow in the end_bytes guard; memset32 over uint16 arrays with no guaranteed alignment; and uint8int promotion in the big-endian assembly. Plus a gate requiring both raw side streams to end exactly consumed, which turns a silent-wrong-output path into a clean decline. An independent spec agent fuzzed 40 000 inputs against the reference and converged on the same ten hazards with measured trigger counts (e.g. the nbits == 31 OOB fires 50/40 000, and max nbits on real data is 23 — so refusing is safe). Separately, fuzzing single-byte corruptions of a real dece archive under ASAN found a pre-existing defect unrelated to dece: the unary length loop in DecodeBlock has NO upper bound in either optimum engine, so a corrupt bitstream walks the model offset up 2 bytes at a time past mem_ — and DecodeAdaptiveKSB both reads and writes the cell, making it an out-of-bounds heap write. Now bounded against the buffer; a valid stream stays inside the length table far below it. 40 corruptions of a -co archive plus 24 across -cO/-cc now run clean under ASAN with no crashes. Lesson: fuzz the decoder on corrupt input, not just the happy path — a decompressor's threat model is malformed archives, and neither the real-corpus sweep nor byte-exactness testing would ever have found this. Shipped 648df9e. Measured on one identical 60-file real corpus, baseline built at fe07c69: -co 55→57, -cO 57→59, -cc 54→56 (+2 each); -cO is now 59/60. Byte-exact on 11/11 dece-bearing cases (both corpus files × three codecs, plus five more real executables under -co); dece_param was 1 in every case observed. native_only_v2.sh unchanged at 76/80 — no synthetic fixture carries a dece block. Suite 2553/2553, ASAN+UBSAN clean, zero regression, verified against the binary built from the pushed commit. With this in, every remaining -co/-cO/-cc corpus failure is the wrong-bytes class — decodes that complete at the declared size with all blocks reporting success.
2026-08-31 Correction to the dece state model: filter state must persist across a RUN of consecutive dece blocks. 648df9e reproduced the reference literally — size = ExeFilter().Decode(...) builds a temporary, so the recent-call/recent-jump caches reset every block and exe_base_ stays 0. The reference is wrong here: A/B-tested against real archives, that model produces wrong bytes on 22 of 88 dece-bearing archives. The model that is exact everywhere: the recent-target caches and the base persist across a run of consecutive dece blocks; the base counts the output bytes produced so far by that run (so the run-relative position is P = base + (out - out_org)); and everything resets as soon as a block without dece intervenes. The per-block probability models are not part of that state — they are locals of the reference's Decode and stay fresh either way. NzExeFilter is now a class the caller keeps per stream: Decode() per dece block (advancing the base by what it produced), Reset() on any block with no dece field; parallel-container streams get their own instance because the base counts stream-local output, not the file-absolute offset. Member/file boundaries inside one stream do not break a run. Why the original tests missed it: the two models differ only when two dece blocks are adjacent, and every file tested in 648df9e carried exactly one — so the model was wrong for a quarter of real dece archives while passing 11/11 repros. It declined rather than corrupting (the guards plus the entry checksum caught it every time; 0 wrong bytes across the whole repro set under both models), but it was a real coverage bug. Found by an independent spec agent that built the reference decoder and A/B-tested four candidate state models against 88 real archives — not by the port's own tests. Shipped 491a54d. Measured on a purpose-built 48-case dece repro set (16 real executables × 3 codecs), baseline built at 648df9e: 33 pass / 15 decline → 44 pass / 4 decline, 0 wrong in both. All 15 previously-declining cases are exactly the multi-dece-block ones; 11 now decode byte-exact and the remaining 4 fail for unrelated reasons (one hits a later gap; one completes at the declared size but mismatches its checksum, i.e. the wrong-bytes class). The 60-file real corpus is unchanged at -co 57 / -cO 59 / -cc 56 — it happens to contain no archive with two adjacent dece blocks, which is exactly why a dedicated repro set was needed. native_only_v2.sh unchanged at 76/80, suite 2553/2553, ASAN+UBSAN clean on multi-run archives, zero regression. This is the FOURTH instance of the same bug shape in this project — state the original advances across blocks that a per-block reconstruction drops (after CM stored blocks, the audio predictor reset rule, and feeding non-LZ block output into the LZ window). Check for it proactively on any new block-scoped component.
2026-08-31 -cD literal model: the predictor stage index never advanced across matches — 34 → 52/60 on the real corpus, the single biggest fix of the session. The first full 8-codec sweep on ONE corpus (-cn 60, -cO 59, -co 57, -cd 56, -cc 56, -cF 53, -cf 52, -cD 34) showed -cD sitting 22 points below its near-identical sibling -cd — a gap nobody could explain because nz_cd_tokens.cpp was the only codec in the tree with no instrumentation at all, and because TryDecodeLegacyLzhd sets an error message the caller never prints, not even under -v. After adding NZOPT_TRACE_CD plus a pre-checksum dump (460d126, e4e9bee), the isolating set was the 22 files that fail under -cD but PASS under -cd — since the two share the entire front end and differ only in the literal model, that pointed at nz_lzhds. Comparing the two codecs on misc/servers.mdb showed identical chunk headers (out_size 4157/8615/6304, flags 0x3) and identical ring advances (4157/12772/19076), clearing the header parse, the token layer and the -cD-only ratebits field; chunk 1 decoded byte-exact with the first divergence 6325 bytes into chunk 2. Root cause: the real FUN_080982e0 does local_5c = uVar13 + local_5c (uVar13 = the match length) and only then local_5c %= local_58; the port did the modulo alone, so every match left the stage index behind by mlen and the error accumulated. That is exactly why literal-only regions were byte-exact while files with matches diverged at a content-dependent offset (Lingo.Icons at byte 4211, inside chunk 1). Hypotheses ruled out and recorded so they are not retried: the text_param read offset (the file first blamed for it fails under -cf/-cd/-cD alike — it is the shared prefilter-chunk gap, so two codecs failing for an unrelated reason were being compared), and resetting the MTF context table per chunk instead of persisting it (A/B: strictly worse everywhere). Also fixed a pre-existing out-of-bounds read this exposed: lzpf_arith.cpp's ReadBits guards only cur < end then loads a full 32-bit word, so with 1–3 bytes left it reads up to 3 bytes past the buffer — an ASAN heap-buffer-overflow that only began firing once -cD decoded files it previously declined. Now zero-fills the missing tail while still advancing the cursor a full word. Third time this session that closing a coverage gap exposed latent UB in newly-reached code. Shipped c831c31. -cD 34 → 52, -cd unchanged at 56 (nz_lzhds is -cD only); of the 8 remaining -cD failures, 4 are the same hard files -cd also fails and 4 are -cD-specific. native_only_v2.sh unchanged at 76/80 — no synthetic fixture has enough matches in a -cD predictor run to trigger it. Suite 2553/2553, ASAN+UBSAN clean, zero regression.
2026-09-01 -cf/-cF never applied the x86 exe un-transform (bit 2 of the block header) — -cf 52→55, -cF 53→56. The real dispatcher FUN_08097570 serves the whole 0x2b family (-cf/-cF/-cd/-cD) and does, for every non-prefilter block: if ((uVar9 & 4) != 0) FUN_080c0540(param_3, *param_4, param_3, param_1[0x4018], -1); param_1[0x4018] += *param_4;. The -cd path already had this as its chunk flag &4 (NzCdExeUnfilter), but the -cf/-cF path only tested bit 2 as part of the (uvar9 & 7) == 4 prefilter check and otherwise ignored it — so every E8/E9 displacement in an executable stayed in the encoder's absolute form. batnball.exe came out with 0x00000410 where the file has 0x0000000b (CALL at 0x400, field at 0x401..0x404, next instruction at 0x405, so 0x405 + 0x0b == 0x410 — the port emitted the absolute target instead of the relative displacement). Two details that matter: it filters the output copy only — the window keeps the unfiltered bytes because later matches reference the unfiltered window (the original copies window→param_3 then filters param_3 in place; filtering the window would corrupt every subsequent match); and the prefilter branch is excluded(uvar9 & 7) == 4 also has bit 2 set, but in the original that branch has its own position-counter update and never reaches the exe test. How it was found: of the 15 -cf/-cF corpus failures, 7 were common to both codecs (so one root cause, not fifteen) and 4 of those passed under -cd → another byte-exact oracle, and 3 of the 4 were executables. Dumping the -cf output put the first divergence at byte 1026 with e8/e9 opcodes sitting right there, naming the filter directly. Also fixes a pre-existing misaligned load in the same path: the hash_table backfill read the window through a uint32_t* cast at byte-granular offsets (UBSan "load of misaligned address"); now memcpy. Fourth instance this session of latent UB surfacing in code a coverage fix newly reached. Shipped f885e1c. All three executables byte-exact under both codecs; what remains is almost entirely the shared hard set most codecs fail (Moly, summer.php, M05.AMF, RESOURCE.001) plus one audio file. native_only_v2.sh unchanged at 76/80 — no synthetic fixture is an executable. Suite 2553/2553, ASAN+UBSAN clean, zero regression.
2026-09-01 A prefilter block must backfill hash_table too — -cf and -cF are now 60/60 on the real corpus. A prefilter block publishes window content that a later LZ77 block can match into, so it needs the same hash backfill a literal block gets; the real dispatcher FUN_08097570 calls FUN_080b6d90 at the end of its prefilter branch for exactly that. This port only did it after literal blocks — with a comment explicitly (and wrongly) narrowing it to "literal blocks — NOT LZ77 blocks", which overlooked the third case entirely. Without it, an LZ77 block after a prefilter block reads the table's untouched init value instead of a real window offset. On image/summer.php under -cf: blocks 0–2 byte-exact, first divergence 951 bytes into block 3 — the first lz-side block after the stream's single prefilter block. A prefilter block has (uvar9 & 7) == 4, hence uvar9 & 1 == 0, so it takes the sparse variant (every 101 bytes, stopping 100 short of the block end), matching the literal path's own uvar9 & 1 == 0 case; variant B additionally resets byte_buffer_8k as it does after a literal block. Same bug class as 41dbc59 (literal blocks never backfilling the hash table), one block type later. Shipped eb66e8c. -cf 55 → 60, -cF 56 → 60. It also closed image/Moly, music/M05.AMF and music/RESOURCE.001, which had been treated as a separate "shared hard set" — they were the same defect, which is why the set looked hard: four files, one cause, spread across codecs. -cd/-cD unchanged at 56/52 (they do not use the lzpf block path). native_only_v2.sh unchanged at 76/80 — its one lzpf fixture with a prefilter block has no LZ77 block after it. Suite 2553/2553, ASAN+UBSAN clean, zero regression. Process note: mid-diagnosis I briefly reported verify() producing a false negative on a byte-exact decode. That was wrong — the comparison used a stale dump left over from the sibling codec after a shell glob failed to clear it. verify() was correct throughout and the decode really was wrong by 16434 bytes. Clean per-codec output directories, not shared ones, when diffing two codecs' dumps.
2026-09-01 The -cd/-cD prefilter sub-chunk decodes natively — native_only_v2 76 → 78/80, and the long-standing "rare CM/BWT sub-chunk" gap is identified and half-closed. The chunk header's low nibble is a dispatch value, not a bit mask: per FUN_080994b0, 0xf is a CM sub-chunk, (nibble & 0xc) == 0xc is a prefilter sub-chunk, and only the remaining values carry the &1/&2/&4/&8 meanings the port applied to all of them. Reading 0xc as "exe|text" made it parse a garbage text-pipeline param and decline with an "unsupported tt bit" message unrelated to the real problem. That chunk is decoded by FUN_080a5bb0 — the SAME core -cf/-cF already use. Not BWT, not a separate audio codec: the captured notes call it "BWT" in the -cd context and an "audio core" elsewhere, and both labels are wrong — they sent several sessions down blind alleys. What differs between the four codecs sharing it is the STATE OBJECT that FUN_080b1600 configures, with GDB-measured immediates: -cf order01=4/nstages=1, -cF 8/1, -cd 8/3, -cD 32/3. nstages is the number of cascaded stages, each owning planes 2s and 2s+1; a block's side-bit header carries one activation bit per plane per stage plus an optional 3-bit shift, and stages apply in descending order. The port hardcoded nstages = 1, so for -cd it read 4 side bits where the real decoder reads 6 — a 2-bit desync of the residual escape stream. That is exactly why the symptom was "right framing, wrong content", and why toggling is_stereo_variant changed nothing: these chunks are mono, so that flag is a no-op for them. LpcPredictor/pred2 therefore became an explicit PrefilterContext of six planes, with -cf/-cF's behaviour falling out as the nstages == 1 case (order01 4 or 8 reproduces the old taps 4/8 selection exactly). Also fixes a separate framing bug: for a 0xc-class chunk the generator FUN_08098cf0 returns early before the second varint (asm 0x08098d7f, 0x08098f30) and FUN_080994b0 takes the size from gen+0x2c, so out_size = size_field - 1 and there is no second varint in the stream — the port read one unconditionally, eating the first payload byte of every 0xc chunk with a non-zero size field (sizes 512/1218/7072/31098/31854/32232 all observed). State lifetime, both rules measured and load-bearing: every LZ chunk resets the whole state object (FUN_080b1950), while a CM chunk and a 0xc chunk with flags & 2 == 0 do not — so state persists across adjacent prefilter chunks. One context per archive, matching the single instance at obj+0x40. Shipped cd5e3f3. native_only_v2.sh 76 → 78/80: -cd and -cD both reach 10/10, closing the stereo_lms.wav failure open for several sessions. Real corpus -cd 56 → 58; -cf/-cF held at 60/60, which was the regression to watch since the refactor touches their shared code. -cD is unchanged at 52 and stays open: its planes 0/1 have order 32, needing FUN_08095d90's order >= 9 branch. LpcBigPredictor is in-tree (ported from the scalar reference FUN_080bddc0, proven output-identical to the MMX path by forcing DAT_081835b8 = 0) but is not yet byte-exact. Suite 2553/2553, zero regression.
2026-09-01 -cd and -cD both reach 60/60 on the real corpus (7a4d4c5): two independent cross-chunk state bugs. (a) The 0xc prefilter branch returned out_size where *recon_advance means the NEW ABSOLUTE ring position — so the next LZ chunk inherited the previous chunk's write base, overwrote the prefilter output it was meant to match against, and resolved every match offset from the wrong origin (regression from cd5e3f3; also the real cause of the two -cd residuals that had been filed as an LMS detail). (b) -cD only: a full stored chunk (header size field == 0) RE-INITIALISES the nz_lzhds per-context MTF table and its order-1 context index — scope verified in three directions, since resetting every chunk, resetting on a prefilter sub-chunk, or resetting on the v2 == 0 pure-literal flavour each breaks a different file. Real corpus -cd 58 → 60, -cD 52 → 60; 8 codecs 462 → 472/480.
2026-09-01 The -cD "order-32 predictor" item was a misdiagnosis: the in-tree LpcBigPredictor is byte-identical to the validated ord32.h, so the recorded next step (diff them) returned an empty diff. Added NZOPT_TRACE_LZHDS (the path had zero instrumentation) plus mixed_audio_text_229k.bin — audio + text + deterministic high-entropy + audio + text — because every existing fixture is a single kind of data and the suite reported -cd/-cD at a clean 10/10 while both bugs were live. Synthetic suite now 84/88 (11 fixtures × 8 methods); all four failures are the one open -co/-cc audio gap.
2026-09-01 Stored LZ block decoded instead of skipped (7f07b6d): param6 == 0 means "no compressed layer", and the port recognised that only for BWT blocks -- a decr_param == 1 block with param6 == 0 was skipped ENTIRELY while the sequence reported success, so the stream came up short and the file was declined as "decode failed". The reference is no oracle here (its LZ branch reads size18, which is assigned only under if (param6), so at param6 == 0 it reads an uninitialised field); the sizes settle it -- PowerPacker.pp's 40492 minus 20091 from its two BWT blocks is exactly the block's 20401-byte payload. Real corpus -co 57 -> 58, total 473/480. Also re-classified the remaining 11 failures by trace: they are THREE defects, and the audio gap alone accounts for 9 of them (the page had called it "the last 2 synthetic failures").
2026-09-01 GDB ground truth for the decr_param == 2 audio defect (926a2a0, no behaviour change): the real stage sequence is -cO: FUN_0809bbf0 -> FUN_08095d90 -> FUN_08096160, and -co/-cc: FUN_0809bbf0 -> FUN_080958d0 -> FUN_08095d90 -> FUN_08096e20. So the audio path is NOT a separate algorithm family -- it is built from the SAME primitives as the lzpf prefilter, three already byte-exact in-tree (DecodeResidualsStereo, whose comment wrongly calls it never-the-real-path; LpcPredictor; LmsObject), plus two unported (FUN_080958d0, FUN_08096160). The residual array is byte-identical across all three codecs, and this port reproduces it exactly for -cO only -- so the reconstruction is fine and the defect is entirely in residual production. Adds NZOPT_DUMP_AUDRESID.
2026-09-01 Audio: the missing inter-channel branch wired (f54131b, necessary but not sufficient — no file flips yet). FUN_080a5330 picks its inter-channel stage from bit 4 of the decoder object's flag byte (GDB-read: -cO 0x03, -cc 0x0f, -co 0x13): clear = FUN_08096160 (4+4 bits, +0x10), set = FUN_08096e20 (3+3 bits, +7). -co is the only one with bit 4 set and the port had only the clear branch, reading 4+4 always — two bits too many, desynchronising everything after. Wired LmsObject/ApplyLmsInterChannel (already byte-exact in-tree) as the alternative. -co residual error 127427 -> 103969 of 128000. Also corrected: there is NO missing FUN_080958d0 — that PC is a branch inside FUN_08095d90, and the push %ebp heuristic that named it is unreliable here. Remaining difference is the residual FRONT END (real: FUN_080c0630 + 2x FUN_080a4ea0 + FUN_0809bbf0; port: the reference's AudioBitcountDecoder + DecodeInt32Array).
2026-09-01 Audio: both remaining causes localised, and they are different (a0ec6f5, no behaviour change). -co's bit-count decoder is a DIFFERENT CLASS reached by vtable (-cO/-cc = 0x0813c848 -> FUN_0809c070, which the port transcribes and gets byte-exact; -co = 0x0813c860 -> FUN_0809bdc0, unported, a Fenwick-tree coder with no decompile). -cc instead has byte-exact bit-counts AND post-residual-decode, so its defect is in the predictor stages: stage 1 (linpred[2], shift 13) diverges at element 2 where the real predictor contributes 0 and the port -1 -- the sum differs, not the rounding. Adds NZOPT_DUMP_AUDCOUNTS/_AUDPOST/_AUDPLANE. Retracts the previous plan to replace the residual front end: that is the (*param_1 & 1)==0 branch and all three codecs have bit 0 set.
2026-09-01 The audio decoder's constants are PER-CODEC; -cc now byte-exact (113326b). Every configurable constant in nz_audio.cpp was frozen at -cO's value -- the entire reason -cO alone matched the binary. GDB-read: six predictor orders, one per pair (plane obj +0x1c08) = -co 64/8/8, -cO 96/8/8, -cc 384/16/8; inter-channel order parameter (obj+0xa870 +0x1404/+0x2814) = 4 / 8 / 16. A wrong order also picks the wrong code path -- -cc's planes 2-3 are order 16 and belong in RunBig, not RunSmall. Synthetic 84 -> 86/88, real corpus -cc 56 -> 59, total 476/480. Refuted with per-stage evidence: neither RunSmall's shift convention nor its factor-update polarity needed changing (the raw asm at 0x08095ca0 confirms sar and delta > 0 -> psubw); two hand-derivations said otherwise and the measurement won.
2026-09-01 -cc reaches 60/60 (aedeae9): the decode was already BYTE-EXACT and our own checksum gate declined it. The per-entry checksum is a type-5 chunk record and does NOT have to sit in the front metadata run -- 01 Track 01.m4a under -cc emits an empty type-5 placeholder up front and the real 4-byte one 971622 bytes in, after its first data chunk, which the front-of-archive tag scan cannot see. TryDecodeLegacyCm now captures a type-5/6/7 record while walking and falls back to it (single-entry archives only). Strictly adds verification where there was none. Real corpus 476 -> 477/480, seven of eight codecs perfect. Found by computing the checksum the entry should have -- a correct decode is its own oracle -- and searching the archive for those four bytes. Scope checked: for the same input -cd/-cf/-co/-cO all keep the record in the front run.
2026-09-01 SYNTHETIC SUITE 88/88 — 100%, zero bridge (85d046e). Ported FUN_0809bdc0, the second per-channel bit-count decoder CLASS (vtable 0x0813c860, selected only by -co), from raw disassembly — 207 instructions, no decompile existed. The architecture notes' "Fenwick tree" reading did not survive reading it: the cumulative search is a linear scan in groups of four with a back-off, and the model is an EXACT SLIDING WINDOW of 2040 symbols with freq[new] += 8 / freq[evicted] -= 8. 32 contexts x 0x900, which is why the persistent context sits at +0x12010; ctx = (15ctx + 256sym + 8) >> 4 selects a table via (ctx + 0x80) >> 9; carryless 14-bit range coder. The initial tables were derived from the live object (its builder FUN_080bd760 has no decompile either): a ramp centred on 2sel, the rest of the ring k % 64, freq[s] = 1 + 8ringcount[s] — which lands every total on exactly 0x4000. This closes the decr_param==2 audio defect entirely: -co, -cO and -cc all byte-exact. Real corpus 477 -> 479/480 with -co 58 -> 60. ONE failure left in the project.
2026-09-01 Model-memory instrumentation for the last failure (11f94a6, no behaviour change). NZO2_DUMP_MEM dumps mem_ at every DecodeBlock entry and NZO2_WATCH logs writes to one cell; the real object is arg0 of FUN_080a5d90 and a 48-byte window at +0x90 matches ours byte-for-byte. Measured: at block-2 entry our model memory differs from the binary's in 580 of 17 313 920 bytes, 474 of which our port never wrote (so they come from the cold state), and our cold state differs in 431 bytes at block-1 entry. Refuted: five EnsureHeadroom variants (the ring wrap is NOT the mechanism); overwriting the cold state with the binary's values BREAKS block 1, so ~207 cells sit at offsets that do not correspond to the real layout; and "fixing" the tier2 align table on the strength of the binary reading 0 at kTier2AlignOff is an ARTEFACT -- that offset is kMemSize, i.e. appended past the real model area, and matching it drops -cO to 7/11. The open question is which cells this port places at addresses the original does not use.
2026-09-01 Two of my own conclusions about the last failure retracted (54a0f05, no behaviour change). The "580 differing bytes of model memory at block-2 entry" was measured but misread: almost all of it is POINTER FIELDS baked into the captured cold blob (heap and static), which this port never dereferences -- the 63287/62804 pairs that looked like drifted probabilities are the high halves of two different heap addresses. And the 0xc0-stride family is a per-slot STATISTIC: the record base is 0x103c140 + (slot-4)*0xc0 and offset 0 is a u32 zeroed before the bit loop and incremented per bit, read only to increment. Filtered, the cold states differ in two unreferenced u32 counters plus the port-invented align table. So the state at block-2 entry effectively MATCHES and the divergence is introduced INSIDE block 2. Next step needs the real chunk-loop address: the real engine does not copy per chunk, so an output-buffer watchpoint checkpoints the end of the block instead. Adds NZO2_DUMP_CHUNK.
2026-09-01 Multi-file silent corruption closed (37a0ccd). The per-file metadata run is PARSED, not tag-sniffed: type 2 = u32 mtime + zigzag varint deltas, type 4 = u16 permission RUNS (V<0x1000 one file, else run of (V>>9)-6 with mode = V & 0x1ff, max 121), type 5/6/7 = per-file checksums. Files are grouped into BLOCKS, one table per block, so reading only the first lost every later file. With real checksums the per-entry verification runs again -- and because they also feed candidate selection, -co/-cO multi-file went from wrong bytes to CORRECT, not merely to a decline. New tests/multifile_v2.sh compares whole extracted trees (contents + mode + mtime) and listings across the metadata switches.
2026-09-01 Payload located by parsing instead of a windowed varint scan, and multi-block chains SPLICED -- the next block's metadata records sit BETWEEN two data records (7b73e5b). Multi-file 38/48 -> 47/48.
2026-09-01 Parallel (-pN) store and CM slices; attribute records applied per stream (be948cc). -cc had been feeding all four streams' chunks to ONE CM decoder, sharing state that never was shared. Offset trap found: a parallel container's type-10 offset is the slice's offset within its FILE, not the output -- both files of a two-file archive have a slice at offset 0 -- so all four tiling paths are now gated on a single-file archive; a multi-file one would have overlapped the files with no checksum able to catch it. Multi-file 64/64 extract, 56/56 list; baselines held at 88/88 and 479/480.
2026-09-01 lzpf dictionary capacity DERIVED, not searched (15005b4). It is bytefloat(p1+1) << 16 -- the same mantissa/exponent byte the -cc and optimum windows already used. The old five-guess search was checksum-gated, so -hn/-nm archives (no checksum to adjudicate) declined outright. Found by tabulating (total, p1, winning cap) across the corpus; every row matched, including the four where ceil-of-total was wrong.
2026-09-01 lzhd column selector: codec bit and RLE size-field are INDEPENDENT (9809520). A raw column WITH RLE (b0 == 0x02) consumed the wrong byte count, so -cdP/-cDP decoded a multi-block archive wrong (self-verify caught it). -cd/-cD never emit that combination. The tell was arithmetic: a correct chunk has litsum + summlen == out_size exactly. Same commit fixed an OOB read -- the chunk cursor could pass the stream end, after which (size_t)(end - cur) underflowed to ~2^64.
2026-09-01 BWT stored bucket (0b8a669). in == 0 in the bucket table means the bucket is stored VERBATIM. Reading it as a literal zero made -co (the DEFAULT compressor) and -cO decline on any mixed-entropy file from ~1 MB -- the most reachable gap the project had. The community reference has the same hole; tracing the raw table reads settled it without GDB, because the entries with in == 0 consumed their varint bytes like any other.
2026-09-01 Parallel container holding several files (a441f48). A stream is a WORKER, not a file, and the type-10 offset is relative to the FILE, not the output -- both files of a two-file archive have a slice at offset 0. One generic assembler plus four per-codec closures; the file order is the order of the stream whose tables name the most distinct files. nz a -r <folder> produces this shape above ~8 MB.
2026-09-01 Decode gaps now: one. -cO on a single .dp document. Suites: multi-file 108/108 + 36/36 + 72/72, native_only_v2 96/96, real corpus 479/480. Every gap this day was found by widening a DIMENSION of the test matrix (twelve selectors instead of eight, MB-scale fixtures, the option switches, parallel containers, edge shapes) -- none by reading code.
2026-09-02 Fuzzed against corrupt input before the v0.3.0-pre binaries shipped (7e05735). 761 cases under ASAN+UBSan found an out-of-bounds WRITE (the tt16 transform ignored its output capacity), a 214-second DoS ending in SIGSEGV on a 191-byte mutated archive, an OOB read on a corrupt Huffman table, and two signed-overflow sites. The reusable invariant: bound decode work against the DECLARED OUTPUT, per entry, not per call. 761/761 clean after.
2026-09-02 -cc declined by default for every user (bc09920). The CM path ran its bridge cross-check whenever the bridge was ENABLED BY CONFIGURATION, and treated "the check could not run" (no legacy binary reachable -- the normal case for a user) as "unverified". Every suite sets NZ_NO_BRIDGE=1, so none of them could see it. multifile_v2.sh gained a phase that runs the binary under env -i with no legacy nz in reach. Lesson: a harness that configures away the default path is not testing the product.
2026-09-02 -cO fails on ~7.5% of real files, not 1 in 480 (667e9be). A 200-file sweep found 15 failures; the 60-file corpus quoted for months held exactly one, so a common bug read as a curiosity. The original round-trips all 15 losslessly -- and the corpus sweep could never have noticed otherwise, because it compares us against the original. A 60-file corpus cannot characterise a 7.5% failure rate.
2026-09-02 Text-transform bit 0x40 is NOT ported -- found because one file failed -cd -cD -co -cO -cc while passing -cn -cf -cF, the signature of a shared path. It is TransformText_6, whose body in the community reference is assert(0). The claim that all emitted text-transform bits were native was wrong. Tractable without GDB: the decoder already has the pre-transform bytes and the golden file is the post-transform output.
2026-09-02 optimum2 literal-model bug: five hypotheses refuted, the decisive one being that disabling the ring wrap entirely still fails, which kills the whole ring/eviction family. Also recorded a trap: input 196608 passes and 196609 fails at exactly the ring capacity, which looks like proof of first-eviction and is not -- one extra input byte changes what the encoder emits.
2026-09-02 Text-transform bit 0x40 decoded from data, no GDB (68c1635). It is a PGN/chess transform: 0xBE..0xFD = a board square (i=b-0xBE, file 'a'+i%8, rank '1'+i/8), 0xFE = an auto-incrementing move number, 0xFF <arg> = repeat a cached line. With the first two alone the repro file's first game reconstructs byte-exact. The technique generalises: NZOPT_DUMP_TTIN gives the chain's input, and when the only unported stage is followed by invertible ones the golden file IS that stage's output. Left open: the cache slot function (7 sets = 7 PGN tags, 2 ways; zero of 68 lines ambiguous; every simple hash over the tag refuted).
2026-09-02 Text-transform bit 0x40 PORTED (9711d45) -- the seventh bit, and absent from the community reference too (assert(0) there). Decoded from (input, output) pairs, then finished from FUN_080a3000, which supplied the two things the data could not: the rank mask and the cache slot function set = (((c0+15)&15)*4) + (c1&3). Located by grepping the disassembly for sub $0xbe, a constant only this transform uses. The last gap was a counter resync by lookahead: a literal number followed by . sets the move counter, so [Round "2.3.1"] had been decoding as 2.1.1. Repro went from failing 5 codecs to byte-exact on all 8; added as a corpus fixture (61 files, 487/488).
2026-09-02 The -cO "one literal bit wrong" class CLOSED, and it was two ring-lifetime bugs (c1318b4). (1) The wrap's LZP-table sweep was 64 bytes low: the real call is FUN_080b9150(param_1 + 0x40) and that function memsets arg + 0x1042bc0, so the + 0x40 belongs to the CALLER and the range starts at the table base obj + 0x1042c00. Dropping it clobbered 64 bytes of live dispatch-APM state and left the LAST 16 entries (hashes 0xfff0..0xffff) uncleared for the archive's life. The LZP fold reaches only ctx7's seed — one of eight mixer inputs — so a stale hit moved one probability by ~1 % (pFinal 1082 where 1038 flips the bit) and changed a decoded bit only at a decision boundary: hence one wrong byte with tens of thousands of exact bytes either side, on a minority of files. (2) FeedWindow collapsed FUN_080b9180's four cases into one "reserve then write"; its SPLIT case fills the ring to its END before wrapping, leaving the cursor at (pos+len) mod capacity instead of len — and that wrap is what clears the LZP table. The compact -co engine has the same feed (0x080bcc60) and the same bug. Found by breaking on the mixer's own pFinal store with a GDB ignore count taken from a new windowed per-bit trace in the port; at the failing bit seven of eight inputs and all eight weights matched exactly, which named ctx7 outright. Also fixed: the container's per-worker record runs are emitted in thread-scheduling order, not stream order — the header walk took the canonical filename table from stream 0 only, so an occasional 3, 0, 2, 1 layout put table_end inside the payload and rejected the archive; it is not -pN-only, since NanoZip is multi-threaded by default. Fresh 155-file corpus: 1221/1240, with only two causes left (an unported image model for uncompressed BMPs, and a -cd/-cD prefilter-chunk content bug).
2026-09-02 -cd/-cD: two of the cluster's three causes closed (34edc77). (1) The prefilter state reset that every LZ chunk must perform (FUN_080b1950, GDB-confirmed on pure-literal chunks too, leaving the state byte-identical to cold) sat inside DecodeChunk's token branch, so a pure-literal chunk between two prefilter runs never reset and the second run resumed warm; residual arrays were identical to the original's, only the LPC cascade differed — a warm predictor on a cold stream rings like an unstable filter, hence a large smooth oscillation instead of noise. Four reset variants giving byte-identical output was the clue that the code was unreachable rather than wrong. (2) The LZ ring is bytefloat(p1+1)·0x10000 — third appearance of that byte after the -cc window and the lzpf dictionary; round(total/0x10000) fit every sample it was measured on and failed the first file where they disagree (p1=33 → 36 units vs 35), whose last chunk began exactly at the ring end and matched past it. Wide corpus 1221 → 1231/1240. Still open: two -cD files whose LAST chunk decodes wrong from its first bytes — the original's FUN_080982e0 is a second pass over a compact buffer, equivalent to this port's interleaved scheme on 153/155 files; buffers captured. Two traps hit again and worth repeating: zsh set -- $var does not word-split, and rebuilding the binary while a corpus sweep runs invents failures.
2026-09-02 v0.5.0-pre published (tag on 11519c7): Linux + Windows, 64- and 32-bit, built from a fresh clone and verified 83/83 on 35 original-made archives (12 selectors × single/multi, -p4, -hn, and the day's repro inputs) with no environment variables, the Windows pair on the real win10 VM. Carries all five decode fixes of the day; the notes state the two remaining gaps (BMP image model, -cD short last chunk) plainly.
2026-09-02 Parallel -cd/-cD rings sized from each stream's own p1 (bytefloat(p1+1)), completing 34edc77: the per-worker type-11 record is now parsed, and a 4×2.3 MB -p4 container whose slices need 36 units (round gave 35) goes from declined to byte-exact.
2026-09-02 The last -cd/-cD cause closed: a match crossing the ring END is copied linearly into zeroed slack (-cd and -cD reconstructs both). With tokens, literals, ratebits and the whole MTF table byte-identical to the original's at every call, a u32 watchpoint on the ring's first word showed the rep movsb for match pos=1 off=3 len=4 reading ring[65534..65537] — past base+cap, where the first wrap's memset(base+pos, 0, cap-pos+0x100) left zeros — while the port's modular copy read ring[0], ring[1]. Wide corpus 1233/1240; the seven left are one BMP (unported image model). Lesson kept: when every input matches and the output differs, watch the output being WRITTEN; and (x >> 0x1f & cap) + x in a decompile wraps the start once — it is not a modular copy.
2026-09-02 BMP image model scoped (not yet ported): it is decr_param = 3, decoded by every codec through FUN_080a9ca0FUN_080a90c0 (hash-bank-A CM), with a 4-byte prologue carrying width − 1, the raw 54-byte BMP header, and the pixels de-interleaved into three colour planes coded 64 KB at a time. The reference's "CM without reset" reading of decr_param 3 is wrong. One port for all seven remaining failures; golden vectors come free from the file itself.
2026-09-02 Image model PORTED — wide corpus 1240/1240, no decode gap left on it. FUN_080a90c0 is the audio decoder's 2-D sibling (LMS planes + residual coder + a 4-stage cascade over the four rows above + an 8-mode pixel predictor + two history rings), not a CM; 51/31/11 is the raw prefix of a pixel split across chunks. Ported as NzImageModel (nz_audio.cpp), wired into the CM family's decr_param 3, the -cd/-cD 0xf sub-chunk and the -cf/-cF bit-3 prefilter-slot block; per-codec flag/order profiles read with GDB. Byte-exact on the first build in all 7 codecs; 37 real BMPs, 8/16/24/32-bit + PGM/PPM/TGA/TIFF synthetics and mixed BMP+WAV+text archives pass. Method: Ghidra headless for all callees at once, one GDB run per codec capturing (in, out, object state) — the object state doubled as the profile table and a per-chunk oracle. Two unrelated -co/-cO declines found on the way (LZ block after two BWT blocks; a 1.4 MB BMP on the BWT path), repros saved.
2026-09-02 v0.6.0-pre published (tag on ea5aebd): four static binaries (Linux/Windows x64+x86) verified 91/91 against the original on 43 archives — the v0.5.0-pre package plus the 60 KB and 2.88 MB BMPs in all seven image-capable codecs — with no environment, the Windows pair on the real win10 VM. Notes list what is pending: the -co/-cO LZ-after-two-BWT-blocks decline, the BWT large-bucket decline, unexercised image modes 0/1/3–7, 0xd/0xe sub-chunks, parallel containers >8 MB, encode.
2026-09-02 -co/-cO param15 source fixed (65a00a5): absolute-offset matches index the accumulated PRE-post-filter stream (the window), not the post-param1 output; closed both declines the BMP sweep found (LZ block after two BWT blocks; a 1.4 MB BMP on the BWT path) — 37 BMPs × 7 codecs 259/259, all suites green (1240/1240, 488/488, 96/96, 144/144+72/72, package 95/95 with four param15 archives added). The BWT large-bucket item quoted as pending in the v0.6.0-pre notes was already closed (0b8a669); its repros decode byte-exact with the v0.6.0-pre binary — notes corrected.
2026-09-02 v0.7.0-pre published (tag on 65a00a5): four static binaries verified 95/95 against the original on 47 archives (the v0.6.0-pre package + the four param15 repros), no environment, Windows pair on the real win10 VM. First release with no known decode failure on any corpus, sweep or repro. Notes ask for real-world archives; encode waits for that feedback (encode.su thread).
2026-09-02 Console made 1:1 with the original (while encode waits for encode.su feedback): a 36-case matrix comparing stdout/stderr/files of nz and nz_recon byte for byte drove it. Ported from the binary: exit status always 0 (NZ_STRICT_EXIT=1 for real codes), the .nz/.exe archive-name rule, self-extracting .exe support (the opener's "resync" is a PE header parser, FUN_080b0e50), the exact [N MB] figure (the codec's memory-usage method, vtable slot 0 — FUN_080aafb0 for the CM family — transcribed to the byte for all 12 selectors and both parallel forms), the progress model (cumulative MB, name re-printed only on file change, 40-column truncation), the error messages and codes (Archive corrupted. Error decoding (code 100/25600), Checksum mismatch [stored computed]: path then continue, the non-archive probe with its "incompatible version" byte), -v (only an IO-buffers suffix), -sp on extract, the overwrite prompt, -v listing widths. Left inherently different: argv[0] in usage, per-worker line order on parallel containers, encode. Harness at ~/.cache/nzre_tools/cli_parity/.
2026-09-02 v0.8.0-pre published (tag on 1c81ed5): the console-parity release — self-extracting .exe support, exact [N MB] figures, the original's messages, codes, progress and exit status (NZ_STRICT_EXIT=1 for real codes). Four static binaries verified 95/95 against the original on 47 archives, no environment, the Windows pair on the real win10 VM (banner Win64/Win32). Codecs unchanged from v0.7.0-pre.
2026-09-02 Every command and switch audited against the original with a second, 78-case switch matrix (stdout + tree of mode/mtime/size compared). Eight real differences found and fixed: switches are recognised anywhere after the command (x arc -y overwrites); values must be attached (-o out makes "out" the archive, -x alone is an unknown argument) and invalid ones (-mfoo, -tx, -sz, lone -) stop the run; -t<n> caps the reported thread count; -br/-bw print IO-buffer sizes in the header; l ignores file arguments; -x<glob> exclusions now apply on extract (they were parsed and dropped); -swapinout and -forceout behave as in the original. The only remaining differences are timing (the IO-out line, progress refreshes).
2026-09-02 Legacy bridge removed — the program is native-only by construction. After an audit (strace over the 95 verification files with an original reachable and the bridge enabled: zero foreign execve, zero probes of an original binary; originals chmod 000: 95/95), every path that could find or run an original nz was deleted: the CWD/$PATH search (FindLegacyBackend*), the extract bridge, the gdb trace bridge, the unknown-switch forwarding, the compression bridge and the -cc cross-check against the original. A stream no native decoder accepts is now reported as Archive corrupted. Error decoding (code 100). NZ_NO_BRIDGE is ignored (the suites still export it).
2026-09-02 v0.9.0-pre published (tag on adbbb18): native-only by construction (bridge deleted) and every switch audited against the original. Four static binaries verified 95/95 against the original on 47 archives, no environment, Windows pair on the real win10 VM. Codecs unchanged since v0.7.0-pre.
2026-09-02 First user bug report fixed: archives above ~1 GB. A 2.29 GB -cf parallel archive "hung" on Windows; on Linux it was declined as corrupt. The container record walker had a 1024-record cap (16 streams × ~144 records overflowed it, half the streams were folded into one undecodable slice); four parallel walkers had 8192-record caps (same bug at ~8 GB). All caps removed — byte-exact in 49 s (the original: 4.7 s multi-threaded; ours decodes the streams sequentially — the next performance item). Footer timer now spans the whole command; progress redraw is time-based like the original's.
2026-09-03 The Windows hang itself (second cause of the first user report): with the record caps gone, the Windows build wrote the complete 2.29 GB file and then spun at 100% CPU forever — one ofstream::write above 2 GB; msvcrt _write() returns an int, so the completed write reported a negative count and the stream layer retried endlessly. Extracted files are now written in 256 MB pieces. Verified on the win10 VM: 83 s, footer printed, SHA-256 identical to the source tar.
2026-09-03 v0.9.1-pre published (tag on 64c005a): the two fixes behind the first user report (record-count caps; Windows >2 GB write hang) plus a clean "Out of memory!" on the 32-bit builds. Four binaries 95/95 on the package; the 64-bit pair also on the 2.29 GB -cf archive (Linux 49 s, Windows VM 75-83 s, SHA-256 identical). Next performance item: decode parallel-container streams concurrently (original: 4.7 s on 16 cores).
2026-09-03 Parallel containers decode their worker streams concurrently (3b8e9b0): the five sequential per-stream loops now run through ParallelForEach() (threads = CPUs, -t<n>/NZ_THREADS), after DisjointCover() proves the output slices tile without overlap. 8-stream 400 MB archives: 3-5× (lzpf 10→3.2 s, optimum1 42→8.3 s, cm 102→19 s); the 2.29 GB -cf report: 49→23 s (original 4.7 s; the rest is serial copies/checksums). ThreadSanitizer clean. Found and fixed on the way: -cd/-cD parallel streams declined on a 0xc prefilter sub-chunk (no per-stream prefilter state), and the archive was read a byte at a time.
2026-09-03 v0.9.2-pre published (tag on 98634f6): parallel-stream decode, the -cd/-cD parallel prefilter fix and the sized archive read. Four binaries 95/95 on the package; the 64-bit pair also on the 2.29 GB -cf archive (Linux 26 s, 4-CPU Windows VM 40 s, SHA-256 identical). Codecs unchanged since v0.7.0-pre.
2026-09-03 Serial copies removed around the parallel decode: stage timers showed 16 of the 23 s on the 2.29 GB archive were PrintDecodeHeader copying the whole decoded context once per Compressor #k line (16 × 2.3 GB); plus payload copies in the re-entrant contexts and a redundant second checksum pass. Now 6.9 s (original 4.7 s): read ~1 s, record walk 0.9 s, parallel decode 3.2 s, write 1.5 s.
2026-09-03 Damaged archives, option (c): extraction now writes every entry whose stored checksum verifies, prints Checksum mismatch [stored computed]: path for the rest and skips them (the original writes the wrong bytes), and returns exit status 2 for damaged or undecodable content (the original always returns 0; usage errors stay 0 for parity). Serial copies of the decoded payload removed (2.29 GB archive 23.5 s → 6.9 s). New docs/ORIGINAL_QUIRKS.md: 24 measured quirks/defects of the 0.09 alpha and what nanozip-re reproduces.
2026-09-03 v0.9.3-pre published (tag on abb9ffe): option (c) for damaged archives + exit status 2, serial copies removed (2.29 GB: 23.5 → 6.9 s), docs/ORIGINAL_QUIRKS.md. Four static binaries verified 95/95 each (Linux locally, Windows on a real Win10 VM).
2026-09-03 Faithful default on damaged archives (towards v0.9.4-pre): measured the original's write model (flush per codec block for -co/-cO/-cc, per 1 MB stream for -cd/-cD, per member for -cf/-cF; files of completed blocks written, the file the failing block starts with created empty, checksum-failed files written with the mismatch line, exit 0) and reproduced it: the single-container decoders keep their completed blocks on failure, the BWT inverse rejects a permutation whose cycle does not close, a clean-but-short -cd decode says Unexpected end of file.; 40/48 damaged archives across eight codecs leave the same tree as the original. Option (c) moved behind NZ_SAFE=1. Policy written into docs/ORIGINAL_QUIRKS.md: replicate first, let the community decide what to fix.
2026-09-03 v0.9.4-pre published (tag on a611908): faithful default on damaged archives, NZ_SAFE=1 for the v0.9.3 behaviour, exit status 0 again; docs/ORIGINAL_QUIRKS.md policy + 27 items. Four static binaries 95/95 each (Windows on a real Win10 VM), damaged-archive acceptance 40/48.
2026-09-03 3000-file stratified sweep + console round 3 (87018b5): the sweep (tests/corpus_select.sh, sweep_run.sh, resumable shards, NZ_TRACE_CONSTRUCTS) found two decode bugs, both fixed — -cd/-cD chunks with exe filter + block-RLE (flags 0x7) skipped the exe un-transform; -cc stored blocks skipped their post-filters — and one open -cD lzhds case (uitrack.pod). The per-block "staged" bytes are understood (Fletcher32 of each stage % 255, LIFO) and verified like the original: damaged-archive parity 42/48. Console: 0700 directories, the mtime model (stored − writer offset + reader's current offset, 32-bit wrap), Cannot write:, -forcemem/-continue/-pause, Unknown command: forms, -fo column, listing truncation, prompt semantics via a pty harness. ORIGINAL_QUIRKS.md at 38 items.
2026-09-03 -cD reset rule (a5c33c3): the lzhds literal model is reset after any pure-literal chunk longer than 255 bytes (driver decompile: if (0xff < size)), not only after a full 32 KB window — closes the uitrack.pod sweep failure; -cD/-cd real corpus 122/122.
2026-09-03 Sweep closed: 24 272/24 272 (d396afc, 17b8bab): three more bugs from the 3000-file sweep — the chess/PGN transform copies the two bytes it hashes after a literal [ verbatim ([[wiki]] markup cached a spurious line), its resync digit class is 19, and the -cd/-cD exe filter restarts its position at an embedded MZ/ELF executable (FUN_080c03f0). 61-file corpus 488/488, release check 95/95.
2026-09-03 Pre-release fuzz round (3fa76c8, a4c1c33): 1228 corrupt/non-archive inputs under ASan+UBSan (~/.cache/nzre_tools/fuzz/fuzz.sh) found a segfault (range coder rewinding before its buffer), two denial-of-service cases (a two-minute store-prefix scan past a truncated file; -cd chunks whose corrupt tokens asked for millions of literals), a heap write in the dictionary text transform, three out-of-bounds reads and several UB sites — all fixed, all suites unchanged.
2026-09-03 v0.9.5-pre published (tag on 73c3160): 3037-file sweep 24 272/24 272 with six decode fixes, per-stage check bytes reproduced (damaged archives 42/48), console round 3, fuzz 1228/1228 clean after eight robustness fixes, README slimmed. Four static binaries 95/95 each (Windows on the real Win10 VM).

Legacy backlog (historical, pre-progress-log)

These predate the progress log above and are kept for archival value; all are superseded by later entries.

  • Task #13: lzpf prefilter+arith mono path complete (FUN_080a5330 + FUN_08095d90 LPC filter). Stereo variant (FUN_0809bbf0) deferred.
  • Task #13b: lzpf prefilter+arith stereo residual decoder (FUN_0809bbf0) ported. Tables extracted from binary via GDB (DAT_081b3a00/39c0/39c1/42f0/4380); algorithm: VLC magnitude from byte + side-bit sign. is_stereo_variant flag path wired in DecodePFBlock. Verified 2026-06-03 with synthetic correlated stereo WAV (stereo_lms.wav in tests/fixtures/lzpf/, 64 KB, ch2 = ch1 + Gaussian noise): both -cf and -cF archives decode byte-exact natively with ApplyLmsInterChannel (port of FUN_08096e20) running on the residual stream.
  • Task #24: 1-byte LZ77 divergence in variant A (semirandom block 18, side_count=8416) — believed fixed by hash-table init=3 fix (2026-05-05). Verified 2026-06-03: exhaustive random/high-entropy corpus (200+ seeds × 5 sizes 4KB–512KB, plus mixed fixtures) all decode byte-exact natively. LZ77+arith path upgraded to 100% native in coverage estimate.
  • Task #14: lzhd native decoder complete — FUN_080b5240 ported as DecLZ (PAQ context mixer + 12-bit arith, 680 LOC); byte-exact on 50 KB text fixture. Parallel variant (FUN_080b50b0) deferred.
  • Task #25: Parallel archive header parser (-pN) fixed in TryParseLegacyCnArchive. NZ chunk format fully decoded: (size<<4)|type varint with nibble-15 stream-ID extension. Scanner accumulates uncompressed sizes across all per-stream type-1 chunks; size_accum overrides partial main-stream sizes after entry build. Byte-exact extraction verified on -p10 archive (344207 bytes).
  • Task #26: Archive writer (RunAddNativeLegacyStream) fixed to emit full codec chunk payload. spec.method = (csize<<4)|11 declares csize payload bytes; writer was emitting only p0+p1 (2 bytes), causing the chunk scanner to consume the first byte of table_span as the 3rd codec byte, leaving a zero-size type-1 chunk. Fix: pad with zeros to reach csize bytes. Encode round-trip x_ok: 5/8 → 8/8.

Clone this wiki locally