Reverse Engineering Notes

Reverse engineering approach

Tools

Tool	Purpose
Ghidra 11 headless	Decompile stripped ELF; batch-export C pseudo-code per function
GDB 12 (linux32 ELF, x86 host)	Dynamic tracing: register dumps at breakpoints, memory captures, bytecode extraction
objdump / readelf	Quick disassembly, section layout
Custom C++ unit tests	Validate each ported function against GDB-captured ground-truth vectors
Python scripts	Parse binary stream dumps, compute expected outputs for test vectors

Workflow

1. Identify the target function address in the 32-bit ELF (linux32/nz).
2. Export Ghidra decompile.
3. Set GDB breakpoints, run the original binary on a known fixture, capture register/memory state.
4. Port the function to C++ in src/lzpf_arith.cpp or sfx_archive.cpp.
5. Write a unit test in tests/test_lzpf_arith.cpp with the GDB-captured vector.
6. Run the full coverage matrix to confirm no regressions.

The 32-bit binary is the primary reference because Ghidra's 32-bit decompile is more readable (fewer pointer-width artefacts) and GDB gives exact struct offsets.

A GDB-ground-truth-vs-port-comparison technique recurs throughout this project's history (see Changelog for many concrete instances): when a ported function looks structurally correct but produces wrong output on some real inputs, capture the real binary's actual internal state at a specific point via GDB, then replay/compare against the port in isolation (a small standalone harness) rather than guessing from the decompile alone. This has found subtle one-line bugs (wrong shift direction, wrong loop bound, wrong table) that pure code review missed.

Reference source, when available, is also compared line-for-line: encode_su/nzdec_v0.7z is a community reference decoder (Shelwien et al.) covering some but not all codecs — where it exists, diffing the port against it directly is cheaper and more reliable than a GDB session (this found the AddBytesFilter/BitReader word-boundary bug, for example). Where no reference exists for a given codec path (most of -co/-cO's real LZ/CM engines, -cd's real coroutine token-LZ), GDB against the real linux32/nz binary is the only source of ground truth — the community reference decoder's own addresses/architecture assumptions do not correspond to the real linux32 binary's internal structure and have led this project astray more than once (see the -co/-cO "DecLZ is not called at all" finding in the changelog).

Real-world corpus testing

tests/native_only_v2.sh's small synthetic corpus (random bytes, lorem ipsum, zeros, one source file, one WAV) is useful as a regression tripwire but is too narrow to catch bugs that only manifest under real-world content variation (bit depth, channel count, text shape, side-stream byte alignment, ...). tests/real_corpus_sweep.sh sweeps an arbitrary directory of real files with the same NZ_NO_BRIDGE=1 methodology, grouping failures by the native binary's own decline reason so gaps are triageable by root cause. A single afternoon of sweeping a real file-format sample collection against this project found three genuine, previously-hidden bugs in code that had been believed "done" (see the 2026-07-29/2026-07-30 changelog entries) — this is now considered one of the highest-leverage activities available on this project, and worth returning to periodically as more codec paths get closed.

Key findings

• LZPF block format: three-mode varint header dispatching literal / LZ77 bytecode / prefilter+arith. LZ77 opcodes 0xf5–0xf8. Arith-coded side stream when uVar9 & 1 == 1.
• lzpf variant A vs B: variant A = 13-bit hash, opcode threshold < 0xf6; variant B = 24-bit hash + 8 KiB byte-context buffer, opcode threshold < 0xf5, adds opcode 0xf5.
• Sliding-window dict: size = (p1+1) × 64 KiB (no min/max). 4-byte left-pad; cursor initialises to 4; wraps to 0 when capacity − cursor < 32768 (mirrors FUN_080b6bb0).
• Hash table init = 3: variant-A 8192-entry table initialises to 3 in the legacy binary, not 0. Using 0 causes wrong local_50 values for f6/f8 opcodes in early blocks.
• last_lz_dest reset: resets to −1 per block (stack local in FUN_08097570), not persisted across blocks.
• Multi-stream chain: large -cf archives embed multiple [tag][data] segments; dict + hash table state persists across all segments.
• Arith decoder (FUN_080a4ea0): two-pass Huffman code-length reader (FUN_080a41d0, 480+ lines ported), canonical Huffman build, MSB-first 32-bit-cache bit reader. Two non-obvious bugs fixed: RangeCoderFinalize must call ReadBits(remaining_bits) after rewinding cursor; BuildHuffman length_table must escape to 9 once length > 8.
• CM text-transform tt_flags=0x10 (word-list) — blocker is a bug in the CM decoder, not the transform (confirmed 2026-06-01). Ground truth via GDB on linux32/nz: break at the transform entry 0x080a3340, dump its input buffer (= the CM decoder's real output). That stream is 99.95% printable text, whereas the ported NzCmDecode diverges at byte 26 (bit 5) and produces garbage. The transform itself is secondary (near size-preserving substitution of common words with 0/1/2 tokens). A standalone harness reproduces the divergence deterministically; CM params were ruled out by sweep. The wrong value is one of the three mixing stages in CM_Input_Bit (linear mixer / modelg APM / cmc). The decode-side transform tree (fcn.080a3340 → a28a0/a1b60, range coder + word table) is mapped and ready to port once the CM decoder is fixed.
• CM -cc decode function is unlocated in this build (2026-06-02) — the documented CM engine addresses (0x0809e600, 0x080a5c70) and the dispatcher (0x080aa850) do not execute during -cc decompression: every breakpoint set on them is unhit, and 0x080aa850 is hit once with method byte = 0 then returns early (no-op). So the real per--cc decode routine is elsewhere in the stripped static binary and still has to be located before the byte-26 mixing-stage bug can be pinned to a specific instruction. Bits 0–208 decode identical to legacy (model state in sync), so this is a deterministic prediction/weight-update formula bug, not flag drift.
• lzpf stereo-split inter-channel predictor FUN_08096e20 (2026-06-02 / 2026-06-03) — root cause of the stereo -cf/-cF audio gap. Mono is fully native byte-exact. Stereo was decoding (header OK, decode_ok=1) but produced wrong samples from offset 44 because the native path skipped inter-channel decorrelation. The function is a 2-stage cascaded sign-sign LMS 4-tap adaptive predictor (MMX path + scalar fallback at 0x8097199); primitives: predict 0x80beaa0, update 0x80beae0, base-predict 0x80be8e0, base-update 0x80be820. Driver: ch1 predicts from 0; ch2 predicts using ch1's reconstructed sample (inter-channel). Ported 2026-06-03 as scalar equivalent ApplyLmsInterChannel in lzpf_arith.cpp; state persists across blocks via caller-managed LmsObject (0x2070 bytes per object, 2 objects per block). Verified byte-exact on synthetic correlated stereo WAV. Coverage estimate for prefilter upgraded to 100%.