fix(gf16): reconcile production codec to standard (1+M/512), reclaim wasted mantissa bit - #94
Merged
Merged
Conversation
…mantissa bit golden_float16.GF16 (the production gf16_* C-ABI codec used by every language binding + FPGA path) normalized to [0.5, 1.0) and stored (mant_f - 0.5)*512, so the top half of the 9-bit mantissa field (256..511) was never produced — ~8 effective mantissa bits instead of 9. It also diverged from every other GF16 in the repo (1.0 -> 0x4000 vs the standard 0x3E00): specs/gf16.tri from_f32_steps, docs/spec-gf16.md, gf_binary.GF16, and formats_root.f32ToGf16 all use the standard implicit-leading-1 form. specs/ops.tri was the only spec agreeing with the wasteful form, contradicting specs/gf16.tri. Owner decision: fix to the standard (1 + M/512)·2^(E-31) normalization. - golden_float16.GF16.fromF32/toF32 now delegate to gf_binary.GF16 — one codec, bit-identical by construction (guards against future re-drift). - Add gf16 exact-bit vectors to testdata/gf_conformance.csv (was excluded) and to gf_binary's encoding-regression guard; add reconciliation + exact-bit tests in golden_float16.zig. - Fix specs/ops.tri (0.5+M/512 -> 1+M/512) to match specs/gf16.tri; fix the bogus IEEE-f16 copy-paste test_vectors in gf16.tri; fix the value formula in src/c/gf16.h. - Reseal specs/TRI-HASHES.md for gf16.tri and ops.tri (shasum). WIRE FORMAT CHANGE: gf16_* now emits standard bits (1.0: 0x4000 -> 0x3E00). No RTL exists in-repo and FPGA docs already describe [1:6:9] b31, so this aligns the codec with the documented hardware convention. Serialized gf16 data matching the old output is incompatible — warrants a format-version bump on release. Verified: zig build + zig build test pass; the built libgoldenfloat.dylib emits every gf16 CSV row exactly via ctypes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…6ee3 # Conflicts: # testdata/gf_conformance.csv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The exported production GF16 codec (
golden_float16.GF16, used by everygf16_*C-ABI binding) wasted a mantissa bit and diverged from every other GF16 encoding in the repo. This reconciles it to the standard(1 + M/512)·2^(E−31)form used by the spec-of-record, docs, and thegf_binaryladder — so there is exactly one GF16 codec.The bug (confirmed with exact-bit tests)
golden_float16.GF16.fromF32normalized to[0.5, 1.0)and stored(mant_f − 0.5)·512. Across a ~4,000-value sweep the 9-bit mantissa field never exceeded 255 — the top half (256–511) was dead → 8 effective mantissa bits, not 9.It also produced different raw bits than everything else for the same value:
golden_float16gf_binary, spec, docs)0x40000x3E000x40800x3F000x42920x4124The standard form was already used by
specs/gf16.tri(from_f32_steps),docs/spec-gf16.md,gf_binary.GF16, andformats_root.f32ToGf16. Onlygolden_float16andspecs/ops.triused the wasteful form — an internal spec contradiction. Thegf16.tritest_vectorswere separately bogus (IEEE-f16 hex, wrong under either convention).Decision
Owner chose to fix to the standard
(1 + M/512)normalization (vs. keeping the old wire format and documenting two encodings).Changes
src/formats/golden_float16.zig—GF16.fromF32/toF32delegate togf_binary.GF16; one codec, bit-identical by construction. Added an exact-bit test + agolden_float16 ≡ gf_binaryreconciliation test.testdata/gf_conformance.csv— added gf16 rows (previously excluded); the generic cross-language reader now pins them for Python/Rust/C++.src/formats/gf_binary.zig— added gf16 to the exact-bit encoding-regression guard (it skipped gf16 before because the codec diverged).specs/ops.tri—0.5+M/512→1+M/512(and normalize range) to matchspecs/gf16.tri.specs/gf16.tri— fixed the IEEE-f16 copy-pastetest_vectorsto correct[1:6:9]bits.src/c/gf16.h— fixed the documented value formula.specs/TRI-HASHES.md— resealed gf16.tri / ops.tri hashes (viashasum).Reviewer notes
gf16_*now emits standard bits (1.0:0x4000 → 0x3E00). No FPGA RTL exists in-repo and the FPGA docs already describe[1:6:9] b31, so this aligns the codec with the documented hardware convention rather than departing from it — but any serialized gf16 data matching the old output is incompatible and warrants a format-version bump on release.ℹ️ Follow-up (not in this PR):
tools/gen/check_tri_hashes.zig— the tool CLAUDE.md mandates for resealing — does not compile (JS-style destructuring + stale Zig APIs) and targets the wrong file. Hashes were resealed by hand viashasum(verified the table is plainshasum -a 256). A separate task tracks rewriting it.Verification
zig buildandzig build testpass (exit 0).libgoldenfloat.dylibemits every gf16 conformance row exactly (checked via ctypes).zig fmtclean; TRI hashes recomputed and matched.🤖 Generated with Claude Code