The int_relative domain, terminal failure, min <= max on int128, and the shared corpus - #3
Merged
Merged
Conversation
…the shared corpus STANDARD.md and conformance/ are now vendored here, verbatim from mas-bandwidth/serialize, with CI jobs that fail when either drifts. The suite runs every vector in the corpus through ReadStream: an accepted vector must decode to the stated value and consume the stated bits, a refused vector must be refused with its destination unwritten. int_relative reconstructs current in a long, in every tier, and refuses the read unless the result lies in the domain (0 to 2^31 - 1) and strictly exceeds previous. The absolute tier's 32 raw bits are read unsigned, so a top-bit-set value is refused rather than accepted as a negative sequence number, and no tier writes its destination before the checks pass. The write and measure sides assert previous in the domain. ReadStream carries a failure latch. The first refusal sets it and every later read returns false, consuming no bits and writing no destination — zero-bit reads included, which the past-end check alone cannot catch. Only reset() clears it, and isFailed() reports it. int128 takes min <= max on all three streams, as every other ranged operation does. A degenerate range costs zero bits: the writer emits nothing, the reader consumes nothing and takes the value from min, and the zero-bit path no longer reaches the 1-to-32-bit group primitive. make test-release runs the whole suite with assertions off, beside the checked run, proving every read-side refusal is a check rather than an assert. Both are CI gates. Version 1.1.0. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings serialize.java into conformance with the amended wire standard —
STANDARD.md and
conformance/atmas-bandwidth/serialize@7e42c0f,
the commit that merged serialize#111. No wire change: every accepted vector and
every golden pin still round-trips byte for byte.
Tagged for release as v1.1.0.
The corpus test, red then green
ConformanceTestsruns every vector in the vendoredconformance/directorythrough
ReadStream. It is the negative control for the int_relative andint128 fixes: run against the reader as it stood before this branch, on the
same vendored corpus, it is red.
Before:
Ten failures, and each is a distinct defect. The six
refuse-past-domainvectors were accepted: the reader reconstructed
previous + differencein anintand let it wrap past the domain top. The threeabsolute-refuse-*vectors were refused for the wrong reason — the raw 32 bits were read signed,
so
0x80000000arrived as a negative number that failed the ordering checkafter it had already been written to the caller's cell. The int128 vector threw
under
-eaonassert min.compareTo( max ) < 0, and would have called the1-to-32-bit group primitive with zero bits without it.
After, on the same corpus:
What changed, item by item
1. int_relative: the domain.
ReadStream.serializeIntRelativenow handsevery tier — the one-bit tier, the five bounded tiers and the absolute tier —
to a new
acceptIntRelative, which reconstructs in along, refuses unlessthe result lies in
[0, 2^31 - 1]and strictly exceedsprevious, and writescurrent.valueonly on acceptance. The absolute tier's 32 raw bits go throughInteger.toUnsignedLong, so a top-bit-set value is out of the domain ratherthan a negative sequence number. A refusal is terminal (item 6).
The two tests that expected a wrapped result are replaced, not accompanied:
intRelative: gaps wider than 2^31 travel through the unsigned domainandintRelative: reconstruction near INT32_MAX wraps in the unsigned domainaregone. In their place, in
IntRelativeTests:intRelative: the whole domain travels, from zero to the top— write, measureand read at the domain edges, including
0 -> 2^31 - 1through the absolutetier.
intRelative: a reconstruction past the domain is refused in every tier— sixdifferences, one per tier, each with the corpus's discriminating shape: the
same bytes refused one step past the domain and accepted one step inside it,
with the destination checked unwritten on the refusal.
intRelative: the absolute tier reads its 32 raw bits unsigned—0x80000000in the absolute tier is refused and writes nothing.
2. The corpus.
conformance/int128.txtandconformance/int_relative.txtare vendored verbatim from the core commit.
test/serialize/tests/ConformanceTests.javaparses the vector format from STANDARD.md's "The vector format" and runs every
record: accepted vectors must decode to the stated value and consume the stated
bits, refused vectors must be refused with the destination left at a sentinel.
No expectation is regenerated, and an operation with no runner fails the suite
rather than being skipped, so a vector file added upstream cannot go unrun.
3. STANDARD.md. Vendored at the repository root, verbatim from the core
commit — this repository had no copy. The new
spec-syncCI job compares itagainst upstream
mainwith the same wording the other ports use, and a secondstep does a full
diff -ruofconformance/against a shallow clone, so anadded, removed or edited vector file fails the job.
4. Ranged int128.
min <= maxon all three streams(
ReadStream,WriteStream,MeasureStream), replacing the strictmin.compareTo( max ) < 0. A degenerate range now short-circuits on bothsides: the writer returns before
writeGroups128, and the reader assignsminand returns before
readGroups128, so the zero-bit case never reaches the1-to-32-bit group primitive in either direction. The measure already added
zero.
compressed_floatis untouched and keepsmin < max.Test:
int128: a degenerate range is legal and costs zero bits, on the corpusvector's bounds (2^100 + 7), checking that the writer emits nothing, the
measure adds zero, and the reader takes the value from
minoff an emptystream — and it runs in both build shapes (item below).
5. Non-mutation. The int_relative absolute tier was the one assign-then-check
site; it is gone. Every other refusal in
ReadStreamalready checked beforeassigning. The corpus test asserts non-mutation on every refused vector, and the
terminality tests assert it after every latched failure. USAGE.md and the
BitStream/ReadStreamclass comments now state the rule and both itslimits: a caller-owned buffer (
serializeBytes,serializeString,serializeWideString) is unspecified after a refusal, and a composite read mayleave earlier members written.
6. Terminality, by latch.
ReadStreamcarries afailedflag. Every readentry point checks it first and returns
falsewithout touching the reader, andevery refusal sets it through a
fail()helper;reset(...)clears it, andisFailed()reports it. The flag rather than a poisoned position, because thezero-bit paths (
serializeInton a degenerate range,serializeInt128,serializeFixed) never consult the past-end check and would otherwise succeedon a failed stream. Tests in
StreamTests:terminality: a refusal latches, and every later read fails and writes nothing— six failure kinds: past the end before any consumption, past the end after
partial consumption, an offset above a range's headroom, nonzero alignment
padding, a malformed string payload, and an int_relative read past the domain.
After each, a one-bit read, a zero-bit read and a bool read must all fail
and leave their cells untouched, and no bits may be consumed.
terminality: reset clears the latch.7. Checked builds.
WriteStreamandMeasureStreamgainedassert previous >= 0onserializeIntRelative, andReadStreamasserts ittoo:
previousis caller state that never arrives off the wire, and withprevious < current.valuealready asserted, the pair pins both ends of thedomain.
8. Docs. README and USAGE.md now say what the page says, in this library's
words: the domain and per-tier reconstruction check for int_relative,
min <= maxand the zero-bit degenerate range on every ranged width, the latch and what
clears it, non-mutation and its two limits, and the corpus. The STANDARD.md
links point at the vendored copy. Nothing left that contradicts the page — the
"bounds must satisfy
min < maxstrictly" sentence and the "strictly increasingunsigned 32-bit sequences" description are both gone.
9. Version. This repository carried no version anywhere — only the
v1.0.0tag — so one is introduced:
SerializeUtil.VERSION, set to1.1.0, with READMEnaming it. The release tag for this round is v1.1.0.
The assertions-disabled run
The audit's note was right that the suite refused to run with assertions off,
and the reason was sound but too broad: a forgotten
-eawould have passedsilently while testing nothing on the write side. That guard is kept and
narrowed rather than removed — the suite still refuses an accidental
assertions-off run, and now accepts an explicit one,
AllTests --release, whichin turn refuses to run with
-ea.make test-releaseis the new target,makeruns both shapes, and CI runs both as separate steps.It is worth having on its own terms: every obligation STANDARD.md places on a
reader is a check, never an assert, and this run is what proves it. A refusal
that only held under
-eawould show up here as an accepted stream. Both shapesare green at 110 tests, 2271 checks.
Gates
make(both shapes) is green locally on macOS/arm64. The repository's pinnedJDK 21 is not on this machine, so the local runs used a JDK 26 with the
Makefile's
JDK_HOMEoverride, at the same--release 17language level andwith
-Xlint:all -Werror; CI on this PR is the run on the pinned toolchain.🤖 Generated with Claude Code