Skip to content

A live interop job: this port exchanges bytes with the pinned C++ reference - #5

Merged
gafferongames merged 1 commit into
mainfrom
interop-cpp
Sep 4, 2026
Merged

A live interop job: this port exchanges bytes with the pinned C++ reference#5
gafferongames merged 1 commit into
mainfrom
interop-cpp

Conversation

@gafferongames

@gafferongames gafferongames commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

The port exists to be wire compatible, and nothing in this repository measured that against the reference itself. The four ports with a live interop gate (c, cs, go, rs) each run one; this one did not.

What the job does

interop builds the C++ serialize library at the release SERIALIZE_TAG pins and runs it head to head with this port, on every push and pull request.

interop/interop.cpp and interop/serialize/interop/Interop.java serialize the same boundary message — every operation STANDARD.md defines, at the values where implementations disagree:

  • raw bits at 1, 7, 31, 32, 33 and 64, at all-ones and all-zeros
  • bool in both codes; align where it pads and where it must write nothing
  • uint8/16/32/64/128 at their domain edges, including the 128-bit all-ones
  • int, int64, int128: degenerate ranges (min == max, zero bits, mid-sequence with a live field after them), both ends of a narrow range, and the full domain at both ends — 32, 64 and 128 bits on the wire
  • fixed: both ends of a Q8.8 range, a degenerate Q16.16, the 34-bit Q48.16 path, the three-group Q112.16 top, a degenerate Q64.64 (zero bits at 128-bit storage) and the four-group Q64.64 top
  • int_relative: every tier of the ladder at both ends, the absolute tier at its smallest difference, and both domain edges
  • float/double as bit patterns: ±0, ±infinity, the largest finite, the smallest normal, the smallest subnormal
  • compressed_float: both range endpoints, the four between-quanta discriminators, a non-zero min, both clamp witnesses, and a one-bit field in both codes
  • bytes: a zero length block from an unaligned position, so the align inside it is load bearing
  • string: empty, full (15 bytes in buffer size 16), multi-byte UTF-8
  • wstring: empty, basic plane, the first code unit above the surrogate block, the largest code unit, an astral code point as its surrogate pair, and full

Then:

  • both halves write it and the files must be byte identical (483 bytes)
  • each half decodes the other's file, checks every value, and re-encodes it byte for byte
  • every proper prefix of the other's stream is a truncated stream, and both halves must refuse every one of them — the Java half additionally requires that hostile data never throws
  • both halves run the shared conformance/ corpus: the pinned reference reader and this port's reader, same files, same accepts, same refusals

The Java half honors this port's read allocation contract (BitReader requires eight zero bytes of slack past the data), which is what makes the truncation sweep a test of the missing bits rather than of what follows them.

Deliberately not carried: a NaN payload. STANDARD.md's bit-transparency claim covers it, but NaN payload bits do not survive every language's float type on the way to the wire, so a difference there would say nothing about the wire format. Each port pins its own NaN patterns in its own suite.

Two small supporting changes

  • ConformanceTests becomes public and gains a main(), so the corpus can be run on its own (make conformance). That is what lets both sides of the corpus claim live in one job instead of being split across two.
  • The Makefile gains conformance and interop targets, built the same way the tests are: against the library classes only, --release 17 -Xlint:all -Werror, run with -ea.

The pin

SERIALIZE_TAG: v1.16.0 in .github/workflows/ci.yml is the one place this repository names a version of the reference; the harness sources name none. The version bump is in SerializeUtil.VERSION (1.1.0 → 1.1.1), with the README's restatement of it kept in step.

Verified locally

Against mas-bandwidth/serialize at v1.16.0 on the pinned JDK: byte identical both ways, both re-encodes exact, all 483 truncated prefixes refused on both sides, corpus green on both readers, and make test / make test-release still 110 tests, 2271 checks, 0 failed.

Note on the red spec-sync job

The STANDARD.md ... match upstream job is red here, and it is red on main for the same reason: upstream mas-bandwidth/serialize moved STANDARD.md after this repository vendored it (#114, then #115). This branch does not touch STANDARD.md or conformance/ — the vendored copies are byte identical to main's, and the corpus still matches upstream. Re-vendoring the amended standard, and porting whatever it implies, is its own change.
🤖 Generated with Claude Code

…erence

The port exists to be wire compatible and nothing measured that against the
reference itself. The new interop job builds the C++ library at a pinned
release and runs it head to head with this port on every push and pull
request.

interop/interop.cpp and interop/serialize/interop/Interop.java serialize the
same boundary message: every operation STANDARD.md defines, at the values
where implementations disagree. Zero bit ranges on int, int64, int128 and
fixed point; the domain edges of the ranged integers and of int_relative,
every tier of its ladder at both ends; the maximum widths of bits, uint128
and the four group fixed point path; both sides of the alignment rule,
including the align inside a zero length bytes; empty and full strings; and
the wide string cases the surrogate rule governs, up to the largest code
point. Floats are given and compared as raw bit patterns, so a value
comparison cannot hide -0.0.

Both halves write it and the files must be byte identical; each decodes the
other's file, checks every value and re-encodes it byte for byte. Then the
hostile half: every proper prefix of the other's stream is a truncated stream
and both sides must refuse it without throwing, and both sides run the shared
conformance corpus, where the refusal vectors are the point. ConformanceTests
gains a main so the corpus can be run on its own, which is what lets both
sides of that claim live in one job.

The reference tag is named in ONE place, SERIALIZE_TAG in the workflow, and
the version in one, SerializeUtil.VERSION.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@gafferongames
gafferongames merged commit f29060c into main Sep 4, 2026
3 of 4 checks passed
@gafferongames
gafferongames deleted the interop-cpp branch September 4, 2026 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant