Releases: mas-bandwidth/schema
Release list
v2.5.0 — every protocol id moves; tables in nine languages; write checks are debug-only
What you get
- Tables in all nine languages. C, C#, Go, Rust, Java, JavaScript, Dart and Elixir join C++: every language reads and writes fixed-class tables (with
?Toptional fields and[E]Tenum-keyed arrays) on the tolerant wire, the JSON text form and the reflection descriptors. Block and cook readers are in every language; the block builder is in C++ and C, and the cook's write side is in C++. - The tables language grew, in the C++ reference. Arrays of pointers and of unions, optional arrays, unbounded
[]T,map[K]V,*bytesand*stringblobs,string/bytes/flagsdefaults,wason tables, enum variants and union arms, and a union arm that takes any field type. ROADMAP.md says what each other language carries today. - A leaner table wire, in C++. Files use the id-table form: each 64-bit name hash appears once, in a table at the end, so a file naming forty ids across ten thousand fields carries forty ids. A message form for tables over a connection, with batching and a bitpacked body. Unknown fields can be retained through a read, values widen on read, and every refusal has a name. The other eight languages write the previous form today and move to this one in a later release.
- The cook and the block form.
schema cook,cook-checkanduncookare complete to the byte; C++ cooks from code as well as from the tool; every fixed table has a block form on the side at zero cost to the ordinary headers. - Packet wire, in all nine languages.
wstring(N)wide strings. Malformed UTF-8 is refused on read in every build mode.string,bytesandflagsfields take declared defaults. A union arm with no payload is encoded as its tag alone. Every enum exportsCountbesideMax. - Write-side checks are debug-only. In C, C++, C#, Rust, Java, Dart and JavaScript's flat tier, a release build no longer checks what you write; a debug build asserts it at the site. Go and Elixir keep refusing in every build, because those languages have no debug-only idiom, and JavaScript's stream tier still refuses in every build for now. Read-side checks run in every build, everywhere.
- Faster in release. With the write-side checks compiled out, Rust's packet write is 11.6% faster and C#'s 9.2%, and C's read path closed to a statistical tie with C++. The README's table was measured on a new machine, so its percentages do not compare with 2.4.0's; PERFORMANCE.md says what moved.
- Docs. TUTORIAL.md takes you from an empty directory to every feature. VERSIONING.md states the commitment for both wires. PORTING.md, COMPETITION.md, COMPARISON-TABLES.md and ROADMAP.md are new. Every page but README.md and ROADMAP.md now lives under docs/.
- Tooling. Only
schema fmtwrites a schema file; every other command leaves your files alone.schema packandschema unpackturn a directory tree into a table file and back.schema tables-baselinecommits a projection thatschema checkdiffs against, so an edit that changes what already-written data means cannot slip through unannounced.
Compatibility
Every protocol id moves. Rebuild and redeploy both peers together: a 2.4.0 peer refuses a 2.5.0 one rather than misreading it. Table files written under 2.4.0 do not open under 2.5.0; the table wire is pre-release until 3.0.0.
A schema stops compiling if a field's range excludes zero and it declares no default (an array cannot take a default, so widen its range), or if an enum has a variant named Count. [E.Max]T in a table body is refused; spell it [E]T. wstring(N) inside a table generates for C++ only.
v2.4.0 — faster generated Dart and JavaScript
What you get
- Generated Dart is 1.6x faster. Chunked write merges, chunked read windows, grouped element loops, and a bulk path for byte content. Regenerate and it's yours — no API change, no wire change.
- Generated JavaScript is 1.5x faster (writes 1.9x). Wide values ride an allocation-free BigInt lane through scratch stores, small fields pack into single-word merges, and string/bytes bodies move four bytes at a time.
- The performance table (README) reflects both, from one committed sweep: Dart at 225% and JavaScript at 318% of generated C++, alongside Rust 153%, Java 156%, Go 209%, C# 224%.
- The batch planner counts honestly in C#: fixed-point, 128-bit, and mixed-arm union bodies no longer receive batch entries their delegation costs don't earn.
- An absolute performance ledger guards the reference: every committed sweep joins a per-language time series (
./bench/tools ledger), and CI fails if the C++ curve ever regresses beyond its noise gate — relative numbers can shuffle; the absolute curves cannot lie.
Wire bit-identical across all nine languages throughout, held by the golden corpus and differential review.
v2.3.0 — tables: declare your own file formats
What you get
- Tables — declare your own file formats. A
tabledeclaration is a data record for content that outlives builds: config files, asset archives, tool output. Declare a root table, nest tables by value, and the generated C++ saves and loads it — no envelope imposed, no document concept, no prescribed structure. Your root table IS your format. - Version tolerance built into the wire. Add, remove, and reorder fields freely; old readers skip what they cannot name, new readers default what old writers never wrote, changed kinds are skipped and counted, out-of-range values clamp and count — nothing crashes on data from a different build, in either direction, and every event lands in a read report. Renames keep identity with
was = "old_name", and id collisions are refused at compile time. - Reflection for tools. Generated descriptors carry every field's name, kind, offset, bounds and nesting — walk properties by name, print, diff, or bind an editor with no RTTI and no schema files at runtime.
- Relocatable, and built wide. Generated table structs are trivially copyable (statically asserted); encode splits into measure and save over caller-provided buffers, and nested tables are length-prefixed — so builds can measure in parallel, prefix-sum, and scatter-write from many threads, and saves into exactly-measured buffers succeed byte for byte.
- Self-contained headers. Generated
<Unit>Table.hhas no dependency on the serialize runtime — includable from any TU. C++ today; every other backend refuses table units by name. - Dart fixes. Fixed-point field defaults are no longer double-scaled, and declarations named after
dart:coretypes (List,String, …) are refused at generation instead of emitting shadowing classes.
SPEC-TABLES.md specifies the full wire — neutral little-endian TLV a third party can implement from the document alone. The packet wire (type, the protocol id) is untouched by all of this: tables and packets version independently by construction.
v2.2.1 — generated C++ string helpers are guarded per package
What you get
- Two generated C++ units in one translation unit now compile. The emitted string helpers are guarded per package, so a TU that includes wire headers from more than one schema package keeps every package's helpers. Regenerate your C++ and the fix is yours; the wire is unchanged. A two-package regression test now holds this in CI.
Generated C is unaffected — its file-scope helpers were always correct.
v2.2.0 — faster generated code in Rust, Go, C# and Elixir
v2.2.0 — faster generated code in Rust, Go, C# and Elixir
What you get
- Faster serialization in four languages. Generated Rust, Go and C# use a flat word codec: field values pack into 64-bit words at compile-time-known shifts, one fused bounds check per chunk. Generated C# composes unions through batch cores with scoped batches. Generated Elixir builds one binary per barrier, packing group bits into fixnum-safe words. Regenerate your types and the speedup is yours — no API change, no wire change.
- Go pairs with serialize.go v1.14.0, which halves the work in the per-chunk bit path. CI certifies the pair.
- A benchmark you can run.
./bench/run.shmeasures the real generated code in every language over the same wire corpus — a representative 438-byte game packet with integers carrying 92% of the bytes — and./bench/run.sh --render <csv>prints the table from any sweep. Raw sweeps are committed underbench/results/. makebuilds the compiler and nothing else. A fresh clone needs only Go; the nine-language conformance chain ismake test.- The same bits everywhere. The wire is bit-identical across all nine languages, held by the cross-language golden corpus — including a degenerate-construct suite (zero-width fields, empty unions, boundary widths).
Per-language performance
Cost to serialize the benchmark packet, relative to generated C++ at 100% — lower is faster:
| Language | % |
|---|---|
| C | 100% |
| C++ | 100% |
| Java | 148% |
| Rust | 153% |
| Go | 208% |
| C# | 221% |
| Dart | 363% |
| JavaScript | 461% |
| Elixir | 1348% |
C and C++ report as a statistical tie: the pair sits inside the sweep's noise band, so neither is called a winner.
Nine Languages
Write your data types once and generate the code that reads and writes them — bit-identical — in C, C++, C#, Go, Rust, JavaScript, Dart, Java and Elixir.
- Three new backends: Dart, Java and Elixir. Each emits the same straight-line serializer the other targets do — monomorphic, constant-width, no reflection and no schema walking at runtime.
- Nine-language conformance runs in CI on shared corpora: every target reads and writes the same bits, proven on every commit.
- One benchmark contract covers all nine languages, and
bench/run.sh --quickruns the mixed shape under a minute per language, reporting nanoseconds per message and percent of fastest. The full interleaved standard remains the certification form. - Dynamically sized unions read correctly in the JavaScript flat tier.
- String reads carry no validation cost: well-formed UTF-8 is the writer's obligation under SPEC 4.7, and the reader trusts it.
The wire did not move. No protocol id changed in this release: a schema built with v2.0.0 and with v2.1.0 produces the same bytes and the same id, so deployed peers interoperate across the upgrade.
Production Ready
Write your data types once and generate the code that reads and writes them — bit-identical — in C, C++, C#, Go, Rust and JavaScript.
- Types, enums, flags, unions, constants and bitpacking: the complete language.
- One declaration produces the reader and the writer in every language, so they cannot disagree.
- The generated code is the straight-line serializer you would have hand-written — no reflection, no schema walking at runtime.
- Bounded arrays and integer ranges are validated wire facts: out-of-range input is rejected at read time, and every value costs only the bits its declared range needs.
- Six-language conformance runs in CI on shared corpora: every target reads and writes the same bits, proven on every commit.
- On a representative gameplay packet: 28 bytes, against Cap'n Proto's 52, Protobuf's 56 and FlatBuffers' 72 — measured by running the real encoders, reproducible from COMPARISON.md.
The Go module path is github.com/mas-bandwidth/schema/v2.
v1.16.0 — success is silent, --verbose lists the files
The CLI is quiet by default: a successful generate, pack, check or fmt run prints nothing, so a build that wraps the compiler stays clean without any wrapper-side filtering. The new --verbose flag on those four commands restores the per-file report — the files generate and pack wrote, the files the formatter rewrote, check's ok line with the package and protocol id.
Commands whose printed output is their answer — id, projection, version — print it exactly as before. Errors and diagnostics always reach stderr, and exit codes do not depend on verbosity.
The compiler library API is unchanged: verbosity is CLI policy, held in cmd/schema where CLI policy lives. The publicapi gate now pins the shape from both sides of the module boundary — silent defaults, and an identical --verbose report from the in-repo and external builds.
v1.15.0 — every emitter generates the precomputed compressed float path
Generated code now uses the runtime's precomputed compressed float entry point in every emitter — C++, C, C#, Rust and Go — so a declaration's wire parameters are computed once rather than re-derived per call, and the emitters share one arithmetic home with the runtimes they target (issue #82).
The C interop and Rust interop pins move to the releases carrying those entry points and the normative integer clamp: SERIALIZE_C_TAG to v1.5.0 and SERIALIZE_RS_TAG to v2.2.0.
Also in this release:
- A bench row names its generated unit, so
real_packetgets a verdict in all five compiled languages (issue #104). - The bench standard gains string and wstring row definitions, with the batch mix rebalanced to at most 15% bulk (issue #64).
- The four non-C++ Objects units the advisory-round note missed are regenerated, and that note now says it is advisory, because the generated code does not honor it.
- The TU-boundary attribution is stated as history rather than in the present tense across four documents.
- Documentation describes what this library is, not speculative future work.
schema v1.14.0 — the extremes close out, and real_packet gets its instrument
No wire change in this release: for any user schema the protocol id and every byte on the wire are identical to v1.13.0. Three merges: the integer extremes close out across the symbolic targets, and the bench suite's largest blind spot gets its instrument.
The extremes corpus rides in every target
A doubled unary minus over an INT64_MIN constant no longer emits rust that rustc rejects as compile-time overflow — the rust emitter gains the house arbitrary-precision carrier gate and folds the unprovable intermediate, matching the C and C# pattern (#101, closes #99). C++ member initializers for uint64 defaults above INT64_MAX now carry their ull suffix through the same guard emitConst always had (#102, closes #100). Both shapes live in the shared corpus with all six targets green on them.
real_packet rows for go, rust and cs
The compressed-float-dense workload is now measured on the three backends where compressed-float cost matters most, driving the same pinned corpus as the C++ reference with a byte-identity check as each row's setup — a corrupted corpus emits zero rows rather than wrong ones. All six runners now stamp the same corpus id, so cross-language ratios on these legs stop being refused. Found and fixed en route: the generated C# bench unit had never compiled (ufixed fields below 16 bits met a runtime whose narrowest overloads are 16-bit); the C# emitter now widens through a temp per the established golang pattern, wire bits unchanged, and the unit gained a compile gate so silence of that kind cannot recur. (#103, closes #80 and #65)