Skip to content

t27b v0.2.0 -- 1,349 of 1,364 reference passes, a benchmark against zig, gcc, clang and rustc, and the signed receipt of this exact binary

Latest

Choose a tag to compare

@gHashTag gHashTag released this 10 Oct 23:23
59e2c87

t27b is the native AArch64 backend for t27: .t27 to AArch64 machine code with no LLVM, an in-process JIT test runner, and a Mach-O object emitter. The binary attached here is the exact binary the t27b lab ran the whole corpus with at the tagged commit, and the coverage numbers below come from the lab's signed receipt of that run.

Tag: t27b-v0.2.0 on master 59e2c87c4 (2026-10-10 22:07Z). Previous release: t27b-v0.1.0 on 2e06cef59 (#7701). Benchmark: #8817, report in #8855. Epics: #6063 (coverage), #8326 (the improvement loop), #6198 (the port of t27b into .t27).

Coverage at this commit (from the signed receipt)

The reference is t27c gen followed by zig test (Zig 0.16), run natively on x86_64. The corpus is every .t27 file under specs/.

v0.1.0 (2e06cef59) v0.2.0 (59e2c87c4)
corpus files 1,584 1,852
files the reference passes 1,099 1,364
t27b passes them, with runtime checks executed 899 (81.8%) 1,349 (98.9%)
t27b passes them vacuously (no runtime assert ran) 150 5
t27b blocks on a construct it refuses by name 50 10
tests that disagree with the reference 0 (over 1,064 files) 0 of 8,629 (over 1,372 files)
JIT-versus-interpreter mismatches 0 0

t27b also passes 3 files the reference cannot compile. In the same lab run, cargo test -p t27b passed 159 tests and the source-level random-program fuzzer (seed 1791671181) ran 1,000 programs with 1,987 tests: 1,932 agree with the reference, 3 disagree, 52 were rejected. The full per-file and per-test results are in t27b-0.2.0-corpus-run.json.

The 10 blocked files:

  • 5 are reference bugs, where t27b is right not to pass (ledger reason reference-bug): basics/15_loops (a type mismatch), port/trinity/src/tri/gen_perceptron (an assignment form), and three that return or store a frame address (trinity_tokenizer, conformance/frame_address_return, conformance/frame_address_store).
  • 5 are unimplemented: tri/t27b/eval_arith.t27 (run-time i128, stage 3), math/constants.t27 (@cos, #8677), two empty typed array literals (compiler/zig_empty_typed_array, queen-lease), and one new file blocked on std.* (bogatyrs_spec_structure, not yet named in the ledger; the ratchet lists it as UNLISTED).

The 5 vacuous passes are stubs with nothing to assert (#8766).

The ledger on master today (docs/reports/t27b_expectations.json at be30c2f78) has 1,363 entries: 1,349 pass, 5 pass vacuously, 9 are blocked (5 reference-bug, 4 unimplemented).

What changed since v0.1.0

68 commits touch cli/t27b or specs/tri/t27b. Every lowering decision lives in a .t27 plan spec, which t27c gen-rust turns into Rust (owner rule "Only t27", #8236).

  • Run-time i128. Stage 1 (#8750): a result in x0:x1, literals and comparisons. Stage 2 (#8795): annotated locals, pair constants, @intCast, checked + - * and @divTrunc.
  • anytype monomorphization (#8738): an anytype parameter the body reads gets one instance per argument type, @TypeOf is folded, and the skipped branch is dropped. Before it, #8028 (an unread anytype is void) and #7875 (anytype and [*]T in a signature no test reaches).
  • Big frames (#8764): frames up to 16 MiB of local aggregates, with slot addresses and prologues from a64 sub_const. This unblocked specs/compiler/lexer.t27.
  • A Verilog path (#8820): t27b verilog, decided in .t27 plans (extern modules, instances, declared ports, the use closure). The actors_hw self-test and JTAG top were written by it (#8830), and the 7-series primitives are declared once (#8833).
  • libm, ported to t27: @log, @log2, @log10 of an f32 (#7918); @floor, @ceil, @round, @trunc, @rem of an f32 (#7863); std.math.pow of an f32, which makes gf16.t27 pass (#8167); std.math.inf, log, log1p and an f64 @round (#7925).
  • Types and coercions: the type-mismatch family (#7747), @floatFromInt of a comptime_int (#7870), @intCast result types (#7905, #8711), integer @rem (#8644), constant / and % (#8027), @sizeOf (#8676), [N + 1]T (#8666), @bitCast, @intFromBool and the NaN/inf tests (#7818), @divTrunc and @panic (#7808).
  • Pointers and optionals: pointer equality (#7831), ?*anyopaque and @ptrFromInt (#7744), optional enums (#7934), ?void (#8695), optional module vars (#7913).
  • Arrays and literals: writable statics for @constCast array literals (#7694, #7786), array-repeat locals (#7912, #8689), a one-element list passed by address (#8692), string array literals for slices (#8174), untyped list locals as tuples (#8055), .{} result types (#7743).
  • std types carried when they are never opened: std.mem.Allocator (#8696), std.ArrayList and std.StringHashMap (#8706).
  • Refusals that match the reference: a reached fn with no body (#8730), an undefined argument the callee reads (#7801), the W585 scaffold (#7750).
  • The port of lower.rs into .t27 (#6198): ast_shape (#8569), ast_scan (#8350), ast_walk (#8258), source_text (#8252), type_text (#8235), comptime_float (#8247), eval_arith (#8086).
  • Lab and ledger: reference verdicts keyed by the generated code (#8208), the reused-verdict count (#8185), the ledger steward rules (#8656), no stored totals in the ledger (#7862), t27c test-report exits non-zero on FAIL or BLOCKED (#8152), bench discards (#8616), the frame-address conformance boundary (#8056).

The benchmark (#8817, report in #8855)

Measured on the Railway t27c lab at master 9a0f4c394 (2026-10-10), a shared 48-thread Xeon host, nothing on a laptop. The numbers are from docs/reports/t27b-bench-2026-10.md; read its caveats before quoting them. Run times there are qemu-aarch64 emulation and only rank the pipelines; instruction counts do not depend on the host.

Axis t27b Against
Compile time per spec (native x86_64 cross-compiler, tests included; geomean of 18 specs, warm median) 4.8 ms gen-c + gcc -O0 object 42 ms (8.8x); gen + zig Debug 3.05 s (640x); gen + zig ReleaseFast 13.8 s (2,900x). With all 20 specs in one zig invocation: 8.6x (Debug) and 46x (ReleaseFast)
Peak memory of the compiler (median over specs) 5.3 MiB zig 354 MiB (Debug), 500 MiB (ReleaseFast); gcc 27 to 32 MiB; clang 68 to 76 MiB; rustc 112 to 120 MiB
What has to be installed one 3.1 MB binary, no assembler, no linker zig 363 MB; gcc for aarch64 about 145 MB; rustc 790 MB
Agreement with the reference 0 of 8,344 tests differ zig ReleaseFast flips 7 files against zig Debug; gen-c + clang disagrees with itself across -O levels on 39 files (undefined behaviour in gen-c output)
Determinism bit-identical rebuilds, also from another directory ties gcc (without -g) and rustc; zig and zig cc record the build directory in DWARF
Coverage at the bench commit 1,296 of 1,311 (98.9%) zig built for aarch64: 1,311 of 1,311

Where t27b loses

  • Generated code is gcc -O0 class. Over the 20 tier-1 specs (chosen by gcc -O0 instruction count, without looking at t27b), t27b executes 3.4x the instructions of gcc -O2, 3.2x zig ReleaseSafe (the build with the same run-time checks) and 4.5x zig ReleaseFast; it is level with gcc -O0 (0.91x). Emulated run time: 3.4x to 4.1x the optimising builds.
  • Code is 1.8x to 2.0x larger than every optimising build.
  • Coverage trailed the reference by 15 files at the bench commit; zig on aarch64 passed all 1,311.
  • No Linux executable. t27b build emits Mach-O objects only, no ELF, and does not link. On aarch64-linux, t27b is a JIT test runner, not a compiler that ships binaries.
  • About 36% generated, counting the backend alone. 36% of t27b's own backend is generated from .t27; counting the t27c front end it mounts, it is 12%. It is compiled by rustc, so it is not self-hosted.
  • The DDC check shows backend agreement, not independence. The three routes share one front end, so the verdict is DDC_BACKEND_AGREEMENT.
  • The fuzzer still finds disagreements: 7 per 1,000 random programs in the run the report cites (3 in this release's run).
  • Per-test state reset costs time: jit.call copies module state before every test, so t27core runs 8.3x slower than zig Debug although it executes fewer instructions.
  • zig amortises, t27b does not: t27b has no multi-file mode, so the compile-speed lead shrinks to 8.6x and 46x when zig builds many specs at once.

Verdict. Not a breakthrough in generated code. t27b is a fast, lean baseline compiler that is exact against its reference: the profile of a first tier, the role that copy-and-patch JITs and Cranelift play as baselines, not of an optimiser. (No copy-and-patch JIT and no Cranelift was measured in this report.)

Verify it yourself

  1. The binary is the one that ran. sha256sum t27b-0.2.0-aarch64-linux gives
    0202a2aef518f3cb52cf14dc5d5413f2118457478d76dfe3195b2b1a192007e4.
    This equals the t27b_sha256 field of t27b-0.2.0-corpus-receipt.json. It was checked on the lab before upload, and again after the copy.
  2. The receipt is signed. Ed25519 under the domain line t27-corpus-receipt-v1, key a05db80f53c317f6 (public key .trinity/keys/a05db80f53c317f6.pub). From a t27 checkout (the keys are read from ./.trinity/keys), t27c corpus-receipt compare R R, with R the receipt, printed this on the t27b lab with t27c 0.5.2, and exited 0:
    base "59e2c87c4bd74c1137f4071b5e63095a285ce5e4" AUTH_MISSING_NONE AUTHOR leaves-bound true
    head "59e2c87c4bd74c1137f4071b5e63095a285ce5e4" AUTH_MISSING_NONE AUTHOR leaves-bound true
    totals true inputs true verdicts true outputs true: EQUIVALENT
    
    It checks the signature, and that every leaf rehashes to its signed root: inputs, verdicts and outputs. AUTHOR means a registered key signed these bytes. There is no verifier nonce, so it says nothing about freshness, and the signature is not rooted in hardware.
  3. Compare runs. Every master run has a receipt at https://t27b-lab-production.up.railway.app/runs/<sha>.receipt.json. t27c corpus-receipt compare BASE HEAD exits 0 EQUIVALENT, 3 IMPROVED_ONLY, 4 NEUTRAL, 1 REGRESSED, 2 REFUSED.
    • Comparing the two release receipts (v0.1.0 as BASE, v0.2.0 as HEAD) prints REGRESSED, exit 1, by the rule's strict per-file reading: 450 files improved and 184 are neutral, but 22 files left the corpus, the reference moved on 8, the asm listing fails on 2 files that pass, and 1 file is a new defect. That file is specs/base/ternary_encoding.t27, whose test is false: zig rejects it at compile time, and t27b runs it and fails the same assert at run time.

Assets

  • t27b-0.2.0-aarch64-linux: ELF 64-bit aarch64, dynamically linked against glibc, not stripped, 3,087,856 bytes. Built by the lab at /work/target/aarch64-unknown-linux-gnu/release/t27b.
  • t27b-0.2.0-aarch64-linux.sha256
  • t27b-0.2.0-corpus-receipt.json: the signed receipt.
  • t27b-0.2.0-corpus-run.json: the full run, with per-file verdicts, per-test results and the fuzzer summary.

Not in this release

  • No macOS binary. The JIT runs on macOS (MAP_JIT); build it with cargo build --release -p t27b.
  • No crates.io package. t27b mounts the t27c front end (bootstrap/src/compiler.rs) by path.
  • AArch64 Linux only. On x86_64 Linux, run the binary under qemu-aarch64, as the lab does.
  • No ELF output and no optimiser. Both are open work, with run-time i128 stage 3, @cos and the port share.