v5.5.0: The compiler type-checks, and the collector under stress

Changed

• The arena grows in place; RAIL_ARENA_MB is where the first collection happens, not a
  wall. The runtime reserves 64 GB of address space at startup (or RAIL_ARENA_MB, if larger;
  untouched pages cost no memory) and bump-allocates inside it. When a collection frees under a
  quarter of the heap, or the free list cannot serve a request, the arena's end moves out
  (doubling, at least enough for the request) instead of the allocation spilling to malloc.
  A spilled object sat outside the arena, where the collector does not look: a spilled string
  printed as its tag byte, and a spilled cell's children were never traced. And a live set
  bigger than the arena collected on nearly every allocation: 300,000 live tuples at 16 MB ran
  for over two minutes; now 0.01 s. RAIL_ARENA_TRACE=1 prints rail_arena_grow for each move.
  The Linux target keeps its fixed 1 GB heap (tools/linux_libc.s sets no reservation), and
  with it the spill path. Tests t248, t249.

Added

• RAIL_GC_STRESS=N and tools/fuzz/gc_stress.py: the collector under load. The collector
  runs only when the arena fills, and at the default 1 GB almost no test program fills it, so a
  collector bug hides behind a correct answer (#109 did). RAIL_GC_STRESS=N makes the runtime
  collect on every Nth allocation and fill every freed block with an address in the unmapped
  first 4 GB, so an object the collector misses crashes or changes the output. gc_stress.py
  compiles every suite test and every self-contained tree program, runs each normally and then
  under the most aggressive N (1, 13, 127, 1021) that finishes in its budget, and compares. With
  the #109 bug put back it flags 44 suite tests; on master it flags 3 (shell's result, threads,
  variable patterns), each fixed below, and it now flags none. CI runs it on every push and pull
  request (job gc-stress). Test t245 runs a program at RAIL_GC_STRESS=3.
• show prints tuples, constructed values and bools. show (1, "a") is (1, a), show (Some (Pair 1 2)) is Some (Pair 1 2), walking the tuple length and the constructor tables
  (they printed their header bytes, nothing visible), and show (x > 3) is true or false
  where the compiler knows the value is a bool: a literal, a comparison, &&, ||, not, a
  function or parameter rail types gives type bool, a let bound to one. A bool is an int at
  run time, so inside a container or through a polymorphic parameter it still prints 1 or 0.
  types.rail emits __bool_ret_<fn> and __argbool_<fn>_<i> for this. t207 and
  tools/fuzz/known/string_ordering.rail now expect true/false. Test t243.
• Constructors are functions. A constructor with fields can be passed as a value
  (map Some xs, let f = Pair), and given fewer arguments than it has fields it is a function
  of the rest (map (Pair 10) xs), its given arguments evaluated once. The front end rewrites
  such a use to a lambda before type checking and code generation; until now the code generator
  emitted a null pointer for it (a TODO) and calling it segfaulted. docs/language-reference.md;
  test t241, tools/fuzz/known/ctor_as_value.rail.
• Variable and boolean patterns. A lowercase name in a pattern matches anything and binds the
  value for the arm's guard and body (| big if big > 100 -> 100 | other -> other); the
  scrutinee is evaluated once. | true -> and | false -> match booleans. Constructors are
  capitalized, so a lowercase pattern is always a variable. The parser rewrites a variable arm to
  _ plus a let, so every backend compiles it. docs/language-reference.md; tests t238, t239.
• Types decide representation. tools/types.rail infers a type for every top-level
  function with no annotations (Hindley-Milner, polymorphic per call-graph group:
  compose : (a -> b, c -> a) -> c -> b, with dyn for Rail's dynamic idioms), and codegen
  reads from it which parameters arrive as raw floats or ints and which results leave as raw
  floats. The call-site guessing passes that decided this (argf_scan, hofpos, the rmap
  fixpoint, more than a thousand lines) are deleted. A claim codegen acts on has to hold for
  every value that can reach it, so inference runs in rounds: wherever dyn or a clashing type
  flows into a slot that claims a type (a parameter, a lambda's parameters, a result, an array
  a callee stores into), that slot is forced to dyn and the program is inferred again, until
  a round forces nothing new. The self-compile takes about 5 s instead of 20 (the guessing
  passes cost 16.6 s; inference with its rounds takes 1.6 s on the compiler's source). Results
  the types prove float leave as raw bits (the MHD kernel's mk_pressure); values the passes
  had guessed int but that come out of untyped data stay generic. 104 runnable programs in
  the tree print the same as before. rail types <file> prints every signature, every type
  error and how many parameters and results have a static representation; 301 of 487 files
  type-check. rail infer reports result kinds from the types. docs/TYPES.md; tests t217,
  t218, t220 to t222.
• Typed constructor fields. A field may name its type (int, float, str, bool,
  dyn, a declared type, or [t]): type Expr = | Num int | Add Expr Expr | Many [Expr],
  and a match binds each field at its type, so rail types infers eval : Expr -> int. Every
  construction is checked against the declared types; one that passes another type is a type
  error, and that field falls back to generic representation, so the program runs as before.
  A field written as a plain name (| Box v) declares nothing, as before; no declaration in
  the tree used type names as fields, so nothing that compiled changes. docs/TYPES.md,
  docs/language-reference.md; tests t225 to t227 and infer_match_typed_fields.
• The parser builds a typed syntax tree. tools/ast.rail declares Rail's AST as data types
  (Node, Arm, Decl, ...), and the parser produces them: pa : [(str, str)] -> (Node, [(str, str)]), pprog_ast : [(str, str)] -> [Decl]. The passes after the parser still read
  the older list form, which ast_decls_list produces, until each is moved over. Every file in
  the tree compiles to byte-identical assembly.
• The type checker type-checks itself. tools/types.rail reads the typed syntax tree and
  keeps its own state typed: types are a Ty data type instead of lists, flows, events and
  body records are data types, and the store is a Store of typed arrays instead of one
  array holding everything. rail types tools/types.rail reports zero errors (it had 453), and
  the compiler's own count drops from 1,739 to 1,318. It found a real inconsistency in itself
  on the way (hm_join returned a bool on one branch and an int on the other). Output is
  unchanged: rail types prints the same for every file in the tree, and every file compiles
  to the same assembly.
• The ARM64 and x86 code generators read the typed syntax tree. cg, compile_func and
  every predicate they use (is_float, is_int, max_sl, the self-loop and early-return
  paths, closure conversion's free variables) read Node through tools/ast.rail's accessors
  instead of list positions, and the arity map is built from typed declarations. x86_cg and
  x86_compile_func share those predicates, so the x86 backend's list copies of them are gone.
  Every file in the tree compiles to byte-identical ARM64 and x86 assembly, and the compiler's
  type errors drop from 1,318 to 869.
• The wasm backend and rail safe read the typed syntax tree. rail wasm and rail safe
  take the parser's Decls directly; the code generator (wg, wg_tail), its float and call-site
  passes, the named-function rewrite (which now builds Nodes) and the safe-mode checker match on
  Node. Lambdas are identified by ast_key (a length-prefixed serialization in
  tools/ast.rail) instead of show on a list. Every file in the tree emits byte-identical
  WAT, and the compiler's type errors drop from 869 to 643.
• The Cortex-M and RISC-V backends read the typed syntax tree. Their environment entries
  are a data type (CmEnt: a parameter in a register, a local in a frame slot, a constructor
  with its tag and arity) instead of lists like ["P", name, reg], and cm_emit_expr and
  cv_emit_expr match on Node; the RISC-V backend's copy of cm_collect_args is gone. Every
  file in the tree emits byte-identical Thumb-2 and RV32 assembly, and the compiler's type
  errors drop from 643 to 567.
• The checks read the typed syntax tree. Parse-error collection (ce_decls), the
  exhaustiveness check, the arity check and the warning pass (tc_infer) match on Node and
  Decl, and so does gpu_map's kernel text (ast_to_metal). Every file in the tree, and a set
  of programs written to trip each check, produce the same errors, warnings and assembly as
  before; the compiler's type errors drop from 567 to 468.
• The optimizer reads and builds the typed syntax tree. Constant folding, map fusion, the
  gpu_map dispatch and the fold ... (range n) rewrites match on Node and build Nodes, and
  its pattern helpers return NNull where they returned []. Every file in the tree compiles
  to byte-identical ARM64 and x86 assembly; the compiler's type errors drop from 468 to 406.
• The auth synthesis builds the typed syntax tree. inject_auth_fns takes and returns Decls:
  the projection, digest, Prover and Verifier functions it synthesizes for an auth type are
  built from Node constructors, and the auth types it collects are (name, variants) pairs.
  The twelve auth programs in the suite and every file in the tree compile to byte-identical
  assembly; the compiler's type errors drop from 406 to 362.
• The AD synthesis builds the typed syntax tree. inject_grad_fns takes and returns Decls:
  forward mode (`grad_deriv...

v5.4.0: Hardening, and the beacon replays from one integer

Added

• \r is an escape in string literals (carriage return, byte 13), lexed like \n and
  \t and emitted through esc_asm_char as \r. Until now a \r literal was the two
  characters backslash and r, and CRLF had to be built with char_from_int 13. Test t211.
  Written by DeepSeek V4.1 Flash through the clerk's letters door on the night of 2026-09-20
  (the Rail ladder, rung C), judged by tools/fuzz/ladder/judge.py C through a real rebuild.
• tools/fuzz/ladder/: the night ladder's rungs and judge. A rung is a pointed task for the
  local night worker with a check program, an expected output and a code judge that rebuilds the
  compiler for compiler rungs, bounds wall clock and memory, and runs the suite. Rungs A and C
  climbed on 2026-09-21; B (the live miscompile mixed_int_from_fn) is still open.

Fixed

• bytes_to_str in stdlib/http_client.rail is linear. It joined once per byte, copying
  the accumulator every step: 100 KB of response took 5.5 GB of resident memory, which was the
  real reason HTTPS responses were capped near 64 KB. It now collects the characters into a list
  and joins once; the result is byte-identical. Test t212 checks 100,000 bytes; the ladder judge
  (judge.py A) caps the check at 1 GB. Written by DeepSeek V4.1 Flash (rung A, 2026-09-21).
• Seven miscompiles behind a green suite, found by the 2026-09-11 hardening
  loop (a fuzzer, a reducer, a bisector and a perf agent on a shared findings
  bus; every one passed 198/198, the fixed point and the CI fuzz seed). Each
  ships with its minimal program as a suite test (t206..t210) and a
  tools/fuzz/known/ case, and tools/fuzz/semantic.py's loop family now
  generates the shapes that were missing.
  • Tail self-call arguments. sl (n - 1) ((acc + 2) / 3) printed a heap
    address: the bottom-test loop had no fallback and emitted mov x20, x9
    with x9 never written. let v = acc * 2 in sl (n - 1) v gave 401 for 200
    and dn ((n - 1) * 1) never terminated: the fallback moved general (tagged)
    values into the raw loop registers as they were. A plain literal argument
    (f (n - 1) 5) was loaded tagged, so the loop ran with 11. The bottom-test
    form is refused unless every argument is direct, the fallback stacks the
    general values and untags them on the way into the registers, and the
    literal loads raw (t206).
  • String ordering. "abc" < "abd" was 0, and 1 with the two lets
    swapped: _rail_lt/gt/le/ge had no string branch and compared offset 8 of
    each object. They dispatch like _rail_eq now (t207). tools/x86_rt.s
    still lacks the branch and says so.
  • String parameters read as ints. app1 a = a + "x" returned a heap
    address and f a b = length (a + b) returned 0: a fn whose params are used
    only in + or an ordering op took the raw-register convention, the
    syntactic int mark, and the early-return raw compare, whatever its call
    sites passed. The call-site proof (every site heap, or sites disagree)
    now vetoes all three, and the proof reaches through a param handed on to
    another fn (t208).
  • Frames under-sized for sibling slots. Five sibling lets in one list
    literal predicted one slot and consumed five; the fifth wrote past the
    frame (SIGBUS). max_sl_list threads siblings cumulatively, an
    application spine reserves its closure slot, and compile_func re-sizes
    the frame from the slot cg actually reached, saying so when the predictor
    was short (t209).
  • Floats out of containers. let (a, b) = (1.5, 2.25) then a + b
    was SIGSEGV at 0x3ff8000000000000 (1.5's bits dereferenced as a pointer);
    through a 2-param fn it was 4.9e-324 (a tagged int read as float bits).
    Two conventions coexisted with no boundary: raw binary64 bits for
    statically-known floats, tag-6 boxes for the generic runtime, and the
    runtime's own mixed arms read the untagged operand as raw bits in one
    routine and as a box in the next. The boundary is now explicit (see
    docs/NUMERICS.md): a raw float entering a generic location (a tuple,
    list or ADT field, a closure-call argument, a parameter without an
    all-sites float proof, a match result, a non-float-returning fn's return)
    is boxed; a generic value read as a float goes through _rail_fval,
    which converts a tagged int, loads a box and takes anything else as raw
    bits. Every generic runtime routine reads through it too, which also
    closes known/mixed_int_from_fn (0.1 - (gint 0) was 0.1). A float
    accumulator loop if n == 0 then acc else go (n - 1) (acc +. x) is now
    recognised as float-returning (the fn may assume its own recursive call),
    and float_arr_new n v no longer marks its LENGTH float (the argument
    positions were read off the application spine) (t210).
• RAIL_ARENA_MB=6000 ./rail_native self fell off the allocator cliff
  once the compile allocated a few hundred MB more. The in-process
  self-hosted linker inherited an arena the compile had nearly filled and
  ran on the free-list scan for minutes (or died in it). The assembly is on
  disk before the linker starts and nothing from the compile is needed
  after, so self resets the arena to its pre-compile mark first, the way
  run_test does after every in-process compile.

Changed

• Codegen of the compiler's own source: 7.6 s to 1.7 s. compile_func
  injected the program's ~2,500 __ret_/__float_ret_ markers in FRONT of
  a function's locals, so every local lookup in cg walked past all of them
  first. They now sit behind the locals (keys are disjoint, so nothing else
  changes). The call-site analysis carries more kinds and costs 4 s more on
  the same source, and recording a call-site slot used to rebuild the whole
  marker map on every observation; the map is now touched only when a slot
  narrows. Self-compile of the compiler's source: 21.5 s to 17.5 s at
  RAIL_ARENA_MB=6000; at CI's 4000 MB, 19 s to 16 s with the peak
  footprint 2.4 GB to 1.5 GB and no collection. A small fixed source goes
  from 0.27 s to 0.31 s. Generated code is unchanged in speed (the perf
  agent's six benches within noise, outputs identical). Seed sha256 and
  suite count (203) updated.

• tools/fuzz/semantic.py generates mixed int-and-float arithmetic by
  default (--no-mixed restores the old grammar) now that
  known/mixed_int_from_fn is closed; its loop family gained the compound,
  let-bound and literal argument shapes.

• A bare ./rail_native self spun for hours instead of finishing. The
  compiler's own source no longer fits the 1 GB default arena's bump window;
  allocation falls to the free-list scan and a self-compile that takes 18 s at
  RAIL_ARENA_MB=2000 was still running at 240 s without it (the daily attest
  audit sat in one for 36 hours on 2026-09-09). CI had set 4000 since June; the
  README, the audit walker, the status page and two tools still ran it bare.
  self now refuses below 2000 MB with the command to run (RAIL_ARENA_MB=6000
  reproduces the committed seed; 4000 is the as/ld fallback CI uses), exits 2,
  and t205 checks the refusal. Every caller in the tree sets the arena, and the
  audit walker caps the compile at 600 s.

• The shell attestation verifier never bound the file to the signature.
  tools/attest/verify.sh compared the file's hash with the UNSIGNED
  artifact.sha256 sidecar field and verified the Ed25519 signature over
  witness.digest_sha256, without requiring the two digests to agree. A
  replacement artifact plus an edited sidecar reused a real signature and
  printed ok. Both digests must now equal the file's, and
  tools/attest/verify_selftest.sh holds the positive control and the
  replacement-artifact negative control against both verifiers (the Rail
  verifier already rejected it). Found by an outside first-pass review,
  2026-09-06.

• tools/verify/check.sh inferred success from the absence of the word
  FAIL, so a test process that crashed without printing it (exit 139)
  produced passed: 4 failed: 0. Section 2 now requires exit 0 and an
  N/N summary line; section 5 runs the Rail verifier against the newest
  release's attested compiler source instead of testing that a file exists;
  section 6 runs the verifier selftest.

• selfhost/f86f082: its sidecar signs bytes the public-surface scrub
  (c4f6050) later rewrote. Retired to .stale with a PROVENANCE.md
  rather than re-signed. VERIFY.md and README.md now say the seed links
  libSystem and that gpu_map needs a Metal device.

• A large negative folded constant as a comparison operand inside a user
  function failed to assemble (probe _ = if 0 < (0 - (65536 * 5)) then 0 else 1 gave mov x9, #-655359, rejected). The early-return prologue's
  16-bit mov guard was tgv <= 65535, which every negative tagged value
  satisfies; the positive half was fixed in f851882, the negative half was
  not. emit_load_int is now parameterized by target register
  (emit_load_int_reg) so the prologue never routes through x0, which
  still holds the first argument there: the first attempt did, and the
  compiler miscompiled ITSELF on the second generation (188/192, float
  garbage) while still reaching a byte-identical gen3 == gen4. A fixed
  point is not a correctness proof; the suite is. Found by
  tools/fuzz/semantic.py within 14 generated cases once probes were
  rendered in tail position. t192 neg_const_cmp_operand. Suite 191 -> 192.

• Two early-return functions in one program collided on .Learly_<n>
  (assembler: symbol already defined). The label now carries the function
  name. Same test.

Added

• Chip-away (2026-09-07): the last 48 em-dashes in tools/compile.rail
  (43 comments, 4 strings: two parser messages, an asm comment, the
  generate prompt) replaced per the house style. The x86_64 "literal bug"
  carried in notes since May is not in the code: `x86_...

v5.3.0 — Independence + 3× faster self-compile

Consolidation release: 19 PRs since v5.2.0 (#22–#40).

Headlines

• Independence checker (Phase 3a, #37) — a reference checker written outside Rail re-derives the integer core: the committed seed no longer asks you to trust Rail to check Rail.
• Self-compile 61 s → 19.2 s (3.2×) — parser cursor O(N²)→O(N) (#29) + exact-size GC free-list buckets (#36); autograd/JIT tapes go O(n²)→O(n) (#27/#28).
• Multi-argument lambdas \a b -> e (#33).

Verification surface

• One-command check-me kit: VERIFY.md + tools/verify/check.sh (#32)
• Public reproducible-build check: verify_reproducible.sh (#23), verified cross-host (#22)
• Self-generated truth matrix docs/STATUS.md (#31), grounding contracts (#34), integer div/mod differential fuzzing (#35), leak-guard pre-push (#25)

Docs & robustness: truth reconciliation (#26/#30/#38), latent O(N³) removed from free-var checks (#39), audit walkers de-noised (#40).

178/178 tests. The committed seed remains bit-reproducible — the nightly reproducibility witness stayed green throughout the series. Full notes: CHANGELOG.md.

v5.2.0 — Rail stands alone (no as/ld/codesign)

Rail's whole toolchain now stands alone — the compiler assembles, links, and ad-hoc code-signs itself, in pure Rail, in-process, with no Apple as, ld, or codesign anywhere in the build. v5.0.0 removed as/ld from the Linux ELF path; v5.2.0 closes the loop on the macOS host platform — including the ad-hoc signature arm64 requires — and extends it to FFI programs. The last external tool in Rail's build is gone.

Because Rail now owns every byte of the output — including the LC_UUID and section padding that ld randomizes — the self-compile is deterministic and bit-reproducible: the committed seed reproduces itself byte-for-byte (sha256 b02dd228…) in a single cycle. That upgrades attestation from tamper-evident to source-reproducible.

Highlights

• Pure-Rail AArch64 assembler (tools/v5/asm.rail) — two-pass, whole-program __text layout; gate-validated 1248/1248 instruction forms byte-identical to as, with the full compiler __text (651,184 bytes) byte-exact.
• Pure-Rail Mach-O linker (tools/v5/link_lib.rail) — bakes @PAGE/@PAGEOFF relocations, builds the __data/__bss/__mod_init image, emits classic LC_DYLD_INFO rebase + bind streams, stubs + __got, full symtab/dysymtab/indirect-symtab, then ad-hoc signs it (N-page signer).
• In-process by default — self_compile rail-links itself at RAIL_ARENA_MB ≥ 5000; below that (or with /tmp/.rail_force_as_ld) the original as/ld path runs, byte-for-byte unchanged. The committed seed is the rail-linked binary.
• FFI stands alone too — multi-dylib linking (second LC_LOAD_DYLIB + per-import ordinal) links libsqlite3 FFI binaries with no ld; stdlib/sqlite.rail gains prepared statements (prepare/bind/step/column/finalize) and zero-cost weak auto-linking.
• 178/178 tests, byte-identical fixed point. Compiler behavior unchanged — this release is the build pipeline, not the language.

Reproducible attestation (verified)

The rail-link path emits every byte itself — no host as, ld, or SDK — so the build is host-independent. The seed compiled on two independent machines, an Apple M4 Pro (macOS 26.3) and an M1 Ultra (macOS 26.4.1), is byte-identical: sha256 b02dd2284a9220c29f9e984b5e3c898a7d255751263a6547e50c7c2a8abe0bb4. Rebuild it yourself and check the hash.

Full notes: CHANGELOG.md.

v5.1.0 — Rail emits its own GPU kernels

Rail now generates Metal Shading Language from its op-DAG, JIT-compiles it via Metal's newLibraryWithSource:, and dispatches the kernel at runtime. Every kernel the GPU executes is emitted by an attested Rail binary — the substrate piece needed for end-to-end attested GPU training.

This release bundles the full GPU substrate the auto-emission pipeline rests on: per-op Metal kernels, the bf16 numerics regime that unlocks stable 10k-step training, the JIT compile foundation, two hand-fused kernels, and the DAG matcher + emitter that drive them.

Auto-emission pipeline

Module	Role
stdlib/jit_node.rail	JIT op-DAG types (JitNode, TracedTensor) + tape primitives + jit_nth. Pure-DAG module — no Tensor/transformer dependency, so codegen consumers import without pulling the training stack.
stdlib/jit_tape.rail	Execution tracers (jit_leaf, traced_rmsnorm, traced_matmul) layered on jit_node.
stdlib/jit_match.rail	DAG matcher. walk_tape returns a list of FuseMatch records (FuseRmsQKV, FuseSiluHad) in tape order, identifying subgraphs that fit known fusion shapes.
stdlib/jit_emit.rail	MSL emitter. emit_msl_for_match (tape, FuseMatch) returns the kernel source. Stubbed against known patterns today; v5.2+ replaces with shape-parameterized codegen driven by JitNode data.
stdlib/jit.rail	compile/dispatch shim. No longer owns MSL text; jit_compile_* pulls strings from jit_emit. External API unchanged for existing consumers.

Hand-fused Metal kernels

Kernel	Fusion	Speedup
fused_rmsnorm_qkv	RMSNorm + 3 matmul (Q/K/V) in one threadgroup-per-row dispatch. Exports Q | K | V | rstd | LN1 from a single packed buffer; rstd + LN1 kept for backward. 4 dispatches → 1.	35× over the per-op chain at seq=512, d=64
fused_silu_hadamard	SiLU(gate) * up in one elementwise dispatch. Exports h_act | sigmoid(gate); sigmoid kept for backward.	18× over the per-op chain

JIT compile foundation

• tgl_jit_compile_from_tmp_file — Metal's newLibraryWithSource: driven from a Rail-emitted .metal file. Returns a pipeline ID cached in g_jit_pipes for reuse across steps.
• tgl_jit_dispatch_1in1out / _2in1out / _rmsnorm_qkv / _silu_hadamard — per-pattern dispatchers with f64↔f32 host staging.

Per-op GPU kernels

• tgl_rmsnorm_save_f64 (1.8× over CPU), tgl_rope_apply_f64 (7×), tgl_silu_fwd_f64 (19×) at training shapes.

Per-op wins translated to ~2% per-step — confirming fusion, not per-op throughput, is the real ceiling, which is why the JIT pipeline above is the load-bearing thesis.

bf16 numerics regime

• tgl_matmul_bf16 + matmul_bf16 wrapper. bf16 has f32's exponent range, sidestepping fp16's step-2759 NaN cliff. Training scripts default to forward bf16 with f64 on embedding + LM-head + backward.

Training scripts (chunked-corpus sampler, 2-block d=64)

• tools/train/lm_v3_chunked_bf16_full_long.rail — bf16 forward, 10k-step stable, ~40% wall under f64 baseline.
• tools/train/lm_v3_chunked_jit_long.rail — Q/K/V + SwiGLU through fused JIT'd kernels. 200-step matched-seed pilot vs bf16 baseline: trajectory shape preserved, no NaN, both converge. 2.85% step-throughput improvement (3×3 alternating runs, seq=512 d=64 d_ff=192).
• tools/train/lm_v3_chunked_fp16_attn_f64_long.rail — falsification experiment ruling out attention as the fp16 culprit.

Tests + benches

7 JIT/GPU smoke tests + 2 benches added. Numerical parity verified: fused QKV max diff 1.67e-6 (f32 floor), silu-hadamard 1.5e-8. DAG matcher 5/5, MSL emitter 4/4, block-integration rstd/ln1/sigmoid parity all green.

Stability

• 140/140 compiler test suite still green.
• 2-pass byte-identical self-bootstrap unchanged — this release adds stdlib + foreign decls + Metal sources; the compiler core is untouched.

---

Full detail in CHANGELOG.md.

v5.0.2 — Attestation pipeline goes fully pure-Rail

Patch release. The first Rail release attested end-to-end through the Rail substrate — no curl, no shasum, no Python anywhere in the attestation path.

Fixes

• stdlib/file.rail: foreign fopen path mode -> int (084791f). fopen returns a file descriptor, not a FILE*; declaring it as ptr bypassed Rail's tagging and tripped a polymorphic untag for odd fds (3 → 1, i.e. stdout). Corrected the foreign return type.
• runtime _fopen: unwrap path + mode before open(2) (95d81de). Wrapped Rail strings carry their ptr at the heap header, so open() saw the header tag byte as the path. _fopen now calls _str_unwrap, matching the _rail_read_file contract. Closes the argv-vs-literal path bug.

Attestation — retired the shell escape hatches (bbda5dd): attest.sh, sign_attestation.sh, and publish.sh deleted; release_index.rail replaces the Python heredoc in attest_release.sh. attest.rail + publish.rail are now canonical.

Stability — new seed rail_native (3b89d0f5) is at the 2-pass byte-identical fixed point.

Full detail in CHANGELOG.md.

v5.0.1 — Attestation hygiene + codegen tightening

Patch release. No new features.

Codegen — closed the half-applied ARM64 compile-fixes patch (emit_x1 large-immediate path, emit_x1 global-V fallback, O-handler RHS exclusion, ?-handler global-V exclusion). Self-compile fixed point and test suite unchanged from the v5.0.0 baseline.

Attestation hygiene

• Backfilled v4.0.0 / v4.0.1 / v4.1.0 release attestations (previously tagged-but-unattested).
• .gitignore whitelist for releases/**/rail_native.attestation.json so a new release can't silently lose that file to the rail_native.* wildcard rule.
• New docs/RELEASES.md operational runbook + 6 known gotchas.

Known limitation — attest.rail was still blocked on the ftell FFI bug, so attestation in this release (including its own artifacts) still used the documented tools/attest/attest.sh shell escape hatch. Closed in v5.0.2.

Full detail in CHANGELOG.md.

v5.0.0 — Self-hosted toolchain (Linux ELF substrate)

Rail produces its own aarch64 Linux ELF binaries. Encoder, assembler, static linker, and ELF writer are pure Rail. For the supported subset of inputs, the build pipeline invokes no external as, ld, or codesign.

What ships

Module	Lines	Role
jit/arm64.rail	+200	23 new encoders for the Linux mnemonic set (ldrb/strb imm/reg-offset/post-index, clz, neg, cmn imm+reg, rev, rev16, fneg, frinta, fcvt s_d / d_s, tbnz, stp/ldp pre/post-index, add/sub imm, asr/lsr/lsl imm). 31/31 byte-verified against as.
stdlib/elf.rail	175	Elf64 writer for static aarch64 binaries (one PT_LOAD for tiny, three for full text+data+bss-via-memsz).
tools/v5/elf_asm.rail	567	Section-aware ARM64 assembler + static linker. Two-pass: pass 1 builds (name, section, offset) label table; layout resolves to vaddrs; pass 2 emits bytes with adrp/:lo12: symbol resolution. Handles .text/.data/.bss/.rodata/.section __DATA,__mod_init_func (skipped), .quad/.byte/.long/.ascii/.asciz/.space/.comm/.p2align/.align, plus writeback stp/ldp variants and the adrp+add :lo12: symbol-load idiom.
tools/v5/compile_elf_full.rail	80	Driver: source .s → 3-pass pipeline → multi-segment ELF. Patches e_entry to _start.

End-to-end verified on aarch64 Linux (Pi Zero 2 W)

Program	ELF size	Result
exit42_linux.s	132 B	exit 42
fib_linux.s	204 B	exit(fib(10)) = 55
hello_linux.s	4105 B	prints "v5 lives\n", exit 9 (adrp + add :lo12: + write syscall)
bss_test_linux.s	4096 B	BSS counter loop, exit 7

Each binary's .text bytes are byte-equivalent to canonical as + ld output. Pipeline invokes neither external assembler nor linker.

compile.rail Linux pipeline fixes (precursor)

Two long-standing bugs in build_linux fixed: duplicate-symbol awk strip ran only on macOS-cross (now runs on Linux too, list broadened to cover the _rail_* runtime helpers that linux_libc.s redefines), and the macOS-only .section __DATA,__mod_init_func block is now stripped. tools/linux_libc.s gains _memcpy and _fmod — the two libSystem references with no Linux-side definition.

Tag-readiness checklist

• compile.rail's real Linux output traverses the new pipeline → byte-equivalent ELF
• Linux ELF substrate verified on aarch64 hardware
• 176 encoders byte-verified against as (89 + 56 + 31)
• No regression on ./rail_native test (136/140; 4 pre-existing tensor failures)
• No regression on ./rail_native self byte-identical fixed point
• CHANGELOG.md v5.0.0 entry
• Leak guard CI green

Deferred

• Pi self-host of ./rail_native test via Rail-only build — current GC layout reserves 1.2 GB BSS, exceeding Pi Zero 2 W RAM. Heap-size knob is v5.1 scope.
• macOS Mach-O end-to-end with dyld stubs — Phase 4b covered the libSystem-free subset; stub-aware Mach-O (LC_LOAD_DYLIB, indirect symbol table, __stubs, __got, bind opcodes) is ~1500 more lines. Tracked as v5.2.

🤖 Generated with Claude Code

v4.1.0 — Repo hygiene + leak-guard CI

Minor release. Comprehensive cleanup pass over the public tree. No
compiled-binary change; no language or stdlib changes.

CI + leak-prevention (B1)

• New workflow .github/workflows/leak-guard.yml — every push and PR
  is grep-scanned for the operator-recon pattern set (Tailscale IPs,
  internal SSH targets, home-directory paths, internal Slack channel
  IDs). Fails the build on any hit. Per-line opt-out via the comment
  marker leak-guard-allow. CHANGELOG.md and the guard file are
  excluded.
• ci.yml triggers extended to include next branch and v* tags.
  Test-count assertion generalised from hardcoded 137/137 to any
  matching N/N (master is 137, next is 140, future may grow).
• .gitignore — explicit ignores for .mcp.json, .ledatic/,
  .fleet/, *.pre-*. Closes the casual-git add recurrence path
  for the v4.0.1 leak class.

Branch hygiene (B2)

21 remote branches deleted from origin:

• 18 feat/* branches fully merged into next (security A/B/C lanes,
  x86 conformance harness, x86 runtime extensions, JIT fixes, docs
  refresh, auto-deploy, punch-list integration).
• jit (merged into next).
• track-mhd-kernel (merged into master).
• history-scrub-prep-2026-05-12 (unused experimental branch).

Remaining: master, next, half-s2-kernels (open compiler work),
compound/exp-008-bytes_to_str (halted POC artifact). Down from 26
branches to 4.

Doc pruning (B3)

~104 operator session-handoff files removed from the public tree:

• docs/plans/ (74 files) — operator session-planning notes
  (SESSION_HANDOFF_, PROMPT_SESSION_, WEEK_PLAN_, PHASE_, etc.).
• notes/ orphan files (12).
• docs/handoffs/ orphans (8).
• jit/ operator notes (9) — SCRATCH, CONTINUATION, SESSION_PROMPT*,
  AGENT_DRY_RUN, NEXT_STAGES, closures, floats.
• SECURITY_HANDOFF.md — internal Fort Knox punch list (the public
  policy lives in SECURITY.md).

Kept: docs referenced from CHANGELOG (notes/bootstrap_convergence_audit_*,
notes/phase3_external_pilot_pitch_v0); jit/ code + README + CHANGELOG;
docs/sessions/ versioned handoffs (CHANGELOG-linked).

Dead-code pruning (B4)

• Deleted tools/autocatalyst_v4.rail (broken — referenced runtime/llm.o
  which never landed in-tree, flywheel-v1 artifact).
• Deleted tools/ac_dashboard.rail (orphan, flywheel dashboard).
• Removed Razer3070 live-path references (decommissioned 2026-04-17):
  • tools/apps/control.rail — Razer fleet row + curl status segment.
  • tools/fleet/fleet_display.rail — razer_status/razer_iter/razer_max/
    razer_ping/razer_loss + RAZER row in the SPI-LCD render.
  • tools/mcp/rail_mcp.rail — tool_fleet_status no longer SSHes for
    nvidia-smi / v6_train.log; description updated.
  • tools/compile.rail — compile_x86 fallback message no longer
    recommends scp-to-Razer; suggests cross-tools or native host.
    Byte-identical bootstrap preserved.
• CLAUDE.md target list: 'Linux x86_64 (Razer WSL)' →
  'Linux x86_64 (cross-compile)'.

Structure pass (B5)

• Deleted 7 docs with no CHANGELOG or code references:
  RAIL_ENGINEER_PROMPT.md, flywheel-data-quality.md,
  flywheel-world-research.md, cascade-training.md,
  rail-plasma.md, railgpt-from-scratch.md,
  self-improving-playbook.md.
• Flattened docs/handoffs/ (down to a single entry after B3 prune):
  docs/handoffs/2026-05-02.md → docs/handoff-2026-05-02.md.

README polish (B6)

• Badge: v3.0.0 → v4.0.0; tagline → "Substrate maturity".
• Intro paragraph adds the v4.0.0 substrate-maturity lede (dual-backend
  parity, JIT in Rail, 30/30 hard-bench, multi-witness attest).
• New Releases section entry for v4.0.0 + a v4.0.1 sanitization note.
• History table extended: 7 new rows spanning v3.7.0 → v4.0.1
  (previously jumped from v3.0.0 to v2.23.0).

Verification

• Leak guard: 0 hits across tracked files for the union pattern set.
• Test suite: 140/140 on the v4.1.0 tree (modulo the documented
  /tmp/rail_out orphan-process collision when run concurrently with
  another rail_native test).
• git push on next: clean fast-forward; tag v4.1.0 cuts at 6 commits
  past v4.0.1, all CI-green via the new workflow.

v4.0.1 — Public-surface sanitization

Patch release. Removes operator-specific infrastructure strings from the
public tree: Tailscale IPs, SSH usernames, home-directory paths, internal
Slack channel IDs, and a stray operator MCP config. No behavior change;
the compiled binary is identical to v4.0.0.

What was scrubbed (~110 files)

• Hard SSH targets in tools/attest/*.sh, tools/fleet/*.sh,
  tools/fleet/fleet_display.rail, tools/apps/control.rail — replaced
  with <witness-user>@<witness-host> / <peer-user>@<peer-host>
  placeholders. Callers must supply real values via environment.
• Tailscale IPs (100.87.231.45, 100.79.50.108, 100.120.203.70,
  100.109.107.54, 100.109.63.37) replaced with role placeholders
  (<witness-tailscale-ip> etc.). Tailscale CGNAT-range addresses aren't
  reachable from the public internet, but they were operational recon.
• Home-directory paths (/Users/ledaticempire/, /Users/user/,
  /home/zemog/) replaced with ~/ or <HOME> placeholders across
  source, docs, docs/plans/, training fixtures, and Objective-C dispatchers.
• Operator service files — tools/fleet/witness.service,
  tools/fleet/witness_push.service, tools/fleet/com.ledatic.*.plist
  renamed to *.example with <user> / <HOME> placeholders. Existing
  install scripts already substitute these at install time.
• Operator MCP config — .mcp.json removed from the tree. It was an
  operator's Claude Code MCP wiring (path to tools/mcp/rail_mcp.py),
  not a build artifact; the MCP server still runs locally with a
  per-user .mcp.json outside the repo.
• Slack channel IDs / DM names in CHANGELOG.md, README.md,
  stdlib/slack_client.rail docblock, docs/sessions/HANDOFF_v3_6.md —
  D0ATHQ1BQD7 and brockbro2 replaced with <DM_CHANNEL_ID> and
  <test-dm>. Slack IDs don't grant access on their own, but these
  were the only remaining specific-channel references in the public surface.

What was intentionally NOT scrubbed

• reillygomez13@icloud.com in tools/deploy/gen_*.rail — public
  contact email rendered onto ledatic.org pages; meant to be public.
• Commit messages in the v4.0.0 surface — rewriting history would break
  existing clones for a topology-recon leak, not a credential leak.
  The forward tree is clean; git history retains the originals.
• ~/.ledatic/ path convention — generic project-named subdirectory,
  not operator-specific.

Verification

git grep -E "100\.(87|79|109|120)\.|zemog@|user@100|reillygomez@|\
ledaticempire@|/Users/ledaticempire|/Users/user|/home/zemog|\
Detro|D0ATHQ1BQD7|brockbro2"

→ empty across tracked files.

Why a patch release

v4.0.0 carried operator-recon strings inadvertently included via the
multi-witness publisher work on the next lineage. The master lineage
was scrubbed in c4f6050 (2026-05-06) but next hadn't received the
same pass. v4.0.1 brings the substrate-track tree to the same hygiene
standard.