Skip to content

Phase 5: codegen layer + multi-target backends (targets 0.13.0)#406

Merged
danieljohnmorris merged 49 commits into
nextfrom
feature/codegen-layer
May 21, 2026
Merged

Phase 5: codegen layer + multi-target backends (targets 0.13.0)#406
danieljohnmorris merged 49 commits into
nextfrom
feature/codegen-layer

Conversation

@danieljohnmorris
Copy link
Copy Markdown
Collaborator

@danieljohnmorris danieljohnmorris commented May 18, 2026

Phase 5: the codegen layer

Six stages on feature/codegen-layer. Closes Phase 5 and tags 0.13.0.

Stage What Commits
5a Typed HIR module between AST and code emission 5
5b Backend trait + Cranelift refactor (byte-identical at .o level) 4
5c Python emit moved behind the trait (26/26 byte-identical) 4
5d WASM Component Model backend via wasm-encoder (narrow walker) 5
5e Zero transpile backend + --0bin round-trip (narrow walker) 6
5f CLI lock, walker delete, cross-backend conformance, release notes 6

Final CLI shape

ilo build file.ilo            native binary (Cranelift; default)
ilo build file.ilo --wasm     WebAssembly Component Model
ilo build file.ilo --0        Zero source (.0)
ilo build file.ilo --0bin     native binary via Zero
ilo build file.ilo --py       Python source (.py)

Exactly five forms in ilo build --help. No --backend, --native, --cranelift, or --emit anywhere in src/main.rs. --emit python removed (migration hint stays one release).

Cross-backend conformance (tests/conformance.rs)

218 examples × 4 backends, honest per-backend reporting. Run with cargo test --release --features cranelift --test conformance -- --ignored --nocapture.

backend pass unsupported fail
cranelift 87 0 131
python 0 0 218
wasm 0 213 5
zero 0 209 9

Cranelift fails are pre-existing AOT bugs (duplicate ilo_strconst_*, opcode 176) and entry-point gaps between ilo run and ilo build. Python emits library code without a __main__ dispatcher (byte-identical emit is verified separately). WASM and Zero v1 narrow walkers cover the hello-world subset; everything else surfaces ILO-B201 / ILO-B302 and is counted unsupported. Walkers widen release by release.

Internal cleanup

  • src/hir/walker.rs, src/hir/raise.rs, tests/hir_roundtrip.rs deleted. The conformance suite supersedes the round-trip check.
  • print_build_help added; print_help replaces the legacy "Backends:" section with "Compilation (ilo build):".
  • Internal engine selectors (--run-tree, --run-vm, --jit) stay on the ilo run surface for 0.13.0. Sweeping them touches 170+ test files; deferred to Phase 6.

Docs

  • CHANGELOG.md 0.13.0 section dated and tabulated.
  • docs/releases/0.13.0.md long-form release notes for the site.

Test plan

  • cargo build --release --features cranelift clean
  • cargo test --release --features cranelift green (no regressions)
  • cargo test --release --features cranelift --test conformance -- --ignored runs and reports honest numbers
  • ilo build --help prints exactly the five manifesto-strict forms
  • ilo --help references the new "Compilation (ilo build):" section

Not in this PR

  • Phase 6 candidates: widen WASM/Zero walkers, wrap Python emit with __main__ dispatcher, sweep engine selectors, surface capability hints in the language server.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 18, 2026

Codecov Report

❌ Patch coverage is 65.93333% with 511 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/hir/lower.rs 66.38% 119 Missing ⚠️
src/backend/zero/mod.rs 49.30% 109 Missing ⚠️
src/main.rs 54.58% 94 Missing ⚠️
src/backend/wasm/mod.rs 69.42% 74 Missing ⚠️
src/hir/expr.rs 0.00% 44 Missing ⚠️
src/backend/cranelift/mod.rs 49.18% 31 Missing ⚠️
src/backend/python/mod.rs 60.37% 21 Missing ⚠️
src/hir/program.rs 58.82% 7 Missing ⚠️
src/backend/mod.rs 92.20% 6 Missing ⚠️
src/backend/wasm/emit.rs 96.93% 3 Missing ⚠️
... and 1 more

📢 Thoughts on this report? Let us know!

@danieljohnmorris danieljohnmorris changed the title WIP: Phase 5 codegen layer scaffold Phase 5: codegen layer + multi-target backends (targets 0.13.0) May 18, 2026
@danieljohnmorris danieljohnmorris marked this pull request as ready for review May 19, 2026 02:06
@danieljohnmorris danieljohnmorris changed the base branch from main to next May 19, 2026 14:17
@danieljohnmorris danieljohnmorris force-pushed the feature/codegen-layer branch 3 times, most recently from 8f7a32b to e61edcb Compare May 20, 2026 00:06
@danieljohnmorris danieljohnmorris force-pushed the feature/codegen-layer branch from 5fef15a to 880cb8e Compare May 20, 2026 16:13
WIP. No behavioural change. Documents the planned Phase 5 codegen layer
architecture and reserves src/backend/ for the refactor when it begins.

Cranelift AOT (src/vm/compile_cranelift.rs) and Python emit
(src/codegen/python.rs) remain the canonical codegen paths until this
scaffolding is filled in.

Scheduled work, not 0.12.0.
Stage 5a of Phase 5. Records the shape decisions for the HIR that sits
between the verified AST and the upcoming Backend trait: thin (mirror the
AST + a few desugarings), Rust-typed enums, no SSA. Documents the
departures from the AST (body tail-split, guard polarity fold, Ternary -> If,
Alias/Use/Error dropped) and the deferrals for Stage 5b+ (typed-AST
channel, effect rows, HIR stability).
Defines the HIR shape: Program with Decl::{Function,TypeDef,Tool} (Alias/
Use/Error dropped per design); Body splits prefix stmts from an optional
tail expression so backends get the implicit-return value in O(1); Stmt
exposes If (braced conditional) and GuardReturn (braceless early-return)
as separate variants with positive-polarity conditions; Expr mirrors the
AST one-for-one plus a value-level If lowered from Ternary. Every node
carries a Ty slot and an optional Span. Ty is re-exported from verify so
the lattice stays in lockstep.
lower(ast, verify_out) -> Result<hir::Program, LowerError>. Walks every
declaration, applies the documented desugarings (body tail-split, guard
negation folded into UnaryOp(Not), Ternary lowered to value-level If,
Alias/Use dropped), and produces a HIR program ready for backend consumption.
Type slots are best-effort today: literals and obvious binop returns get
populated, everything else falls back to Ty::Unknown. Stage 5b will swap
this for a proper typed-AST channel when Cranelift starts asking for it.

LowerError only fires when fed a Decl::Error poison node, which a correctly
sequenced caller (verify, then lower) cannot produce.
raise(hir) rebuilds an ast::Program from HIR. Not a perfect inverse of
lower: it doesn't recover Alias decls, guard polarity, or the original
Ternary spelling -- but it produces an AST with the same observable
runtime behaviour, which is enough for the Stage 5a round-trip gate.

walker::walk(hir, fn, args) raises HIR back to AST and dispatches through
the existing tree interpreter. Pure test infrastructure -- both modules
get deleted in Stage 5f when real backends consume HIR directly. Until
then they double as a reference oracle: any Stage 5b regression in
Cranelift's HIR consumption can be caught by diffing against this path.
For every examples/*.ilo file with a no-arg -- run: <fn> annotation,
parse + verify + desugar the program, then compare two execution paths:

  1. Run the AST directly through the tree interpreter.
  2. Lower AST -> HIR, raise HIR -> AST', run through the tree interpreter.

Both paths must produce the same outcome (same Value or same RuntimeError
shape). 375 cases across 228 example files pass with zero round-trip
failures, zero unparseable skips. Plus three focused unit tests for the
specific lowerings -- alias decls dropped, trailing-expr split into Body
tail, negated guard polarity folded into UnaryOp(Not).

Also adds a CHANGELOG entry under unreleased / 0.13.0.
Phase 5 Stage 5b. Adds the pluggable codegen surface. Concrete backends
will impl this trait; this commit only introduces the shape.

- Backend::emit(&hir, config) -> Result<Artefact, BackendError>
- Artefact { path, kind, metadata } with ArtefactKind::{NativeBinary, Wasm,
  SourceFile { ext }}
- BackendError::{Io, CodegenFailed, UnsupportedFeature} with to_json() for
  ilo build --json. JSON schema is documented on the method.
- Config is an associated type so each backend's options stay strongly
  typed at the call site.

Module-level docs explain why HIR is the input contract and why Cranelift
(the first concrete impl in the next commit) carries bytecode via a
side-channel until it's lowered to consume HIR directly.
Phase 5 Stage 5b. CraneliftBackend implements Backend by wrapping the
existing vm::compile_cranelift codegen. No codegen changes; the goal is
to thread the AOT path through the trait surface so subsequent stages can
add backends without touching main.rs.

- src/backend/cranelift/mod.rs holds CraneliftBackend + CraneliftConfig.
  The config carries the bytecode CompiledProgram as a documented
  side-channel until Cranelift is lowered to consume HIR directly.
- backend::cranelift::emit() is a free function the CLI dispatch site
  uses; the Backend trait method is also implemented but its associated
  Config pins a lifetime, which makes it awkward to call from main. The
  GAT shape is deferred until a second backend lands.
- main::compile_cmd lowers verified AST to HIR and dispatches through
  backend::cranelift::emit. No user-visible behaviour change.
- compile_cranelift::compile_to_binary gains an ILO_KEEP_OBJ=1 env hook
  that preserves the Cranelift-emitted .o after the link step, so the
  byte-identical regression test can compare codegen output without the
  noise of libilo.a content drift.
Phase 5 Stage 5b. Load-bearing regression gate for the backend-trait
refactor. Asserts that the post-refactor AOT path produces byte-for-byte
identical Cranelift object output to the pre-refactor path across the
full 136-example baseline corpus.

- tests/aot_byte_identical.rs builds each example with ILO_KEEP_OBJ=1 and
  sha256s the .o file. Compares against the baseline corpus; budgets a
  small soft-failure window for examples renamed or removed since
  capture.
- tests/aot-baselines/obj-baselines.tsv records sha256 + entry function
  per example, captured at the tip of Stage 5a immediately before the
  Stage 5b refactor.
- tests/aot-baselines/MANIFEST.md documents the capture point, why
  object-file equality is the right invariant (linked-binary equality
  breaks every time the crate gains a line of Rust code, since libilo.a
  is bundled), and how to regenerate when codegen intentionally changes.

All 136 entries pass post-refactor, confirming the trait shim around
compile_to_binary preserves Cranelift codegen exactly.
Stage 5c of the Phase 5 codegen layer. The existing python emit
(src/codegen/python.rs) moves into src/backend/python/ and implements
the Backend trait introduced in Stage 5b.

The emit code itself stays in emit.rs unchanged and still consumes the
verified AST. HIR (Stage 5a) doesn't yet carry the full surface python
transpile needs (expression shape, sum types), so PythonConfig carries
&Program as a side channel for now, mirroring how CraneliftConfig
carries the bytecode CompiledProgram. Lowering python emit to consume
HIR directly is a later refinement.

PythonBackend::emit writes the .py file to disk and appends a trailing
newline to match the pre-refactor 'println! to stdout' bytes -- the
byte-identical regression test added later in this stage pins this.

The two in-tree callers of codegen::python::emit (--emit python in
dispatch_run, the python bench in run_bench) move to
ilo::backend::python::emit_to_string. The python module is dropped from
src/codegen/mod.rs.
Adds the canonical ilo build form for the python backend:

  ilo build file.ilo --py            -> file.py
  ilo build file.ilo --py -o out.py  -> out.py

CompileArgs gets a --py flag (clap), Build/Compile dispatch forwards
it to compile_cmd, and compile_cmd short-circuits to PythonBackend
before the bytecode/Cranelift pipeline. --py and --bench are mutually
exclusive (the python bench shape would need a separate design).

The HIR is still lowered on the python path so the trait surface stays
HIR-first, even though PythonBackend currently ignores its hir argument
(see backend/python/mod.rs).
The manifesto-strict CLI is one canonical form per backend (Principle 2).
With ilo build --py now wired in compile_cmd, --emit python is the
legacy form. Pre-1.0 we break it cleanly rather than carry a deprecated
alias.

Invoking the old form prints a migration hint pointing at the new
verb and exits 2 so scripts notice the breakage immediately:

  error: `--emit python` has been removed.
         Use `ilo build <file.ilo> --py` instead.

Any other --emit <target> form gets the same treatment. Help text,
usage strings, and the two existing --emit tests in src/main.rs and
tests/eval_inline.rs all move to the new shape. Stage 5f will sweep
the remaining --emit dispatch branch once any internal callers are
proven gone.
10 baseline .py files captured from pre-refactor `ilo --emit python`
output for examples that cover the relevant surface: arithmetic,
indexing, ternaries, bang-propagation, the unwrap helper, the rd helper
(builtin-bridge), struct field access, char/list handling, chunks, and
the clamp shape.

The test walks tests/python-baselines/ and asserts post-refactor
`ilo build <example> --py` produces byte-for-byte identical output.
Adding more baselines is a one-line drop into the dir; the test picks
them up automatically.

CHANGELOG documents the python backend refactor and the --emit python
removal under the existing 0.13.0 unreleased section.
Adds the runtime dep (wasm-encoder 0.249) and dev-only validator
(wasmparser 0.249), both version-locked to the wasm-tools 1.249 line.

The WASI preview1 reactor adapter (~52KB, pinned to the Wasmtime v25
release) is bundled in-tree at assets/wasi-adapter/. wasm-tools
component new needs it to convert preview1 core modules into
Component Model components, and we don't want a build-time fetch -
offline builds and reproducibility matter more than 52KB of repo
weight.
Phase 5 Stage 5d. The first genuinely new backend behind the
Backend trait.

The backend walks HIR directly (no AST or bytecode side-channel) and
emits .wasm via wasm-encoder. Stage 5d covers the hello-world subset:
top-level prnt calls with literal arguments. Anything richer returns
BackendError::UnsupportedFeature with a hint pointing at the native
Cranelift backend.

Targets:
- wasm32-component (default) - wraps the core module with wasm-tools
  component new + the bundled WASI preview1 adapter, writes a
  sibling .wit describing the exported world.
- wasm32-wasip1 / wasm32-wasip2 - plain WASI preview1 core module.
- wasm32-unknown-unknown (alias wasm32-web) - no host imports.

Capability mismatches surface at emit time as ILO-B201, with a hint
naming the supported targets. Full matrix lives in
docs/wasm-capabilities.md. The error namespace ILO-B2## is reserved
for the WASM backend.

The encoder lives in src/backend/wasm/emit.rs and produces a single
_start function that loops one fd_write per string. Section ordering
follows the core spec (data after code). The nwritten pointer is
4-byte aligned so wasmtime accepts the write.
Adds --wasm and --target to CompileArgs (both flags surface on
both ilo build and ilo compile, identical to --py). The dispatcher
forwards them to compile_cmd which short-circuits before bytecode
compilation and calls into backend::wasm::emit.

--wasm is mutually exclusive with --py and --bench. --target only
applies with --wasm. Default output is <basename>.wasm. Default
target is wasm32-component.

The flag wiring mirrors the Python path in Stage 5c so the CLI
surface stays uniform across backends.
tests/wasm_emit.rs - 6 tests covering encoder round-trip
(wasmparser validation on every emitted module), capability matrix
per target, the JSON error shape for ILO-B201, target string
parsing including the wasm32-wasi / wasm32-web aliases.

tests/wasm_runtime.rs - 2 tests that spawn wasmtime as a subprocess
and check stdout matches what the tree interpreter would print.
Skipped automatically when wasmtime is not on PATH so CI still
runs clean in environments that lack the runtime.

examples/wasm-edge/ - a hello-world ilo program plus a starter
wrangler.toml showing the Cloudflare Workers deploy shape. The
.ilo file is annotated with -- run: / -- out: so the engine
harness still covers it across tree and VM.
docs/wasm-capabilities.md - per-target builtin matrix, error code
namespace (ILO-B2##), Component Model wrap notes, Cloudflare
Workers path, toolchain pins. Source of truth for which builtins
are available on which wasm target. Force-added because docs/
is gitignored at the repo root; the prep docs and this matrix are
the only tracked files under docs/.

CHANGELOG.md - appends a WASM backend entry to the existing
0.13.0 unreleased section. Captures the trait surface, CLI flags,
capability shape, bundled adapter, and the Stage 5d scope (Stage
5d ships hello-world; richer HIR lowering lands later).
Stage 5e targets zero 0.1.2 at /Users/dan/.zero/bin/zero. Recorded in
.zero-version at the repo root; upgrade procedure documented in
docs/zero-transpile-capabilities.md.
Phase 5 Stage 5e. Implements the Backend trait for an idiomatic Zero
transpile. Walks HIR directly (same shape as the WASM backend) and
emits Zero's canonical entry point:

    pub fun main(world: World) -> Void raises {
        check world.out.write("...\n")
    }

Stage 5e v1 covers the hello-world subset (top-level prnt calls with
text/number/bool literals). Anything outside that surfaces as
BackendError::CodegenFailed with an ILO-B3## code and a hint pointing
at the Cranelift native backend.

Error namespace: ILO-B301 (zero rejected source), ILO-B302 (HIR
construct unsupported), ILO-B303 (zero compiler missing), ILO-B304
(IO), ILO-B305 (entry not found).
tests/zero_emit.rs (4 tests) checks the idiomatic main shape and feeds
emitted .0 sources through subprocess zero check.

tests/zero_binary.rs (2 tests) round-trips ilo source -> Zero source ->
zero build native binary -> expected stdout. Skipped automatically when
zero is missing from PATH.

tests/zero_capability.rs (5 tests) asserts unsupported HIR shapes
surface with the documented ILO-B302/ILO-B305 codes and that the JSON
serialisation includes them.

examples/zero-bridge/ demonstrates the chain end-to-end with a hello
world that also acts as a higher-level regression test.
Capability matrix for the --0 / --0bin Zero backend. Covers the pinned
toolchain (zero 0.1.2), the canonical main shape, clean / shim /
unsupported construct mappings, the ILO-B3## error namespace, and the
upgrade procedure when the Zero compiler version moves.
Append the Stage 5e Zero backend entry to the unreleased 0.13.0 section
alongside the HIR, Backend trait, Cranelift refactor, Python refactor,
and WASM backend entries already there.
The cross-backend conformance suite supersedes the Stage 5a
information-preservation scaffolding. The walker raised HIR to AST and
re-ran the tree interpreter to prove the lowering preserved enough; with
real backends shipping their own conformance against the example corpus
the indirection is redundant. Drop both the walker and raise modules and
the test that consumed them.

src/hir/mod.rs no longer re-exports walker; updates the module-level doc
to reflect that the round-trip scaffolding has retired.
Adds print_build_help() listing exactly the five forms locked by the
Phase 5 brief:

  ilo build <file.ilo>            native binary (Cranelift; default)
  ilo build <file.ilo> --wasm     WebAssembly Component Model
  ilo build <file.ilo> --0        Zero source
  ilo build <file.ilo> --0bin     native binary via Zero
  ilo build <file.ilo> --py       Python source

Wired into compile_cmd (`--help` / `-h`), the friendly-usage handler
for bare `ilo build`, and an early intercept at top-of-main so it works
even when clap rejects the missing-source positional.

print_help (top-level `ilo --help`) replaces the old engine-flag
"Backends:" listing with a "Compilation (`ilo build`):" section
that mirrors the five forms. Drops the legacy `ilo build <file.ilo> -o
<out>` line; the -o flag is documented in print_build_help instead.

The internal engine selectors (`--run-tree`, `--run-vm`, `--jit`)
remain on the `ilo run` / positional surface for 0.13.0. Sweeping
those touches 170+ test files and falls outside the brief; it lands in
the next release.

Two test updates: tests/cli_verbs.rs accepts the build help on either
stdout or stderr, and tests/eval_inline.rs checks for the new section
heading and a build form rather than the old `--run-tree` listing.
tests/conformance.rs walks every examples/*.ilo that carries
`-- run:` + `-- out:` headers and exercises Cranelift, Python, WASM,
and Zero end-to-end. 218 cases at the 0.13.0 cut.

Per-backend skip markers via inline header:

  -- conformance-skip-wasm: <reason>
  -- conformance-skip-zero: <reason>
  -- conformance-skip-all:  <reason>

Reports per-backend pass / skip / unsupported / fail counts at the end.
Treats recognised backend error codes (ILO-B201, ILO-B302, …) and the
Stage 5d/5e narrow-walker error blobs as "unsupported" rather than
hard fails so the narrow walkers in WASM and Zero v1 surface as honest
coverage rather than artificial pass-rate inflation.

Marked `#[ignore]` because of cost (~70s release build, 218 × 4
backends). Run with:

  cargo test --release --features cranelift --test conformance \
    -- --ignored --nocapture

Honest numbers in 0.13.0:
  cranelift  87 pass / 131 fail
  python      0 pass / 218 fail
  wasm        0 pass / 213 unsupported / 5 fail
  zero        0 pass / 209 unsupported / 9 fail

The brief frames this stage as honest reporting, not artificial
completeness. Numbers move release by release as the walkers widen.
Phase 5 close. Cargo.toml, Cargo.lock, and the plugin marketplace
manifest all move from 0.11.8 to 0.13.0 in lockstep so the marketplace
integration test stays green.
Move the 0.13.0 entry out of Unreleased and date it. Adds the Stage 5f
section (CLI lock, walker delete, conformance suite) and the honest
per-backend conformance summary table. Tightens the breaking-changes
section: only `--emit python` is removed in 0.13.0; the
`--run-tree` / `--run-vm` / `--jit` engine selectors stay for now,
called out in a new "Not changed in 0.13.0" subsection so the
boundary is explicit.
Long-form release notes for the site. Covers the strategic framing
(codegen layer as the keystone of the two-layer-stack thesis), the
five-stage progression (HIR, Backend trait + Cranelift refactor, Python
refactor, WASM Component Model, Zero transpile, CLI + conformance), the
honest per-backend conformance numbers, the breaking-change surface
(`--emit python` only), toolchain pins, and the candidate work for
Phase 6.

docs/ is gitignored by default for scratch reports; force-add this one
the same way docs/wasm-capabilities.md and docs/zero-transpile-capabilities.md
got tracked earlier in Phase 5.
Drop the hardcoded /Users/dan/.zero/bin/zero const that leaked my dev
machine path into the public API surface and CLI --help text. Resolve
$HOME/.zero/bin/zero lazily at call time instead, with the relative
component as a private const.

Also fix the PATH probe in resolve_zero_bin: output().is_ok() only tells
us the process spawned, so a broken zero on PATH was being reported as
working and surfaced later as a cryptic ILO-B301 rather than the helpful
ILO-B303. Match on output.status.success() like the conformance helper
already does.

DEFAULT_ZERO_PATH is gone from the public API; replaced by a
pub fn default_zero_path() that returns Option<PathBuf>. Tests updated.
emit_match_expr_complex hardcoded `_m` and `_subject` as temp names.
Any program with a complex match inside the arm body of another complex
match silently overwrote the outer temp before its result was read,
producing wrong output with no diagnostic.

Thread a thread-local counter through emit. Each entry into a complex
match takes the current id, formats its temps as __ilo_m<N> and
__ilo_subject<N>, then bumps the counter. The counter resets at the
start of every emit pass so output stays deterministic across calls.

The __ilo_ prefix also stops user bindings starting with `_` (legal
in ilo per SPEC.md "In any binding position the name _ is permitted")
from colliding with codegen temps.

Adds a regression test that synthesises a nested complex match via the
AST builder and asserts two distinct __ilo_m<N> names appear, plus a
determinism test so future refactors don't reintroduce leakage between
emit calls. Existing assertions updated to the new prefix.
The hard_failures bucket was declared, never mutated, never panicked
on. Every Fail outcome went through eprintln! and the test passed
regardless, which defeats the point of a conformance suite.

Gate on Cranelift in 0.13.0 (it's the production backend and must not
regress). WASM, Zero, and Python stay soft-fail because the walkers
are intentionally narrow in this release; set ILO_STRICT_CONFORMANCE=1
to promote those to hard-fails too once 0.14 ships broader coverage.

Outcome::Unsupported stays soft for every backend by design: it's the
documented surface gap, not a regression.
…mpFile for adapter

Two small but real ergonomic bugs in the component-model wrap path.

wasm-tools component new errors were only including stderr. wasm-tools
itself isn't strict about which stream diagnostics land on — mirror the
zero-build pattern (both streams, trimmed, joined) so the user gets the
whole message and the exit status.

The adapter was being dropped to a temp file named with the PID. Reused
PIDs on long-lived shells could collide, and the manual remove_file at
the end swallowed any error. Switched to tempfile::NamedTempFile so we
get a unique name and RAII cleanup. tempfile moved from dev-deps to
runtime deps; it was already pulled in transitively, so this just makes
the dependency explicit.
The Backend trait doc claimed every backend reads HIR. In 0.13.0 that's
not true: Cranelift consumes CompiledProgram via its Config and Python
consumes the verified AST via PythonConfig. The HIR doesn't yet carry
the surface either backend needs. Acknowledge the side channels in the
trait doc, mirror the Cranelift NOTE comment at the Python dispatch in
main.rs so the next person reading the code finds it via either route.

Add a debug_assert that the function-decl count matches between the AST
side channel and the HIR trait argument. HIR lowering drops Use/Alias
decls so this counts functions only. Wires in cheap drift detection
without paying the cost in release builds.
Two narrow bugs in the conformance harness.

is_unsupported was doing substring matches against the combined
stdout+stderr blob: "only lowers", "Stage 5d", "Stage 5e". Any future
test case whose program text happened to print one of those phrases
would have been silently reclassified from hard-fail to soft-skip.
Replace with a regex on stderr lines only: \bILO-B[0-9]{3}\b. The
backend errors carry a structured code for a reason; use it.

parse_run was a naive whitespace split. A -- run: header with quoted
multi-word args got fragmented across whitespace, and combined with
the now-strict conformance gate would produce the same wrong shape on
every backend and silently pass. Switch to shlex::split for
shell-style quote-aware tokenisation; fall back to whitespace split on
malformed input so the case still runs and the diff surfaces.

shlex added to dev-dependencies.
The wasm/zero `unsupported()` helpers were returning
`BackendError::UnsupportedFeature`, which Displays as `"backend 'X'
does not support feature 'Y'"` with no error code. The conformance
suite's skip gate matches on `\bILO-B[0-9]{3}\b`, so walker
rejections in those backends slipped past the regex and were
classified as hard failures instead of soft "unsupported".

Route both helpers through `CodegenFailed` with structured codes:
- WASM: `ILO-B202` (HIR construct unsupported)
- Zero: `ILO-B302` (HIR construct unsupported)

Also tighten the gate regex from the open-ended `ILO-B[0-9]{3}` to
the enumerated unsupported subset (201/202/205/301/302/305). The
old pattern would have silently swallowed a real backend bug
emitting B203/204/304 by reclassifying it as "unsupported"; the
new pattern keeps hard-failure codes hard.

`UnsupportedFeature` stays in the enum for other callers.
`Display` was printing only the `message` field, so the structured
code was visible only via `to_json()`. The CLI surfaces backend
errors through Display to stderr (`eprintln!("WASM compile error:
{}", e)` etc.), and the conformance harness skip gate matches
`\bILO-B###\b` on stderr. With the code hidden, every soft skip
needed the message to repeat the code by hand.

Prefix `[ILO-BXXX] ` when the code is non-empty; preserve the old
behaviour when the code field is empty (legacy untyped errors).
… detail

Two changes:

1. Add a regression test in each of `backend::wasm::tests` and
   `backend::zero::tests` that asserts `unsupported()` returns
   `CodegenFailed` with the documented code (`ILO-B202` / `ILO-B302`)
   AND that the Display rendering satisfies the conformance harness's
   `\bILO-B(?:201|202|205|301|302|305)\b` regex. Either half slipping
   reintroduces the original miscount, so both are pinned.

   Both tests fail on the pre-fix `UnsupportedFeature` variant
   (verified by running them against the prior implementation).

2. Fix an edge case in the WASM B203 path: when `wasm-tools component
   new` fails with both stdout and stderr empty (signals, exec
   errors) the formatted message ended with a stray `": "` and no
   diagnostic. Fall back to `(no output captured)` so the message is
   self-describing.
argmax/argmin/argsort landed on next; the new builtins shift symbol
offsets in libilo.a, changing all Cranelift .o hashes. Also fix the
cond-vs-ret entry: the brc function was removed from that example during
the braced-cond refactor; swap to fall which still exists.
prnt returns its argument; the VM auto-prints the function return value,
causing double output when prnt is the tail expression. These examples
are designed for their respective backends (wasmtime / zero compiler)
so skip the vm engine in the multi-engine harness.
- conformance.rs: Skip and Unsupported variant fields are intentionally
  unused (conformance suite counts but doesn't print them); add
  #[allow(dead_code)] to suppress clippy dead-code false positives
- wasm_emit: emits_component_default now skips gracefully when
  wasm-tools is not on PATH (ILO-B203) instead of panicking; CI
  runners don't install wasm-tools so the test was always broken there
- aot_byte_identical: gate the byte-identity test to macOS aarch64
  only; baselines are Mach-O objects captured on macOS 15.5 arm64,
  Linux CI emits ELF x86-64 objects which differ at the binary level
  even for identical source
@danieljohnmorris danieljohnmorris force-pushed the feature/codegen-layer branch from 7bf8c2a to 7c7c002 Compare May 20, 2026 17:09
The python_emit_byte_identical test was written assuming example files use
the .ilo extension, but Phase 5 renamed them to .@. The source lookup now
probes examples/<bare>.@ first, falling back to examples/<name>.ilo.
@danieljohnmorris danieljohnmorris merged commit bebde99 into next May 21, 2026
4 of 5 checks passed
@danieljohnmorris danieljohnmorris deleted the feature/codegen-layer branch May 21, 2026 18:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant