feat: use<- chain flattener (ILO-409)#711
Merged
Merged
Conversation
Collaborator
Author
|
needs manual rebase (conflicts after partial auto-resolve in: ) |
danieljohnmorris
force-pushed
the
feature/use-chain-flattener
branch
from
88cf14f to
f6250d4
Compare
|
Collaborator
Author
|
needs manual — rebase conflict in non-doc file(s) |
Collaborator
Author
|
needs deeper rebase — touches src/builtins.rs, src/runtime/mod.rs, src/verify.rs, src/vm/mod.rs (next-branch rebase conflicts) |
Four rows in the SPEC builtins table, paragraph + example in the HTTP section, regenerated ai.txt via build.rs, plus a 'when to reach for getx vs get' paragraph in skills/ilo/ilo-builtins-io.md. Site companion change pushed separately to ilo-lang/site.
SPEC.md gains the CLI invocation line in the inventory plus a full **ilo test** paragraph next to ilo check, covering engine selection, the -- engine-skip: passthrough, and the all-pass / any-fail exit codes. ai.txt gets the matching agent-spec entry inline. The skill's ilo-agent.md gets a Testing section with the three canonical invocations so an agent writing tests for its own programs sees the shape without round-tripping to SPEC.
Pins the lexer's whitespace-sensitive disambiguation in the identifier- syntax section: `a-b` is always one ident, `a - b` is subtraction. Mentions the new two-kebab-half hint surfaced by ILO-T004 so agents reading the spec see the canonical prefix and infix-with-spaces forms. ai.txt regenerated by build.rs from SPEC.md.
regression_reserved_names_doc has been failing on main since the crypto-primitives merge (PR #560 added b64 / b64-dec / hex without updating the enumerated 3-char reserved list). Adds them so the test goes green for any branch off main. ai.txt regenerated.
Add three new builtins for sliding-window aggregations over numeric lists: rsum (running sum), ravg (running mean), and rmin (running minimum). Output length is len xs - n + 1; empty when n > len xs. The asymptotic point is the whole point. The natural recipe in ilo today is map (i:n>n;sum (slc xs i (+ i n))) ..., which is O(n*w) and explodes for fat windows on long inputs. rsum/ravg use a running-sum (one add and one subtract per step), and rmin uses a monotonic deque, giving O(n) amortised total for all three. Tree-bridge eligible alongside the cumsum/cprod/ewm aggregate family: the tree interpreter does the work, VM and Cranelift inherit through OP_CALL_BUILTIN_TREE at zero opcode cost. ILO-R009 propagates on Cranelift via tree_bridge_propagates_error so the n=0 / negative-n / non-numeric-element error parity holds across engines. Builtin tags appended last to preserve every existing on-wire tag.
Three doc-only discoverability fixes hit by multiple personas in 2026-05-20 sessions. Canonical signatures in SPEC.md and src/verify.rs were already correct, but the skill docs weren't front-loading them enough for agents to find on first read. - SKILL.md gets a "Quick reference - things agents miss" block covering text concatenation (+, fmt, cat), HTTP custom headers (every verb takes an optional M t t map), and jpth being dot-path not JSONPath. - ilo-builtins-io.md HTTP section gets a bold lead on headers and a runnable mset/get!/pst! example. - ilo-builtins-io.md JSON section front-loads the dot-path warning before the jpth signature line. - ilo-builtins-text.md gets a concat/format quick-reference at the top so agents reaching for "how do I join two strings" pick + over cat. Personas: scrapingbee, tui-client (#26b headers), bearer-token-client (#26c jpth), 5+ across sessions (#26e concat confusion).
Trim the headers/jpth/concat additions to single dense lines so the budget-checker doesn't get worse than baseline. Net result: io drops from 1837 to 1748 (under its previous overage), text returns to ~baseline.
Adds --explain ILO-T043 reachable entry with the canonical fix walk- through (tail-position move, ret-wrap, ?h restructure). SPEC.md tail- call rules section now points at the new warning. ai.txt regenerated by build.rs picks up the SPEC change.
- SPEC.md builtins table notes that `rnd` returns random, not round, with the alias pair `rand`/`random` and a pointer at `rou`/`round` for rounding. - SPEC.md aliases table gains the `rand` -> `rnd` row. - ai.txt regenerated from SPEC.md via build.rs. - skills/ilo/ilo-builtins.md math section now spells out the round-vs-random trap so agents writing programs through the skill see the disambiguation in context. - CHANGELOG 0.12.1 lists `rand` as an additive ergonomic alias.
Add bisect xs:L n target:n > n, Python bisect_left semantics: returns the leftmost index i such that xs[0..i] < target <= xs[i..]. Empty list returns 0; target greater than every element returns len xs; ties resolve to the leftmost equal index. NaN target propagates as NaN to match argmax/argmin policy. The asymptotic point is the whole point. The natural recipe in ilo today is len (flt (x:n>b;< x target) xs) or hd (flt fn (enumerate xs)), which is O(n) per lookup. interp1d, sorted-lookup, percentile pickers and histogram binning all need this on inner loops; collapsing the scan to one builtin call gives O(log n) at the same token cost. Caller owns the sortedness precondition; we do not validate it, matching srt/unq/Python bisect precedent. Tree-bridge eligible alongside the argmax/argmin/argsort index- returning aggregates: the tree interpreter does the work, VM and Cranelift inherit through OP_CALL_BUILTIN_TREE at zero opcode cost. Returns plain n (not Result), so no tree_bridge_returns_result entry needed. No error propagation needed either: bisect cannot fail on sorted-numeric input. Type errors at the bridge raise ILO-R009 identically across engines via the normal interpreter arm. Builtin tag appended last to preserve every existing on-wire tag.
Add FixPlan { path, edits: Vec<FixEdit> } and FixEdit { line_start,
line_end, before, after } matching the Zero PR #137 schema. Wire
fix_plan serialization into --json output. No diagnostics emit plans yet.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add Diagnostic::derive_fix_plan() which pattern-matches on code and builds a structured FixPlan from the primary span + source text: - ILO-T004 / ILO-T003: parse "did you mean 'X'?" from hint, replace span - ILO-T032: bare fmt/fmt2 → prepend "prnt " before the call - ILO-L002: underscore ident → hyphenated form from suggestion backticks Wire enrich() closure in check_cmd to call derive_fix_plan() after attaching source and diag_path (file-only; absent for inline code). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Unit tests in diagnostic::tests::derive_fix_plan_* cover: - T004 typo rename, T003 type rename, T032 fmt prefix, L002 hyphen - absent-without-hint and absent-without-source guard cases - JSON shape (line_range array, before/after keys, path field) Integration tests in json_output_contracts exercise the full check --json → NDJSON stderr pipeline for each wired diagnostic. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
JSON_OUTPUT.md: expand ilo check section with diagnostic NDJSON shape, optional fix_plan schema (Zero PR#137 style), and table of codes that emit structured edits. ai.txt: add [fix_plan] note in ERROR DIAGNOSTICS section describing the field and which codes populate it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends derive_fix_plan() with three new code handlers:
- ILO-T008 (return type mismatch): wraps the offending return expression
with `str`/`num` when the hint identifies a cast; no-ops for non-cast types
- ILO-P011 (reserved keyword as identifier): renames the span token to
`<name>2` (safe mechanical rename with no semantic ambiguity)
- ILO-T041 (nil-coalesce on Result): rewrites `expr ?? default` to
`?expr{~v:v;^_:default}` by splitting on the ` ?? ` operator
Adds 5 unit tests in src/diagnostic/mod.rs and 4 integration tests in
tests/json_output_contracts.rs exercising the live binary via ilo check --json.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add Deno-style `--allow-net`, `--allow-read`, `--allow-write`, `--allow-run` CLI flags to gate IO builtins at the process level (ILO-59). - `src/caps.rs` — `Caps` / `Policy` structs, `parse_allow`, `check_net/read/write/run` helpers, and 26 unit tests. Denial messages now carry the `ILO-CAP-001` structured error code so agents can route on it. - `src/cli/args.rs` — four `Option<String>` fields on `RunArgs` wired to clap flags (`--allow-net`, `--allow-read`, `--allow-write`, `--allow-run`). - `src/main.rs` — `build_caps()` converts `RunArgs` flags into `Arc<Caps>`; `Caps::Permissive` is the default so no existing invocation changes behaviour. - `src/interpreter/mod.rs` + `src/vm/mod.rs` — `caps.check_*` calls at every IO builtin site. - `tests/capability_flags.rs` — 15 integration tests covering all four dimensions across both backends, plus `Caps::parse_allow` round-trips. - `examples/capability-sandbox.ilo` — runnable demo. - `SANDBOX.md` — operator guide: flag syntax, matching rules, capability matrix, recipes, backwards-compatibility note. - `ai.txt` + `SPEC.md` — capability matrix and flag reference added. `Caps::Permissive` is `#[default]`. Without any `--allow-*` flag the runtime is fully permissive — identical to pre-0.13 behaviour. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Ship `wro path s` (write-overwrite): truncates the target file before writing, creating it if missing. Returns `R t t` matching `wr`/`wra`. Wires enum, name, dispatch, verify, tree-bridge eligibility, example, regression tests (5 cases), SPEC, ai.txt, and skills/ilo doc. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Principle 1's naming rule said every hyphen doubles token cost, but stdlib ships ~20 hyphenated builtins. Added an explicit "by design, not contradiction" callout to both MANIFESTO.md and ai.txt explaining that stdlib names are a closed memorised vocabulary (same resolution pattern as the residual-English-keywords note in principle 4). Froze the set: no new hyphenated builtins will be added, existing names stay without a deprecation window. No code changes — doc-only resolution. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements `ilo trace <file.ilo> [func] [args...]` which runs the tree-walking interpreter and emits one JSON line per statement execution. Each line carries schemaVersion, line number, source text, all current bindings, and the statement result value. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add SAFETY comments to every unsafe block in the AOT .rodata deserialise path (ilo_aot_publish_program, jit_string_const, ilo_aot_parse_arg, compile_cranelift OP_LOADK), document invariants in src/aot/README.md, and wire up a cargo-fuzz target (fuzz/fuzz_targets/rodata_deserialise.rs) with a nightly CI job (.github/workflows/fuzz.yml). No nightly toolchain in this environment so the fuzz run is deferred to CI. Refs ILO-64. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds two new crypto builtins that hash hex-decoded bytes rather than UTF-8 text, enabling pure-ilo Bitcoin Merkle tree computation and other binary-protocol hashing without shelling out to Python: - `sha256-hex hex:t > t`: SHA-256 of hex-decoded bytes, lowercase hex. - `sha256d hex:t > t`: double-SHA256 (sha256(sha256(x))), Bitcoin shape. Both error ILO-R009 on odd-length or non-hex input. Tree-bridge eligible (VM and Cranelift JIT inherit via the tree interpreter). Includes cross- engine regression tests, two examples (sha256-hex.ilo, sha256d-bitcoin.ilo), and SPEC.md + ai.txt updates. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add `Phase` enum and `phase` field to `ErrorEntry` in `src/diagnostic/registry.rs`, covering all 88 registry entries; `ilo explain --json` now includes `"phase"` in its output envelope - Create `conformance/provenance-surface.json` mapping 25 surface features to their compiler function, source path, example fixture, and owned error codes - Generate `conformance/diagnostics/<CODE>.expected.json` golden files for the top 20 error codes (L001-L003, P001-P005, T001-T009, R001/R003/R005) - Add `tests/golden_diagnostics.rs` with 22 tests (one per golden file + provenance-surface validity + key-shape guard); runs under `--features golden`; supports `--bless` / `ILO_GOLDEN_BLESS=1` for one-line snapshot updates - Add `golden` Cargo feature and document the bless workflow in `CONTRIBUTING.md` Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… (ILO-397)
Cross-referenced every (Builtin, arity) pair in is_tree_bridge_eligible against
the verify.rs BUILTINS signature table. No gaps found: all bridge-eligible
builtins that return an ILO Result type ('R ...') are already present in
tree_bridge_returns_result.
Adds a regression test (vm::tests::tree_bridge_eligible_result_builtins_are_in_returns_result)
that encodes the complete eligible-and-result-returning set and asserts both
directions: every such builtin is in tree_bridge_returns_result, and
tree_bridge_returns_result reports true for each.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add tests/regression_cranelift_parity.rs with 9 cross-engine tests (VM + JIT + AOT) covering the three symptoms from the 2026-05-21 persona dogfood run: grp-nil on AOT with closure-captured key fn, mset accumulator perf at scale, and main>_ prnt-drop on AOT. Also add an ILO-371 dispatch contract comment in compile_cranelift.rs explaining the ACTIVE_PROGRAM TLS requirement for every HOF callback path, and pointing at the new regression file. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…d types (ILO-367)
An anonymous record `{name:"jane" age:30}` is now accepted wherever a named
record type (e.g. `person`) is expected, provided every field declared on the
named type is present in the anon record with a compatible type. Extra fields
on the anon record are permitted (width subtyping). The check applies at
function call sites and at return-type boundaries.
New helpers: `anon_satisfies_named` and `compatible_ext` (extends `compatible`
with types registry access). Five new unit tests cover: exact match, extra
fields, missing field (rejected), wrong field type (rejected), and return
position.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extends the labelled-arg resolver from ILO-71 to the `?` match-subject
parser paths in both stmt and expr positions. `?fn lbl:val{arms}` and
`r=?fn lbl:val{arms}` now rewrite to a `Call` subject with positional
args resolved by name, identical to how `fn lbl:val` works in regular
call position.
- `looks_like_labelled_call_match_subject`: lookahead probe detecting
`ident:value` pairs before `{`; disambiguates type-context colons
using the same `>` / `:` signal as `peek_labelled_arg_label`.
- `parse_match_stmt` and `parse_match_expr`: new labelled-subject branch
after the existing positional-call rewrite block; consumes leading
positional operands then hands off to `resolve_labelled_args`.
- Four parser unit tests covering all-labelled reversed, stmt, expr, and
mixed-reversed-label shapes.
All 3382 tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extends `ilo trace` (ILO-72 / #605) with two new flags: - `--depth statement|expr` (default: statement) — when `expr` is selected, additional JSON-line events are emitted for every function call and binary-op sub-expression, interleaved before the parent statement event. Each expr event carries `kind:"expr"`, `expr` (source text), `refs` (variable names touched), and `result`. - `--watch <name>` (repeatable) — filters output so that only events whose bindings/refs include the named variable are emitted. Works for both stmt and expr events; unknown names produce no output cleanly. Implementation touches: - `TraceDepth` value-enum + new fields on `TraceArgs` in cli/args.rs - `ExprTraceEvent`, `EXPR_TRACE_HOOK`, `CURRENT_STMT_SPAN` thread-locals and `run_with_trace_opts` in interpreter/mod.rs - `fire_expr_trace_event`, `collect_refs` helpers in interpreter/mod.rs - Call and BinOp arms of `eval_expr` fire expression events - `emit_stmt_event` / `emit_expr_event` with watch-filter in cli/trace.rs - 5 new integration tests in tests/cli_trace.rs (all 8 pass) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds hyphen-form discoverability aliases for all six map builtins to BUILTIN_ALIASES in src/ast/mod.rs: map-get→mget, map-set→mset, map-has→mhas, map-del→mdel, map-keys→mkeys, map-values→mvals. Regression tests in tests/regression_map_op_aliases.rs pin alias resolution across all alias/canonical combinations. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add examples/mget-or.ilo demonstrating the mget-or builtin across text-key, numeric-key, and text-value maps, and tests/regression_mget_or.rs pinning hit/miss/type-mismatch behaviour across VM and Cranelift engines. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Audits Gleam 1.x features against ilo 0.12.x, evaluating each against the six Manifesto principles. Top absorb candidates: use-style R T E flattener, typed todo/panic, | alternatives and multi-subject ? match. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ecks jpar! unwraps to _ (Unknown). mget on _ returns O _ (Optional Unknown). Previously, builtin type checks like len/hd/at/map/flt/srt/rev/zip/mkeys etc only accepted Ty::Unknown as a "skip check" escape — O _ was a concrete Optional type and would emit spurious ILO-T013 errors. Adds is_opaque(ty) helper returning true for _ and O _, and threads it through every builtin argument type-check arm that previously only passed Ty::Unknown through. Downstream chains of the form: r=jpar! body; v=mget r "items"; len v now verify cleanly. Real type errors (e.g. mget on a known L t) still fire. Regression tests cover the block-validator and fix-plan-emitter persona shapes from the 2026-05-21 A/B run (ILO-373). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…elpers (ILO-341) Extracts the 10 largest `if builtin == Some(Builtin::...)` arms from `call_function` into out-of-line `#[inline(never)]` helper functions, mirroring the existing `rolling_window_run` / `lstsq_run` / `matvec_run` pattern introduced in #494 / #506 / #515. Arms extracted (by size, largest first): - `ifft_run` (was 198 lines → arm now 11 lines) - `matmul_run` (was 130 lines → arm now 17 lines) - `rgxall_multi_run` (was 82 lines → arm now 19 lines) - `rgxall1_run` (was 60 lines → arm now 19 lines) - `rgxall_run` (was 60 lines → arm now 18 lines) - `quantile_run` (was 54 lines → arm now 16 lines) - `where_run` (was 60 lines → arm now 19 lines) - `stdev_run` (was 43 lines → arm now 9 lines) - `variance_run` (was 43 lines → arm now 9 lines) - `median_run` (was 43 lines → arm now 9 lines) - `rgx_run` (was 42 lines → arm now 18 lines) - `rgxsub_run` (was 44 lines → arm now 21 lines) 39 arms remain above the 40-line threshold (tracked in ILO-341 follow-up). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add `<-` bind operator that desugars `x <- expr ; rest` into
`?expr{~x: rest; ^e: ^e}`, flattening multi-step R-T-E chains.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
danieljohnmorris
force-pushed
the
feature/use-chain-flattener
branch
from
f6250d4 to
c0d00ec
Compare
Collaborator
Author
|
mini pc is reviewing this |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
<-lexer token (ArrowLeft) placed before<so logos picks the longer matchx <- expr ; restinto?expr{~x: rest; ^e: ^e}, flattening multi-step R-T-E chains<-operators are handled recursively viaparse_body_withtests/use_chain.rscovering single/multi-step ok, first/second error propagation, and mixing with plainletbindingsTest plan
cargo test— 3367 lib tests passcargo test --test use_chain— 7 use-chain integration tests pass🤖 Generated with Claude Code