chore: sync next from main (catch-up 302 commits incl .ilo→.@ rename)#567
Closed
danieljohnmorris wants to merge 311 commits into
Closed
chore: sync next from main (catch-up 302 commits incl .ilo→.@ rename)#567danieljohnmorris wants to merge 311 commits into
danieljohnmorris wants to merge 311 commits into
Conversation
docs: document run builtin output schema (#5bj)
Context-aware ILO-P009 hints for ternary-brace misfires
tailored ILO-P001 hint + docs for glued-negative-literal misparse
add ILO-P102 hint for top-level bindings without main wrapper
make find_libilo_a honour CARGO_TARGET_DIR and .cargo/config.toml
rndn, atan2, fmod, clamp, and the unary trig forms (tan, asin, acos, atan, log10, log2) were absent from builtin_arity_tables, so the parse_call_arg nested-call branch never fired for them. A chained shape like 'abs rndn 0 1' fell through to parse_operand, returning a bare Ref(rndn), and the outer call swallowed the remaining operands producing 'arity mismatch: abs expects 1 args, got 3'. With the arities registered, parse_call_arg recurses into the inner call exactly arity operands deep, matching the existing behaviour for pow and min/max. Workaround paren-grouping is no longer needed. Closes pending #5ao.
9 regression tests pin the new parse behaviour across every public engine: abs/sqrt/pow outer composed with atan2/fmod/clamp/rndn/log10 inner, plus the original pending repro shape (y=+1 rndn 0 0.1) and a stochastic rndn run that asserts parse + execute succeed. examples/chained-num-args.ilo gives agents an in-context template for the now-working chain shape and runs through the examples harness, so any future grammar regression that drops the structural fix shows up there too.
SPEC.md Call Arguments section and ai.txt CALLS line now state that known-arity calls can chain directly without parens, with examples for the canonical 2-arg-numeric-under-unary shape. Variadic and unknown-arity calls still need paren grouping, called out explicitly so agents reach for parens only when they actually help.
Closed-form OLS via the normal equations: lstsq xm ys returns the coefficient vector b minimising ||xm.b - ys||^2. Same precision tier as solve (LU with partial pivoting); numerically inferior to QR/SVD for ill-conditioned designs. Collapses the 5-line recipe (transpose + matmul + matmul + solve + index-fiddling) into a single call, saving ~30 tokens per OLS use. Hit by the linear-regression persona today. Tree-bridge eligible: VM and Cranelift inherit through the bridge with no new opcodes. Errors as ILO-R009 on rank-deficient design, underdetermined system (cols > rows), row/length mismatch, or empty input. Doc sync: SPEC.md row + linalg prose, math skill builtins list + worked example, CHANGELOG Added entry. ai.txt regenerated by build.rs. 10 cross-engine regression tests in tests/regression_lstsq.rs cover perfect-fit line, intercept-only mean, multivariate exact fit, noisy 100-point overdetermined fit (seeded rndn), 1x1 system, underdetermined error, dimension-mismatch error, rank-deficient error, and the all-equal-ys boundary. examples/lstsq.ilo gives agents a worked OLS example with -- run/-- out assertions across every engine. Resolves pending.md #5am.
The first commit added the lstsq arm inline in call_function, which tipped the giant dispatch frame over the cargo-nextest stack budget. CI hit a SIGABRT on interpret_braceless_guard_fibonacci - the same recursion-depth bomb #506 (sha2/hmac) and #494 (caps fields) papered over previously and that #5av plans to fix structurally by decomposing call_function. Move the lstsq body out to a standalone #[inline(never)] helper `lstsq_run` next to lu_decompose / lu_solve. The arm in call_function is now a single return call, contributing zero frame growth. Local tests pass either way (default cargo test stack > nextest), so the fix is CI-driven.
feature: add lstsq builtin for ordinary least squares
parser: chain 2-arg numeric builtins without parens
`rand-bytes n > t` returns n CSPRNG bytes from `getrandom`, encoded as base64url-no-pad. Distinct from `rnd` (seedable uniform float for simulations) and `rndn` (seedable Normal float): this is the path for JWT jti claims, CSRF tokens, session IDs, and nonces. Output is base64url-no-pad so it drops straight into headers, cookies, and query strings without further encoding. Encoded length is deterministic: ceil(n * 4 / 3) chars. Tree-bridge eligible (arity 1, no FnRef, no I/O wrap) - VM and Cranelift JIT inherit at zero opcode cost. Cap at 1 MiB; negative or non-finite n surfaces as ILO-R009. CSPRNG output is never seeded. The base64url-no-pad encoder is hand-rolled (no `base64` dep yet on main); when the crypto-primitives branch lands it can fold into the shared encoder without changing rand-bytes semantics.
add rand-bytes for cryptographically random tokens
Four new text-in/text-out builtins for the OAuth, JWT, and webhook flows that dominated the bearer-token, jwt-signer, and webhook-receiver personas. All tree-bridge eligible (pure, no FnRef, no I/O) so VM and Cranelift inherit them at zero opcode cost. - urlenc s > t RFC 3986 percent-encode (total) - urldec s > R t t inverse; Err on bad escape or non-UTF-8 - b64u s > t base64url-encode, no padding (total) - b64u-dec s > R t t inverse; Err on bad input or non-UTF-8 Adds percent-encoding 2 and base64 0.22 as deps. Tags appended at the end of Builtin::ALL to preserve every existing on-wire tag.
urlenc uses NON_ALPHANUMERIC minus the unreserved punctuation (-._~) so the encoded set exactly matches RFC 3986. urldec validates every percent escape up-front before calling decode_utf8 — the crate's decoder is lenient and passes stray % through, which would silently break the R t t contract. b64u uses URL_SAFE_NO_PAD so encode strips padding and decode rejects standard-base64 padding for a strict no-pad round-trip. Tree-bridge propagation handles VM and Cranelift automatically.
16 regression tests covering each builtin's happy path, the canonical edge cases (UTF-8 multibyte, URL-safe alphabet, no-padding contract), and the typed-error contract for each decoder (stray %, short escape, invalid alphabet, standard-b64 padding, non-UTF-8 decoded bytes). Each test fans across every available engine via ENGINES so future tree/VM/ Cranelift divergence is caught. The example file exercises every builtin with -- run/-- out assertions so tests/examples_engines.rs picks it up as a higher-level regression test across every engine too.
Adds the four builtins to the canonical builtin table in SPEC.md, the one-line agent spec in ai.txt, and a new section in skills/ilo/ilo-builtins-text.md. The SPEC section calls out the strict no-pad contract for b64u-dec and the typed-error shape for both decoders so agents reach for ! or the explicit Err arm.
Two CI fixes in one shot. 1) call_function's debug-build stack frame grew with the four new dispatch arms inlined and pushed the fib(10) recursion test past the 2 MiB thread stack on Linux. Lift each arm into its own #[inline(never)] helper so the dispatch frame stays small and recursive tree-walks keep headroom. 2) The expanded ilo-builtins-text SKILL section pushed the module past the 1000-token cl100k cap. Compress to a single-line summary; the full builtin contracts live in SPEC.md / ai.txt where the cap is higher.
Replace every uses: foo/bar@vN reference with uses: foo/bar@<40-char-sha> # vN in the three workflow files. This prevents a malicious tag overwrite on any action (actions/checkout, dtolnay/rust-toolchain, codecov/codecov-action, gitleaks/gitleaks-action, etc.) from silently injecting code into our CI. Tag comments preserved so reviewers and dependabot can still see the intended major version at a glance.
ci: pin GitHub Actions to commit SHAs (#5br)
feat(builtins): URL + base64url encoding cluster (urlenc/urldec/b64u/b64u-dec)
The skills doc described fmt2 as a list-splat variant of fmt, which is wrong. fmt2 is a decimal formatter: fmt2 x:n digits:n > t. The mis-description was hit by the form-urlencoded-client persona on 2026-05-21, who reached for fmt2 expecting a list splat. Replaces the line with the correct signature, the canonical 3.14159 -> 3.14 example, and the compose-with-fmt idiom from SPEC.md. Refs pending.md #5bk.
Adds tree/vm/cranelift regression for the doc-canonical shape
`fmt "x={}" (fmt2 v 2)` documented in SPEC.md and now in the
skills text-builtins doc. Calls the entry with arg 3.14159 and pins
output to x=3.14 on each engine, so the doc example stays honest.
docs: surface pst!/get! HTTP auto-unwrap forms (#5bm)
Six new builtin rows in the SPEC.md builtins table plus a dedicated "Crypto primitives" section explaining the intended call site (HMAC signature verification with ct-eq, SHA-256 fingerprinting, the b64 vs b64u distinction, why ct-eq instead of =). ai.txt is auto-regenerated from SPEC.md by build.rs, picked up here. skills/ilo/ilo-builtins-text.md gets a one-line Crypto section so agent-facing skill content stays in lockstep — the line covers all six builtins with signatures plus the key safety note (never = on secrets, always ct-eq).
feature: crypto primitives - sha256, hmac-sha256, base64, hex, ct-eq (#5ag)
PR #503 (seed builtin) assigned OP_SEED = 189 and PR #518 (VM tail-call optimisation) later assigned OP_TAILCALL = 189. In the dispatch match OP_SEED is listed first and silently shadows OP_TAILCALL, so VM tail-call dispatch has been broken on main since the TCO landed. Clippy also promotes the resulting unreachable-pattern warning to an error, blocking every PR's lint check. Renumber OP_SEED to 190 (next free byte). OP_TAILCALL keeps 189 since it's on the hot dispatch path and has been wired up longer. No other references to OP_SEED need updating - the constant is named, not spelled-out, in the compile/jit/cranelift sites.
fix(vm): resolve OP_SEED/OP_TAILCALL opcode collision
Watchdog thread fires ILO-R016 when wall-clock exceeds --max-runtime (default 60 s); record_output charges every print and fires ILO-R017 when stdout exceeds --max-output-bytes (default ~100 MB). Both write a structured diagnostic to stderr and exit 1. Caps are off until install() is called, so library use is unaffected. Origin: mandelbrot persona ran an infinite loop and wrote 165 MB of stdout before the harness intervened (2026-05-20).
Adds the two global flags to the Global args struct and installs the guard from both the Cmd::Run arm and the bare-positional dispatch. Both ultimately execute user code and so both need the watchdog armed before the program enters its first instruction.
record_output is called from the Builtin::Prnt path in the tree
interpreter, OP_PRT in the VM, and jit_prt in the Cranelift JIT.
Each charges the formatted value's byte length plus 1 for the
trailing newline, so a runaway wh true{prnt 0} loop trips the
budget honestly regardless of engine selection.
Subprocess tests exercise the VM and JIT paths for both ILO-R016 (infinite-loop runtime abort) and ILO-R017 (stdout overflow abort) plus the happy path (well-behaved program unaffected) and --max-runtime 0 (disable). Adds examples/runtime-guard.ilo as a documentation example the engine harness exercises across every engine.
ilo run: cap runtime + stdout to prevent runaway programs
The runtime caps blurb added in #563 pushed ilo-agent.md from 933 to 1056 tokens, over the per-module 1000 cap enforced by scripts/check-skill-tokens.py. Tighten the wording back inside the limit without losing the diagnostic codes or the cause hint.
doc: trim ilo-agent runtime caps to fit 1000-token budget
Triple-quoted ("""...""") strings desugar to the existing single-
quoted literal in the lexer, so logos' string regex consumes them and
every downstream stage (parser, interpolation, escape decoding) stays
unchanged. A scanning pass finds the closing """ and emits a
synthesised single-quoted form with raw newlines preserved.
When the closing """ sits on its own line, strip_triple_indent drops
the leading newline and removes the common leading whitespace from each
content line, matching Python PEP 257 and Rust's indoc! macro. The
terminating newline of the last content line is preserved. Inline form
(closing on a content line) keeps the body verbatim with no dedent.
Span attribution maps every emitted byte back to its original source
byte so diagnostics still point at the right location.
Nine regression tests pin behaviour across the VM and Cranelift JIT:
single-line form, inline multi-line, dedented multi-line, content-byte
verification, escape decoding, {name} interpolation (single and multi-
line), empty body, and embedded single quote. A backend drift can't
silently re-break the surface.
Adds examples/triple-quoted-strings.ilo so the examples_engines harness
exercises the feature on every engine and so agents reading the
examples directory see the canonical shape in context.
SPEC.md gains a Triple-quoted strings subsection under String Literals covering raw newlines, dedent rules, escape passthrough, interpolation parity, and the embedded-single-quote edge case. ai.txt gets the token-minimal agent-spec entry inline next to the existing escape table.
lex: triple-quoted string literals with PEP-257 dedent
`get` / `pst` return `R t t`, body only. That blocks every workflow that needs response metadata: conditional requests (304 Not Modified branching), redirect following, pagination Link headers, rate-limit headers, cookie capture, status-code branching beyond Ok/Err. This adds `getx` / `pstx`: rich-response variants that surface status, headers, and body as a Map[Text, _] in the Ok arm. Existing `get` / `pst` shapes stay untouched, so token-cheap GETs keep their footprint and there's zero migration cost. getx url > R (M t _) t getx url headers > R (M t _) t pstx url body > R (M t _) t pstx url body hdrs > R (M t _) t Ok-map carries three keys: `status` (n), `headers` (M t t, lowercased), `body` (t). Non-2xx responses surface as Ok with the status on the map; only transport failure (DNS, connection refused, timeout) returns Err. Tree-bridge dispatched (returns Result, no FnRef args), so VM and Cranelift JIT inherit identical semantics without dedicated opcodes. Appended last to `Builtin::ALL` to preserve every existing on-wire tag, and added to `tree_bridge_returns_result` so the bang forms (`getx!`, `pstx!`) auto-unwrap correctly.
src/graph.rs::test_subgraph_type_inclusion_via_dep used getx as a user fn name. Renamed to getp so the test keeps exercising the same code path now that getx is reserved.
Seven new wiremock-backed integration tests in tests/http.rs covering status/headers/body extraction, the 304-stays-Ok invariant, both arg-count forms, transport-failure Err, and two verify-time arg-type rejection cases. All run on --vm and --jit (cranelift feature). examples/http-rich-response.ilo for the examples_engines harness covers the bang-form Err propagation across the 1- / 2- / 3-arg variants.
Surfaces the in-tree -- run: / -- out: / -- err: annotation format as a user-facing command. The same format that tests/examples_engines.rs already exercises, exposed so end-user programs and test suites can assert behaviour from the same files agents read as in-context examples. ilo test <file> runs one file; ilo test <dir> walks .ilo files recursively. Each case spawns the current ilo binary with the chosen engine flag (defaults to --vm; --engine jit / --engine all widen the matrix), asserts stdout (-- out:) or stderr (-- err:) against the expected payload, and prints PASS / FAIL with the source line. Final line is N passed, M failed; exit 0 on all-pass, 1 otherwise. The runner lives in src/cli/test_runner.rs, dispatched from main on Cmd::Test. Path defaults to examples/ when omitted so ilo test in a fresh checkout does something useful out of the box.
Seven subprocess tests pin the dispatch path: passing case, failing case (exit 1 + FAIL line), -- err: assertion shape, directory recursion, --engine all running both engines, missing-path error, and no-annotations-found error. Each spawns the ilo binary so the test exercises real CLI parsing + the runner's own subprocess fan-out, two layers deep.
SPEC.md gains the CLI invocation line in the inventory plus a full **ilo test** paragraph next to ilo check, covering engine selection, the -- engine-skip: passthrough, and the all-pass / any-fail exit codes. ai.txt gets the matching agent-spec entry inline. The skill's ilo-agent.md gets a Testing section with the three canonical invocations so an agent writing tests for its own programs sees the shape without round-tripping to SPEC.
Four rows in the SPEC builtins table, paragraph + example in the HTTP section, regenerated ai.txt via build.rs, plus a 'when to reach for getx vs get' paragraph in skills/ilo/ilo-builtins-io.md. Site companion change pushed separately to ilo-lang/site.
feat(http): getx/pstx with status + headers + body Ok-map (#5bn) — fresh reimpl
feat: ilo test subcommand for -- run: / -- out: assertions
Collaborator
Author
|
Replaced by #NEW — base/head swap forced a real 3-way merge that couldn't happen with head=main. |
5 tasks
danieljohnmorris
added a commit
that referenced
this pull request
May 21, 2026
chore: merge main into next (resolve #567 conflicts, 302 catch-up)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bring next current with main. Conflicts need resolution: .ilo→.@ rename + ai.txt regenerations + dispatch-table builtin additions.