spec(#130): add on_result expression: matcher (§5.4)#155
Merged
AlexChesser merged 4 commits intomainfrom Apr 15, 2026
Merged
Conversation
Adds a general-purpose expression matcher to `on_result` that reuses the §12.2 condition grammar verbatim — no new syntax. Unlocks branching on step fields other than `response` and `exit_code` (e.g. `stdout`, `stderr`, cross-step references) without inventing a parallel grammar. Decouples producing confidence signals (#128 native runner, #130 logprobs — deferred) from consuming them: once the grammar gains numeric operators, `expression: "{{ step.id.confidence }} >= 0.75"` works without further spec work on on_result. Scope kept small deliberately: - `expression:` supports §12.2 operators only (==, !=, contains, starts_with, ends_with). Numeric comparisons and boolean combinators are explicitly out of scope for v0.3 and tracked as a joint §12.2 + §5.4 extension. - Unresolved template variables abort the pipeline (matches §11 contract); silent non-match is deliberately not an option. - Named matchers (contains:, exit_code:, field:+equals:) remain the recommended form when they fit. Cross-reference added in §12.2 so the shared grammar is discoverable from both sides.
Extends the shared condition/expression grammar with a `matches` operator so `expression:` (§5.4) is a true superset of the named `matches:` matcher — consistent with how `contains` is available at both sites. Regex semantics chosen for predictability: - Rust `regex` crate syntax (linear-time; no backreferences/lookaround) - Unanchored by default — `matches 'PASS'` fires on "tests PASSED"; users add `^`/`$` for exact matching - Case-sensitive by default (differs from `contains`/`starts_with`) because regex gives users explicit control via `(?i)`; this matches standard regex expectations rather than inventing an asymmetric rule - Invalid regex fails at pipeline load, not at match time - YAML single-quoting recommended to avoid backslash escaping traps The named `matches:` in §5.4 is clarified as shorthand for the expression form, locking down its previously-underspecified case sensitivity before implementation work begins.
Replaces inline `(?i)`-style flags with conventional `/pattern/flags` regex literals, matching JavaScript/Perl/Ruby convention. `/warn|error/i` reads as what it is; `(?i)warn|error` demands the reader know a Rust-regex-specific inline-flag grammar. Design choices: - Flags supported: i (case-insensitive), m (multiline), s (dotall). - g (global) rejected at parse time — matching is boolean, "global" is meaningless. - Unsupported Perl flags (x, etc.) rejected — users can still reach inline (?x) if they really need verbose mode. - Parse rule: first / opens, last / followed by [ims]* closes — so forward slashes inside the pattern rarely need escaping. - Single-quoted YAML strings strongly recommended so \d/\b/\s work as written (no YAML double-escape trap). No change to engine choice (still Rust `regex` crate, linear-time, no backrefs/lookaround) — just the surface syntax.
The regex spec was living inside §12.2's `matches` operator subsection,
which meant any future consumer (template filters, output_schema
patterns, etc.) would either redefine it or link into a subsection
header. Promoting it to §12.3 makes it directly referenceable.
§12.2 `matches` row: thin pointer to §12.3.
§5.4 `matches:` named-matcher row: thin pointer to §12.3.
§12.3: literal form, flags, parsing rule, matching semantics, YAML
quoting — all in one place.
spec/README.md: §12 index entry updated to surface §12.3.
No behavior change — pure reorganisation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a general-purpose expression matcher to
on_resultthat reuses the§12.2 condition grammar verbatim — no new syntax. Unlocks branching on
step fields other than
responseandexit_code(e.g.stdout,stderr, cross-step references) without inventing a parallel grammar.Decouples producing confidence signals (#128 native runner, #130
logprobs — deferred) from consuming them: once the grammar gains numeric
operators,
expression: "{{ step.id.confidence }} >= 0.75"workswithout further spec work on on_result.
Scope kept small deliberately:
expression:supports §12.2 operators only (==, !=, contains,starts_with, ends_with). Numeric comparisons and boolean combinators
are explicitly out of scope for v0.3 and tracked as a joint §12.2 +
§5.4 extension.
contract); silent non-match is deliberately not an option.
recommended form when they fit.
Cross-reference added in §12.2 so the shared grammar is discoverable
from both sides.