feat: opt-in compile-time model annotation (label/risk/param; runtime stays $0)#78
Merged
Merged
Conversation
…s, confirm-don't-trust; runtime stays $0) The reviews' 'use the model at compile time, not just repair time' cheap win. A StepAnnotator Protocol proposes step labels, richer risk classifications, and parameter inferences from a demonstration; the model runs ONCE at compile, OFF by default, behind an interface (fake for tests, lazy Anthropic impl). A proposed risk UPGRADE applies (safe direction); a downgrade or consequential param is FLAGGED needs_operator_confirmation, never silently trusted. The runtime/replayer is untouched — zero model calls at replay. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKrVJJy5jWVCkXAqgUqtqZ
abrichr
added a commit
that referenced
this pull request
Jul 13, 2026
…oops/branches) from multiple demos, reject-if-underdetermined (#81) * feat: workflow-program IR Phase 2 — loops, branches, subflows, exception paths (the state machine) Evolve the compiled artifact from a linear action list into a parameterized STATE MACHINE (RFC docs/design/WORKFLOW_PROGRAM_IR.md §2), closing the review's "a workflow is not a list of actions" gap. Phase 1 (typed params, guards, wait_until) added the pieces; Phase 2 adds the control flow a trajectory cannot carry: LOOPS over a worklist, guarded BRANCHES, reusable SUBFLOWS, and EXCEPTION paths — the program the PBD literature (Rousillon, WebRobot, Skill-DisCo, PROLEX) says a demonstration compiler must express. IR (openadapt_flow/ir.py), additive and backward-compatible: - State (action | branch | loop | subflow_call | terminal) + Transition (guarded edge) form a ProgramGraph; an action state's payload IS a Phase-1 Step (the unchanged hardened leaf), a transition's guard IS a Phase-1 Predicate. - Relation (worklist) + LoopSpec (bounded per-row body subflow); Workflow gains optional program / subflows / data_sources. When program is None the linear steps list runs exactly as today. - lift_to_program: mechanical degenerate lift (RFC §2.6) — a linear bundle is the single-path graph. Interpreter (runtime/replayer.py): a deterministic graph interpreter ($0, zero model calls) that REUSES the linear per-action pipeline unchanged — every action state runs through _run_step, so identity / effect / risk / heal gates fire identically inside loop bodies and branches. Adds guarded transition selection (first match wins, no-match HALTs fail-safe), bounded worklist loops, subflow dispatch, and on_exception routing (graph try/except); unhandled failures and halt/escalate terminals stop the run. Bounded against non-terminating graphs (step budget + nesting depth). Linear path is byte-for- byte unchanged (program=None branch). Tests (tests/test_program_ir_phase2.py, 18): loop runs body 3x / 0x / run-time worklist / bound enforced; branch takes each arm (param + screen predicate) and dead-ends HALT; subflow reused as loop body AND direct call; on_exception catches a failed action and continues; identity- and effect-gates fire inside a loop body; the lifted linear graph replays byte-identically to the linear replayer; program round-trips through save/load. Full non-e2e suite green in isolation (859 passed; the concurrent-agent FileNotFoundError errors are environmental). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKrVJJy5jWVCkXAqgUqtqZ * feat: multi-trace induction — infer a parameterized program (params/loops/branches) from multiple demos, reject-if-underdetermined Implements RFC docs/design/WORKFLOW_PROGRAM_IR.md §3 steps [4]+[5]: the induction loop the PBD lineage (Rousillon, WebRobot, Skill-DisCo, PROLEX) says a demonstration compiler must have. "One demonstration is evidence, not specification." openadapt_flow/compiler/induction.py: - induce_program(traces) aligns multiple demos structurally and infers a Phase-2 ProgramGraph: PARAMS (values that VARY across traces at an aligned position; constant => literal), LOOPS (a repeated body whose count DIFFERS => LoopSpec over an inferred Relation), BRANCHES (a divergent step under a detectable condition => guarded branch, guard proposed/flagged), and OPTIONAL steps (present in some, absent in others, no condition => guarded skip). All deterministic, ZERO model calls. - validate_held_out / reproduction_score: leave-one-out held-out validation (infer from N-1, check reproduction of the held trace). - Reject-rather-than-guess: contradictory / underdetermined traces are QUARANTINED (no program emitted, certified=False) and routed to the disambiguation flow (#74), mirroring the identity gate's posture. - The optional compile-time Proposer (the #78 StepAnnotator fits behind it) only PROPOSES interpretations — flagged, never silently trusted, never flips an underdetermined point to certified. Touch-points kept minimal: reuses the Phase-2 IR + Phase-1 ParamSpec/Guard/ Predicate verbatim (no new IR fields), reuses disambiguation's question model, and the emitted program replays through the EXISTING interpreter unchanged (compile.py untouched; compiler/__init__ re-exports the new API). Tests (tests/test_induction.py, 17 tests): a synthetic MockMed corpus of trace variants covers (a) param, (b) loop, (c) branch/optional, (d) contradiction=> reject; held-out scores a good induction high and an over-specialized one low; underdetermined is flagged not guessed; the induced program round-trips through the real Phase-2 interpreter (faked backend/vision, zero model calls). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CKrVJJy5jWVCkXAqgUqtqZ --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The reviews' explicit 'cheap win' — use a model at COMPILE time (not runtime) to label steps, propose richer risk classifications, and infer parameters. Model runs once at compile, OFF by default, behind a
StepAnnotatorProtocol (fake for tests, lazy Anthropic impl — no key needed unless enabled). Confirm-don't-trust: a proposed risk upgrade applies (safe direction); a downgrade or consequential param inference is flaggedneeds_operator_confirmation(mirrors #74/#75), never silently weakening a safeguard. Runtime/replayer untouched — zero model calls at replay. 84 tests pass. Additive/opt-in — default-off is byte-identical to today.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com