Skip to content

v0.3: data-flow threading, DBOS adapter, MCP trigger server, PyPI packaging - #1

Merged
trevhud merged 24 commits into
mainfrom
integrate/v0.3
Jul 3, 2026
Merged

v0.3: data-flow threading, DBOS adapter, MCP trigger server, PyPI packaging#1
trevhud merged 24 commits into
mainfrom
integrate/v0.3

Conversation

@trevhud

@trevhud trevhud commented Jul 3, 2026

Copy link
Copy Markdown
Owner

Four parallel work streams merged and gated together.

Data-flow threading — nodes declare inputs: bindings (grammar in rote.ir.parse_input_ref); all three adapters thread real payloads through the DAG, each verified against its real runtime. Pipeline-level typed input_schema added to PipelineInput.

DBOS adapter — the "no workflow runner" target: durable execution as an MIT Python library over Postgres/SQLite. Third adapter, zero IR changes. E2E runs the emitted BDR pipeline on a real DBOS runtime: parallel waves, durable HITL park/resume via recv/send.

rote register + rote serve — one MCP server exposing every graduated pipeline as a tool (FastMCP 3.x provider sourced from a registry manifest, live tools/list_changed, Temporal + Cloudflare trigger backends). Real-stdio and real-Temporal tests.

Packaging — distribution renamed rote-cli (PyPI rote is taken; import/CLI stay rote), wheel-installable with the graduator skill bundled, tag-driven Trusted Publishing workflow, releasing docs, README repositioned (Compiled AI citation, when-not-to-use note).

Gate: 227 fast + 5 slow tests, mypy strict, ruff, sanity-check, uv build + twine check — all clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48

trevhud and others added 24 commits July 2, 2026 19:26
The Temporal and Cloudflare adapters each carried their own copies of
_execution_waves (Cloudflare imported it from temporal.py, creating a
false dependency between sibling adapters), _pipeline_hash, and the
case-conversion helpers. These are runtime-agnostic — they operate on
the IR and plain strings only — so they now live in _common.py where a
third adapter can reuse them without reaching into a sibling.

ir_duration_to_human is the generalized name for what Cloudflare called
_ir_duration_to_cf ('5m' -> '5 minutes'); the human-readable form is not
Cloudflare-specific. The adapter keeps its local alias so its call sites
and tests are unchanged.

Pure refactor: no behavior change, no emitted-code change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
Two additions that give the IR a real data-flow story:

1. PipelineInput.input_schema — optional JSON Schema for the pipeline's
   input payload. The typed schema already existed redundantly inside
   entry nodes' signature_spec.input_schema $defs; promoting it to the
   pipeline level lets adapters validate input before the workflow
   starts. Optional so existing pipeline.yaml files stay valid.

2. Node.inputs — parameter name → source reference, with a deliberately
   tiny grammar (parse_input_ref is the single source of truth):
   'pipeline.input', 'pipeline.input.<field>', '<node_id>.output',
   '<node_id>.output.<field>'. No deep paths, no expressions —
   anything fancier belongs in an extracted pure_function node.
   _validate_dag_integrity rejects bad syntax, unknown node ids,
   self-references, and pipeline fields outside the declared contract.

The BDR baseline now carries the promoted CampaignBrief schema and
bindings for both entry nodes plus the full downstream chain (review
gate → hubspot → exclusion checks → personalization → report).
dnc_list_id and vetted_count are deliberately unbound: one is
deployment config, the other an aggregate the grammar refuses to
express — both are documented inline in the yaml.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
ruff check and mypy --strict both failed on main, which matters now
that the tag-driven release workflow gates on them:

- examples/ excluded from ruff (machine-emitted regression snapshots;
  hand-linting them diverges them from real tool output)
- NodeKind: (str, Enum) -> StrEnum; all runtime uses go through .value
- anthropic_api driver: narrow content blocks via block.type literal
  comparisons (mypy discriminated-union narrowing) and type the
  message/tool payloads with the SDK's param types under TYPE_CHECKING
  (keeps the anthropic dep optional); the SDK added content-block
  variants that broke the old getattr-based dispatch under strict mypy
- adapters registry: dict[str, Callable[[], Adapter]] instead of the
  invalid bare-callable annotation + ignore
- temporal.py template lines split E501-clean; emitted output verified
  byte-identical against examples/bdr-outreach/expected/runtimes/
- ruff format across src and tests

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
pyproject.toml, per current packaging guidance (PEP 639, hatchling
>=1.27): SPDX license expression + license-files, drop the deprecated
License:: classifier, single-source the version from rote/__init__.py,
add a Documentation URL. The graduator skill bundle is force-included
into the wheel at rote/skills/rote-graduate — previously a wheel
install couldn't run rote graduate at all because the bundle only
existed in the repo checkout. _default_graduator_skill_dir() now
resolves the packaged copy first, repo layout second.

NOTE: the name 'rote' is taken on PyPI by an unrelated package
(active, May 2026). rote-cli was available as of 2026-07. The name
decision is left to the maintainer; see docs/releasing.md.

.github/workflows/release.yml: tag-driven (v*) release via PyPI
Trusted Publishing (OIDC, no token secrets). Gates: fast test suite,
ruff, strict mypy, sanity-check.sh, tag==version check, twine check,
and a clean-venv wheel smoke test that asserts the skill bundle
resolves from inside the wheel. Build and publish are separate jobs
per pypa/gh-action-pypi-publish guidance; attestations are default-on.

docs/releasing.md documents the one-time PyPI pending-publisher and
GitHub environment setup plus the release procedure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
Adds a 'Why compile agents?' section citing Trooskens et al.
(arXiv:2604.05150) with the headline numbers verified against the
paper (57x tokens at 1k transactions, 450x P50 latency, 100% vs 95%
reproducibility at temp 0, ~40x TCO at 1M/month — all from the BFCL
benchmark, caveated as such). Draws the durability-by-wrapping vs
determinism-by-compilation line while framing Temporal/Cloudflare as
compile targets, and adds a when-not-to-use-rote note so exploratory
agent work isn't oversold. Status table now points at the release
pipeline docs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
Every emitted activity call used to receive {} — the emitted workflow
orchestrated correctly but carried no data (README roadmap #5). Now the
workflow binds its entry argument to pipeline_input and each node's
result to <node_id>_result, and renders each activity's payload from
the node's IR inputs bindings:

    pipeline.input        -> pipeline_input
    pipeline.input.f      -> pipeline_input["f"]
    foo.output            -> foo_result
    foo.output.f          -> foo_result["f"]

HITL gate results participate too (the gate binds its signal payload to
<gate_id>_result), so an approval payload flows straight into the next
activity. Nodes without inputs keep the empty payload for back-compat.

check_input_refs_available (shared in _common.py) rejects, at emit
time, references to nodes that run in a later wave or to loop-body
sub-nodes — those would otherwise surface as NameErrors inside a
running workflow.

The e2e test now starts the workflow with a complete campaign brief and
asserts payload delivery empirically: mocks capture what they receive,
and the test verifies the brief reached the entry node, the review
gate's signal payload reached hubspot_upsert, and the report node
fanned in fields from three different sources.

Also fixes the ADAPTERS registry annotation (dict[str, Callable[[],
Adapter]] instead of a type-ignored bare callable) — found by mypy
while validating this change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
Graduating a skill produced a deployed workflow, but triggering it
still meant leaving Claude for a terminal or dashboard. `rote serve`
closes that loop: one FastMCP (3.4+) server exposes every graduated
pipeline as a callable MCP tool, so the deterministic workflow is
reachable from the same place the fuzzy skill used to run.

Design choices worth recording:

- The contract between `rote register` and `rote serve` is a manifest
  file (~/.rote/registry.json) — same filesystem-contract philosophy
  as the driver layer. Entries store the tool name, description,
  inputSchema, and runtime trigger config (Temporal address/queue/
  versioned workflow type, or Cloudflare worker URL).
- Tools are sourced through a custom FastMCP v3 Provider that re-reads
  the registry per request, so registering while serving needs no
  restart. FastMCP 3.4 never emits notifications/tools/list_changed
  for provider-sourced changes, so sessions are tracked via middleware
  and a file watcher pushes the notification through the low-level
  SDK's send_tool_list_changed() — Claude Code refreshes live;
  Desktop/claude.ai need a reconnect (documented in docs/mcp-trigger.md).
- Trigger tools return {workflow_id, status: "started"} immediately
  with a <tool>_status companion for polling, instead of MCP Tasks
  (which FastMCP 3.4 supports server-side): the extension is still a
  spec RC, Claude clients don't request task augmentation yet, and a
  task handle would tie multi-day workflow observability to this
  process's lifetime when the durability lives in Temporal/Cloudflare.
- inputSchema prefers a structured PipelineInput.input_schema when the
  loaded model has one (arriving concurrently) and synthesizes a
  permissive schema from the untyped required/optional lists until then.

Validated empirically: in-memory FastMCP client (list/call/mid-session
list_changed push), the Temporal backend against a real time-skipping
test server, and a slow test driving a `rote serve` subprocess over
real stdio with the Cloudflare backend POSTing to a local http.server
stub. 164 tests (160 fast + 4 slow); mypy strict clean on the new
modules; sanity-check clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
Third runtime target after Temporal and Cloudflare, chosen to pressure-
test the IR against a library-only durability model (Postgres/SQLite
checkpointing, no orchestrator process). Verified against dbos 2.26.0.

Design choices, each validated empirically against a real DBOS runtime:
- Parallel waves fan out via Queue.enqueue -> handle.get_result(),
  DBOS's documented concurrency primitive, keeping the emitted workflow
  synchronous and its checkpoint ordering deterministic.
- HITL gates map the IR signal name onto DBOS.recv topics (the official
  human-in-the-loop pattern); timeouts convert to seconds at emit time
  and raise TimeoutError -- silence is not approval.
- RetryPolicy maps max -> max_attempts (+1 for the initial attempt) and
  backoff -> backoff_rate (exponential 2.0, linear/constant 1.0);
  retry_on survives as guidance comments since DBOS's should_retry
  takes a predicate, not categories.
- llm_judge nodes with signature_spec get generated Pydantic models
  (JSON-Schema-to-Pydantic converter, the Python analog of the
  Cloudflare adapter's Zod emission) plus direct Anthropic/OpenAI
  structured-output calls; the legacy signature path falls back to
  importing the user's module like Temporal.

The e2e test (slow-marked) runs the emitted BDR app on a real DBOS
runtime over SQLite -- no Docker needed, SQLite is DBOS's supported
local-dev system DB -- and proves wave ordering, durable parking at
both HITL gates (exact step-set stability + PENDING status), resume via
DBOS.send, payload flow into the exit-node result, and two DBOS.recv
checkpoints in the system database's step log.

Also fixes the pre-existing dict[str, callable] annotation in the
adapter registry, which mypy strict rejects.

No IR changes were needed -- third adapter, zero schema pressure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
Mirror of the Temporal threading change for the TypeScript target. The
workflow binds instance params once (const pipelineInput =
event.payload;) and renders each step's payload object from the node's
IR inputs bindings. HITL gate event payloads participate via the
existing <gate_id>_result binding, so an approval event flows straight
into the next step.

TypeScript-specific findings, both verified empirically via the tsc e2e
test (this is why that test exists):

- step.do's generic constraint is T extends Rpc.Serializable<T>, and
  Record<string, unknown> does NOT satisfy it ('unknown' values are not
  structurally serializable). Annotating stubs with it breaks overload
  resolution at every call site; leaving the return inferred yields
  Promise<void>, and 'void as Record<...>' fails TS2352. Stubs now
  declare Promise<never> — honest for an always-throwing stub, and
  never both satisfies the constraint and stays castable.

- Node-output field access is emitted as
  (foo_result as Record<string, unknown>)["field"] so the workflow
  compiles today (against never-typed stubs) and keeps compiling
  against whatever concrete return type the user gives a stub later.

The wrangler-dev e2e now starts instances with a complete campaign
brief and uses echo mocks (each step returns the payload it received),
asserting inside the real Cloudflare Workflows runtime that the brief
reached both entry nodes, the review gate's event payload reached
hubspot_upsert, and whole-output + field bindings resolved correctly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
The graduator agent reads ir-schema.md, not ir.py, so the new IR
fields need rubric coverage or no future graduation will emit them:

- input.input_schema: promote the entry payload's JSON Schema (which
  the agent already produces inside the entry nodes' signature_spec)
  to the pipeline level.
- inputs: a dedicated 'Data-flow bindings' section with the four-form
  reference grammar, BDR examples for every form, and the edge cases
  we hit hand-drafting the baseline (unbound deployment config,
  aggregates the grammar refuses to express, loop-body sub-nodes,
  fan_out, HITL gates).

Also: SKILL.md Phase 5 checklist mentions both fields; README roadmap
item 5 marked done (with the fan_out per-element dispatch follow-up);
CLAUDE.md gains the step.do Rpc.Serializable gotcha and updated test
counts (170 = 167 fast + 3 slow).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
The dbos branch imported shared helpers from temporal.py with a
migration marker; the data-flow-threading branch landed _common.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
# Conflicts:
#	CLAUDE.md
#	pyproject.toml
# Conflicts:
#	README.md
#	src/rote/adapters/temporal.py
#	src/rote/cli.py
#	src/rote/ir.py
The DBOS adapter was built concurrently with data-flow threading and
still passed {} to every step. Bring it to parity so all three
adapters honor the IR's inputs: bindings:

- The emitted @DBOS.workflow now takes the pipeline input as
  pipeline_input and builds each step's payload from the node's
  inputs: mapping, reusing the Temporal adapter's reference-to-
  expression rendering (pipeline.input[.field] / <node>.output[.field]
  -> pipeline_input / <node>_result locals). Payloads land both in
  direct step calls and queue.enqueue fan-out waves.
- HITL gate resume payloads (DBOS.recv return values) already bound
  <id>_result, so gate approvals now flow downstream exactly like the
  other runtimes (contact_review_gate.output.approved_contacts ->
  hubspot_upsert).
- Forward references are rejected at emit time via the shared
  check_input_refs_available guard instead of failing as a NameError
  inside a running workflow.
- Re-emitted the committed BDR baseline; the "TODO: pass real payload"
  placeholders are gone.
- test_dbos_adapter.py mirrors the Temporal threading emission tests
  (incl. an AST check that every dispatched payload dict matches the
  node's declared bindings); test_dbos_e2e.py now records the payload
  each mocked step receives and proves, on a real DBOS runtime over
  SQLite, that the brief reaches the entry nodes and the gate approval
  payload flows through to downstream steps.

232 tests (227 fast + 5 slow), ruff/mypy/sanity clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
PyPI's rote is an unrelated memoization library (May 2026, active,
not reclaimable). Distribution and import names don't have to match,
and uvx rote-cli / pip install rote-cli is the documented install.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
A graduated skill pushed to a public repo should be one click from
running. The adapter now emits a README.md carrying the Deploy to
Cloudflare button (markdown image link per
developers.cloudflare.com/workers/platform/deploy-buttons/, with a
REPLACE-WITH-YOUR-REPO-URL placeholder since the repo URL is unknowable
at emission time) plus a quickstart covering the fetch trigger,
`wrangler workflows trigger`, per-gate `send-event` commands, and the
`rote register`/`rote serve` MCP path — mirroring the DBOS adapter's
README shape.

Also emits `.dev.vars.example` (dotenv format) because the deploy-button
flow reads it to prompt the deployer for secrets; wrangler.jsonc already
carried every field the button requires (name, main, compatibility_date,
binding defaults), so no config change was needed.

The MCP-free invariant test scans only .ts files, so the README's
documentation-level MCP mentions stay out of its scope by construction.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
Claude Code is the primary distribution front door: a user inside
Claude should be able to graduate a skill without touching Python
tooling. This adds the marketplace manifest at .claude-plugin/
(marketplace name `rote`, plugin sourced from ./plugin) and a plugin
with two skills:

- graduate: confirms the source skill dir, asks the runtime target
  (dbos/cloudflare/temporal with one-line tradeoffs), runs the CLI in
  the background via `uvx --from rote-cli rote graduate` (git source
  as pre-release fallback), sets the ~13 min / 30-40 turn /
  subscription-auth expectations, then summarizes the emitted IR by
  node kind.
- serve: walks `rote register` + `claude mcp add --scope user rote --
  uvx --from 'rote-cli[serve]' rote serve`, including the Claude
  Desktop/claude.ai reconnect caveat.

The invocation is `uvx --from rote-cli rote ...` everywhere because
the distribution is rote-cli (PyPI `rote` is an unrelated memoization
library) while the executable stays `rote` — uvx can't find a
`rote-cli` executable in the published wheel. README gains a "Use
from Claude Code" quickstart section documenting both paths.

Verified: `claude plugin validate` passes on both manifests;
marketplace add + install + skill discovery succeed against a scratch
CLAUDE_CONFIG_DIR; `uvx --from rote-cli rote --version` (PyPI) and
`uvx --from git+https://github.com/trevhud/rote@integrate/v0.3 rote
--version` both run; `rote serve` from the [serve] extra answers an
MCP initialize over stdio.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
…dent, full-content pipeline hash

From the silent-failure and CodeRabbit review passes on PR #1:

- serve: registry digest commits only after a successful parse, and the
  watcher survives corrupt files / transient FS errors instead of dying
  silently (regression-tested with a corrupt-then-recover scenario)
- serve: watcher task is drained on shutdown so its exceptions surface
- backends: Cloudflare trigger validates the emitted worker's {id,...}
  contract instead of fabricating success; non-JSON responses and
  Temporal start/describe failures wrap in contextual BackendError
- adapters: emitted prompt interpolation (DBOS + Cloudflare) raises on
  unresolvable placeholders instead of silently inserting ""
- dbos: README template dedents before interpolating multi-line gate
  rows — with 2+ HITL gates the whole README shipped indented
- _common: pipeline hash covers full validated contents, not just
  name/version/counts, so rewires get a new workflow type
- registry: tool-name sanitizer can't emit a leading separator; CLI
  register reports a corrupt registry cleanly

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
Real 57-turn Sonnet run against the updated rubric. The agent emitted
a typed input_schema (8/12 properties typed), inputs: bindings on
21/22 nodes, and all five node kinds; the resulting IR validates and
emits cleanly through all three adapters.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJdgd3mj2jk6dKCLUjdS48
@trevhud
trevhud marked this pull request as ready for review July 3, 2026 14:43
@trevhud
trevhud merged commit a50e346 into main Jul 3, 2026
@trevhud
trevhud deleted the integrate/v0.3 branch July 3, 2026 15:13
trevhud added a commit that referenced this pull request Jul 30, 2026
A `fan_out` node means "invoke once per element of an upstream list".
Only DBOS did that; python, temporal, cloudflare, dbos-ts and inngest
passed the whole list in a single invocation.

That was not a missing feature so much as a runtime leaking into node
semantics. The same pipeline.yaml produced two incompatible contracts
for the code a user fills in: BDR's `personalize_email` judge declares
`contact` as one contact and its Zod/Pydantic schema says so, but five
of six runtimes handed it the entire array. Fill in the stub against
DBOS, re-target Temporal, and it breaks — which invariant #1 (the IR is
runtime-agnostic) exists to prevent.

Which input is the fanned list now resolves in the language-neutral
`_common.fan_out_element_param`, not in a Python-only helper. Two
adapters disagreeing about that would make one pipeline mean two
different things, so it cannot live in a per-language module. Precedence
is unchanged (fan_out edge marker > incoming-edge source > only
node-bound param) and ambiguity is still an emit-time error, never a
guess.

Per runtime: DBOS enqueues a durable step per element (unchanged);
python uses pool.map; temporal gathers one activity execution each;
cloudflare and inngest Promise.all over per-element steps; dbos-ts
allSettled + unwrap. All preserve input order.

Two traps this surfaced, both documented in CLAUDE.md:

- Cloudflare and Inngest key a durable step by NAME, so element steps
  are index-suffixed. A constant name makes Cloudflare serve element 0's
  cached result for all N — the run succeeds, the output has the right
  length, and every entry is identical. DBOS is exempt: it identifies a
  step by execution order.
- A parkable (MCP-bound) fan_out node on Cloudflare runs its elements
  sequentially, same reason a parkable step leaves its parallel wave —
  waitForEvent inside a promise combinator is undocumented and its
  timeout throws, which would reject every sibling.

Also collapses temporal's three dispatch shapes onto one
`_execute_activity_expr`. The wave branch had previously been a separate
copy that silently omitted retry_policy; one renderer means a field
cannot be added to one shape and forgotten in the others.

Emitted code routes the fanned list through a guard (`fanOutList` /
`_fan_out_list`) naming the node and the IR reference. Found by running
it: the first live fan_out run died with "Cannot read properties of
undefined (reading 'map')" — no node, no reference, in generated code
the user never wrote. Python's "'NoneType' object is not iterable" is no
better, so both languages get it; diagnostics are part of the emitted
contract and a guard on one runtime only is its own parity gap.

Testing. tests/test_fan_out_parity.py asserts the contract across all
six, and is built to fail against batch dispatch rather than merely pass
against the new output: each check requires the element param to bind
the loop variable AND the whole-list expression to be absent. Verified
empirically — against origin/main the five broken runtimes fail every
behavioral assertion while every dbos parametrization passes.

The live e2e mocks were themselves masking this. Three TS suites had
`exclusion_check_sequence` returning `passed: []`, so BDR's fan was a
correct no-op that no assertion caught. Mock values now derive from the
IR via `_helpers.fan_out_source_keys`, and the suites assert the fan ran
once per element with each invocation receiving exactly one. Inngest's
memoization check is now fan-aware (== FAN_OUT_ELEMENTS for fan_out
nodes, == 1 otherwise), which is what catches a step-name collapse.

1159 fast + 27 slow pass; ruff, strict mypy, sanity-check clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SBuSDL7Svm7iDzgmhTpDSW
trevhud added a commit that referenced this pull request Jul 30, 2026
* Dispatch fan_out nodes per element on all six runtimes

A `fan_out` node means "invoke once per element of an upstream list".
Only DBOS did that; python, temporal, cloudflare, dbos-ts and inngest
passed the whole list in a single invocation.

That was not a missing feature so much as a runtime leaking into node
semantics. The same pipeline.yaml produced two incompatible contracts
for the code a user fills in: BDR's `personalize_email` judge declares
`contact` as one contact and its Zod/Pydantic schema says so, but five
of six runtimes handed it the entire array. Fill in the stub against
DBOS, re-target Temporal, and it breaks — which invariant #1 (the IR is
runtime-agnostic) exists to prevent.

Which input is the fanned list now resolves in the language-neutral
`_common.fan_out_element_param`, not in a Python-only helper. Two
adapters disagreeing about that would make one pipeline mean two
different things, so it cannot live in a per-language module. Precedence
is unchanged (fan_out edge marker > incoming-edge source > only
node-bound param) and ambiguity is still an emit-time error, never a
guess.

Per runtime: DBOS enqueues a durable step per element (unchanged);
python uses pool.map; temporal gathers one activity execution each;
cloudflare and inngest Promise.all over per-element steps; dbos-ts
allSettled + unwrap. All preserve input order.

Two traps this surfaced, both documented in CLAUDE.md:

- Cloudflare and Inngest key a durable step by NAME, so element steps
  are index-suffixed. A constant name makes Cloudflare serve element 0's
  cached result for all N — the run succeeds, the output has the right
  length, and every entry is identical. DBOS is exempt: it identifies a
  step by execution order.
- A parkable (MCP-bound) fan_out node on Cloudflare runs its elements
  sequentially, same reason a parkable step leaves its parallel wave —
  waitForEvent inside a promise combinator is undocumented and its
  timeout throws, which would reject every sibling.

Also collapses temporal's three dispatch shapes onto one
`_execute_activity_expr`. The wave branch had previously been a separate
copy that silently omitted retry_policy; one renderer means a field
cannot be added to one shape and forgotten in the others.

Emitted code routes the fanned list through a guard (`fanOutList` /
`_fan_out_list`) naming the node and the IR reference. Found by running
it: the first live fan_out run died with "Cannot read properties of
undefined (reading 'map')" — no node, no reference, in generated code
the user never wrote. Python's "'NoneType' object is not iterable" is no
better, so both languages get it; diagnostics are part of the emitted
contract and a guard on one runtime only is its own parity gap.

Testing. tests/test_fan_out_parity.py asserts the contract across all
six, and is built to fail against batch dispatch rather than merely pass
against the new output: each check requires the element param to bind
the loop variable AND the whole-list expression to be absent. Verified
empirically — against origin/main the five broken runtimes fail every
behavioral assertion while every dbos parametrization passes.

The live e2e mocks were themselves masking this. Three TS suites had
`exclusion_check_sequence` returning `passed: []`, so BDR's fan was a
correct no-op that no assertion caught. Mock values now derive from the
IR via `_helpers.fan_out_source_keys`, and the suites assert the fan ran
once per element with each invocation receiving exactly one. Inngest's
memoization check is now fan-aware (== FAN_OUT_ELEMENTS for fan_out
nodes, == 1 otherwise), which is what catches a step-name collapse.

1159 fast + 27 slow pass; ruff, strict mypy, sanity-check clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SBuSDL7Svm7iDzgmhTpDSW

* Close three test gaps found by mutation-testing the adapters

Follow-up audit to the fan_out work. The recurring failure mode in this
repo is a test that observes something *adjacent* to the behavior, so
this measures the suite instead of trusting it: apply a semantic
regression to an adapter, run the suite, and anything that still passes
is an untested behavior.

18 mutations (retry policy, timeouts, step naming, payload threading,
MCP allowlist narrowing, the ANTHROPIC_API_KEY scrub, invariants #4 and
#7); 15 caught, 3 survived.

**temporal: per-node `timeout:` was ignored and nothing noticed.**
Making `_activity_timeout` return the default unconditionally left the
whole suite green. A step declaring a 30s budget silently got 5m —
five times too long on a wedged call, with nothing in the emitted code
hinting why.

**cloudflare: same gap on HITL gates**, and worse there, because
`waitForEvent` THROWS on timeout: a gate inheriting the 7d default
instead of its declared 1h turns a fast-fail approval window into a
week-long hang, and the reverse fails a legitimately long wait.

Both new tests carry a negative half — asserting only that the declared
value appears would still pass if the adapter emitted it for every
node, which is exactly how the original gap hid.

Writing the cloudflare one immediately surfaced a real emitted-output
bug: `1h` rendered as `"1 hours"`. Cloudflare's WorkflowDuration accepts
either spelling so nothing broke, but emitted code is a reviewed
artifact and "1 hours" reads as a defect. Now singularized.

**The third survivor is an equivalent mutant, not a gap.** Swapping the
tier order in `fan_out_element_param` cannot change the answer:
fan_edge_sources ⊆ edge_sources, and a tier is accepted only when it
singles out exactly one param, so if the broader tier singles one out it
is necessarily the same one. Commented in place so the next sweep
doesn't chase it. The precedence test added alongside is still real —
verified by a mutation that IS observable (always take the
alphabetically-first param), which it catches.

**Also: the temporal e2e's fan was a no-op.** Its
`exclusion_check_sequence` mock returned `passed: []`, so
personalize_email never ran — the same vacuous-fixture bug just fixed in
the three TS suites, missed because that suite passed and I only chased
failures. Its `CAPTURED_PAYLOADS` dict is keyed by node and so keeps
only the last call, structurally unable to observe a fan at all; an
ordered CAPTURE_LOG now sits alongside it. Verified by mutating the
snapshot the e2e actually imports back to batch dispatch: it fails with
"expected 3 activity executions, got 1".

That last point is worth knowing: the temporal e2e imports the committed
`examples/.../runtimes/temporal/workflow.py` snapshot, not fresh
emission, so mutating the adapter does not reach it. The snapshot is
regenerated by the adapter tests' fixtures (which emit into the repo on
purpose, so `git diff` shows emission changes at review).

Audited two more patterns and found no open gaps: value-taking flags
asserted only by presence (`--output-last-message`,
`--append-system-prompt`) turn out to have their values checked
elsewhere, and the remaining empty-collection fixtures are inert
because nothing iterates them.

CLAUDE.md gains a "Testing discipline" section with the five defects of
this shape, the three habits that catch them, and the instruction to
verify a new test by breaking the code rather than by inspection.

1162 fast + 27 slow pass; ruff, strict mypy, sanity-check clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SBuSDL7Svm7iDzgmhTpDSW

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant