Skip to content

python to rust migration

Jump to bottom
Kadyapam edited this page Jun 25, 2026 · 3 revisions

Python → Rust playbook migration gotchas

The travel SPA's playbooks were first authored against the retired Python NoETL runtime. The production runtime is now the Rust noetl-server-rust + noetl-worker-rust stack, which parses every playbook against a strict typed schema (orchestrate-core/src/playbook.rs) and validates structure before it will execute. Several Python-era shapes the Python engine accepted leniently are rejected by Rust.

This page is the running list of those drifts and the corrected shapes. It exists because each one is a silent trap: the playbook registers fine (registration is permissive), then fails at execute time. Validate every migrated playbook by executing it, not just registering it.

1. Tool kind must be a real Rust ToolKind

The tool.kind value is a closed enum. Accepted kinds: http, postgres, duckdb, ducklake, python, workbook, playbook, playbooks, secrets, iterator, container, script, snowflake, transfer, snowflake_transfer, gcs, gateway, nats, shell, artifact, noop, task_sequence, rhai, subscription, wasm. There is no catch-all — an unknown kind makes the step's tool: block match neither variant of the untagged ToolDefinition enum, producing:

400 Bad Request - {"error":"workflow[N]: data did not match any variant
of untagged enum ToolDefinition at line L column C","status":400}

The line/column point at the offending step's mapping.

kind: agent → kind: playbook (sub-playbook / MCP calls)

The Python agent tool shape is not a Rust kind. Convert each MCP / sub-playbook call:

# Python (rejected by Rust)
tool:
  kind: agent
  framework: noetl
  entrypoint: automation/agents/mcp/firestore
  payload: { method: tools/call, tool: get_doc, arguments: {...} }

# Rust (accepted)
tool:
  kind: playbook
  path: automation/agents/mcp/firestore   # entrypoint → path
  payload: { method: tools/call, tool: get_doc, arguments: {...} }
  • entrypoint: → path: — the value is unchanged; it already equals the child playbook's metadata.path.
  • drop framework: noetl (Python-only).
  • keep payload: — the Rust playbook tool forwards it to the child as workload input (same {method, tool, arguments, …} contract the MCP playbooks read).

Caveat — the playbook tool does not return the child's result. See §5.

2. Every workflow needs a step named start

The Rust runtime's validate_playbook rejects any workflow that has no step literally named start:

422 Unprocessable Entity - {"error":"Workflow must have a step named 'start'","status":422}

The Python engine treated the first declared step as the entry point; Rust requires an explicit start. Add a noop entry step that routes to the real first step:

workflow:
  - step: start
    tool: { kind: noop }
    next:
      spec: { mode: exclusive }
      arcs:
        - step: normalize_input   # the original first step

This validation runs after parse, so a ToolDefinition error (§1) masks it — fix the kinds first, then this surfaces.

3. HTTP / callback request bodies: json: not data:

The Rust http tool reads a JSON request body from json:. The Python-era data: key is silently ignored, so the request goes out with an empty body. This bit the gateway session-validate and authorization playbooks (noetl/ai-meta#133 / #134):

# Python
tool: { kind: http, method: POST, url: "...", data: { foo: "bar" } }
# Rust
tool: { kind: http, method: POST, url: "...", json: { foo: "bar" } }

Same rule for callback payloads posted back into the runtime.

4. Pass inputs through input: — don't reach for ambient context

Python steps could pull values out of an implicit context object (context.get("x")). The Rust runtime renders a step's templates against an explicitly-declared input: block; values a step needs must be bound there (then referenced by name in code:). The same fix replaced context.get(...) with input: + a pick in the auth playbooks. Also watch accessor depth on prior-step results — e.g. .command_0.rows collapsed to .rows during that migration.

5. The playbook tool returns status, not the child's result

The Rust playbook tool (noetl-tools/src/tools/playbook.rs) has two modes, and neither returns the child playbook's result data:

  • async (no return_step): returns {status: "started", execution_id, path, async: true}.
  • blocking (return_step set): polls GET /api/executions/{id}/status and returns the execution status payload (status, current_step, progress, is_cancelled) — still not the child's output.

So a downstream reference like {{ call_google_places }} resolves to a status envelope, not the Google Places results. For a playbook that consumes its MCP children's outputs (itinerary-planner feeds call_google_places / call_duffel_offers / call_amadeus_hotels into normalize_tool_response, and load_slot_state.data into extract_turn), this is a functional blocker, not just a cosmetic one.

This needs a runtime capability that does not exist yet (a synchronous, result-returning sub-playbook invocation). Tracked as gated issue noetl/ai-meta#136. Do not work around it by inlining MCP logic or round-tripping results through Firestore — that changes the architecture.

6. Step-result accessor paths differ (.context.data.)

A python step returns a result dict. The Rust worker wraps it, so a field the step set at the top of result (e.g. result["first_tool"]) is exposed downstream at {{ <step>.context.data.first_tool }}, not {{ <step>.context.first_tool }}. itinerary-planner's routing when: conditions read the shallower path and resolve null, so the tool-dispatch arcs never fire. Audit every {{ <step>.context.<field> }} reference against the actual result envelope. Tracked: noetl/ai-meta#135.

How to validate a migrated playbook

# register (permissive — does NOT catch the drifts above)
noetl --host=localhost --port=8082 catalog register playbooks/<name>.yaml
# execute (strict — this is what surfaces the parse + validation errors)
noetl --host=localhost --port=8082 exec "<catalog/path>" --runtime distributed \
  --set <key>=<value>
# then read the event trace for per-step pass/fail
curl -s "http://localhost:8082/api/executions/<exec_id>" | jq '.events'

Register the child MCP playbooks too — a parent kind: playbook step 404s if the child isn't in the catalog.

Related

Travel SPA

Architecture

Integration

Operations

See also

Clone this wiki locally