Skip to content

v0.17.0

Choose a tag to compare

@github-actions github-actions released this 27 Oct 16:35
· 40 commits to main since this release
772b117

Added

  • rote setup: the guided first run, and /rote:setup. One resumable
    command chains the steps rote has: import the MCP servers from the client
    you run in (rote mcp import's planner), pick one of your skills (your
    own first, then plugin skills, identical copies once, rote's demo last),
    log in to the servers that skill uses (a browser login for a person,
    rote mcp login --no-wait for an agent; a skill that does not say which
    servers it calls is offered every server that needs a login, for you or
    the agent to pick from), then rote try's compile, install and paired
    eval, and the verdict. A server that marks its tools readOnlyHint needs
    no vouching: the comparison calls only those tools.
    Each step is a plan you confirm and can skip, and a re-run skips what is
    done (read from the registry, the token store, rote try's files, and the
    pick in $XDG_DATA_HOME/rote/setup.json). Under an agent every question
    and consent is one action_required object with ask (what to put to
    the user) and next. --approve <step>=<code> approves the plan the user
    saw, once (a replay or a changed plan asks again), a paid step also needs
    --live, and --skip <step> is the user's no. The
    plugin skill relays it. Answers are remembered in setup.json (with
    --reset to forget them), so next carries only the new answer and never
    a path, and servers the user was offered and passed on are not offered
    again. Bare rote on a machine with nothing set up prints a short banner
    that points to rote setup. scripts/rehearse-setup runs the whole flow
    in a temp home, as an agent and on a pseudo-terminal, with no inference.

  • Judge passthrough: input data a judge returns unchanged is copied in
    code.
    signature_spec.passthrough: {<output field>: <input path>}
    takes those fields out of the model's request schema, and every runtime
    (the six workflow runtimes and the compiled skill's driver, DECIDE
    included) copies them from the validated input before the whole output
    schema is checked, so the declared output is unchanged and the copy
    cannot drift. The IR checks each target is a declared output property
    and each source a bound input path of the same schema. deal-monitor's two
    fanned judges use it: about 134 fewer output tokens per scoring call
    (61%) and 36 per classify call (24%). System-1 offers the same-schema
    candidates and the model picks the ones the skill returns unchanged.

  • rote try [<skill>]: compile one of your skills and see it do the same
    thing.
    It picks a skill (from rote skills list, best ranked first, or
    the one you name), then walks three confirmed steps: compile it to
    --runtime skill (the pinned argv, --local and --decider rules
    included, printed in the plan), install it (rote skill install's plan
    and person check), and run the paired eval (rote eval --run --runtime skill's harness) on the skill's input.json or --input. The result is
    the fields matched per pair, each difference with both values, the
    original against itself as the noise floor, and wall time, turns, tokens
    and cost per side. No same-result claim is made from zero exact fields.
    Each paid step shows its estimate (docs-digest's recorded runs, scaled by
    skill size) and needs a person's yes or --yes --live. The approved
    high end caps the ledger. Re-running the same command resumes: done
    steps are read from <out>/try.json, the apps registry and
    <out>/try-result.json, and cost nothing. Under an agent, each consent
    point returns action_required (exit 2).

  • rote eval <compiled> --run --runtime skill: the paired eval of a
    skill and its compiled skill.
    Each side runs as a claude -p subagent
    under the same isolation flags, on the same input, with Bash loaded on
    both (the original's is denied, and the compiled side may run only rote skill run <name>). The compiled skill is emitted, installed into a throwaway
    state and driven through rote skill run. A gate on either side is
    answered by the same simulated person (rote_gates). Per side it reports
    wall time, all four token buckets, cost, turns, and the context the skill
    left (K = I - T*B, with B from one control run per side). Each pair gets a
    field-by-field compare (rote.proof), with the original against itself as
    the noise floor. A compiled trial fails when it has no finished run folder
    ("bypassed the driver"), when a gate in its journal was not answered by
    the person after an ask_person call, or when the agent reports a value
    the driver's result contradicts. --case <id> runs a bench case (its
    fixtures, gate answers and reference score). --budget USD is required,
    and a shared --ledger stops the eval before a run that would cross it.
    Results append to the eval corpus as kind: skill_ab. A source skill as
    the target is compiled first with a pinned argv (--local, --agent claude, --decider rules, --structure haiku, --tiebreak none).

  • rote.proof, the field compare moved from the paused rote try draft
    (#250): equal / drifts / wording fields, and no same-result claim
    from zero exact fields ("no fields to compare exactly").

  • --runtime skill emits a compiled skill: a skill directory any agent
    client loads.
    rote emit --runtime skill and rote compile --runtime skill write SKILL.md with the original skill's name and description and a
    short body, a README, and scripts/: the run plan, the stdlib driver that
    runs it, the MCP helper when a step calls MCP, and the compile's real
    extracted/ modules, every file hashed in .rote-manifest.json. The name
    and description come from the source SKILL.md (--skill <dir>, or the
    pipeline's source_skill when its provenance.json vouches for it), and
    are refused rather than guessed. Stubs are refused at emit
    (--allow-stubs lets them through), and so are agent loops, until the
    driver's tool proxy lands. --judges helper has rote's inference helper
    answer judges on the IR's model instead of the agent, and is recorded in
    the manifest.

  • rote skill install / run / trust / list / runs. `rote skill install

    ` checks the dir is exactly what rote emitted and records the hash of every file in it, as a confirmed plan. It then offers to save the Claude Code allow rule `Bash(rote skill run *)` in the project's `.claude/settings.local.json`, as a separate step `--yes` never covers (`--save-allow-rule`). `rote skill run ` runs the installed skill's driver only while every file matches that record: a changed, added, missing or symlinked file is refused with an ERROR block naming `rote skill trust `, which shows the change as a diff summary (`--diff` for the diffs) and records it on a yes. The driver runs with rote's interpreter under `-I -B` and an allowlisted environment: MCP endpoints come from the registry only, and the registry, login store and run folders must be the ones the install recorded. `rote skill trust --allow-writes` lists every tool that may write and records the consent for that plan. It needs a person at a terminal, because `--yes` cannot stand in for one.
  • A compile grounds tools on the servers you logged in to. A logged-in
    server whose tool the skill calls (a code span or a call, never a
    product name in prose) grounds the compile, and a logged-in server's
    other tools are offered to the structure pass with their descriptions,
    so a skill that names no server ("use read-only Jira search") can bind a
    step to searchJiraIssuesUsingJql. Only a server the plan binds becomes
    grounding.

  • A skill compile stops when compiling would not pay off. A compiled
    skill stops for the agent at each judge and agent loop, and each stop
    re-reads the conversation. Measured: expense-audit (no stops) ran 3.6x
    faster and 62% cheaper compiled, Atlassian's search-company-knowledge (4
    judges) 4x slower and 3x costlier. A --runtime skill compile with more
    than 2 judges and loops stops right after the structure pass (cents),
    naming them; --compile-anyway overrides it. /rote:compile and
    /rote:setup give the agent the same rule, so it judges a skill before
    anything is spent.

Changed

  • A flagless rote compile emits a compiled skill (--runtime skill),
    logged in or not.
    It was dbos logged out, and rote cloud (cloudflare
    plus auto-deploy) logged in. Measured on claude-sonnet-4-6, original
    skill against compiled skill, with every compared field matching within
    the original's own noise: expense-audit 154 s / $0.34 to 43 s / $0.13,
    ops-report 55 s / $0.13 to 42 s / $0.12, docs-digest 17.9 s / $0.093 to
    16.3 s / $0.049. The summary ends with the next step: `rote skill install

    `, then swapping it in for the original skill, or `rote try ` for the side-by-side proof. rote cloud is now asked for, not assumed: `--cloud`, a cloudflare runtime, or `deploy: rote-cloud` (which makes cloudflare the built-in runtime). Every explicit flag, `ROTE_*` variable and config value resolves as before. `rote config` reports the new built-in, and `rote init` offers local hosting and `skill` first.
  • The rote register / rote serve registry moved under the XDG data
    dir.
    The default is now $XDG_DATA_HOME/rote/serve-registry.json
    (~/.local/share/rote/serve-registry.json), beside the app registry,
    instead of ~/.rote/registry.json, the one per-user file outside the XDG
    dirs. ROTE_SERVE_REGISTRY_PATH still overrides it. While the old file
    exists, rote register and rote serve exit 2 and print the mv
    command that moves it. If both files exist, rote reads only the new one
    and asks you to delete the old one.

  • The base install carries only the MCP client; the workflow runtimes are
    extras installed when you pick one.
    pip install rote-cli now brings
    fastmcp-slim[client] (the client half of fastmcp: Client,
    StreamableHttpTransport, OAuth) and httpx, so an MCP login, compile's
    tool grounding and every compiled MCP step work on a plain install. dbos
    and temporal stay extras, so a user who picks temporal never downloads
    dbos, and rote serve (fastmcp's server half) is the serve extra. The
    base install is 64.3 MB (65 packages) in a clean venv, against 27.6 MB
    without the MCP client and 104.0 MB with dbos and full fastmcp too.

  • rote run on a dbos or temporal app whose extra is missing offers to
    install it.
    It shows a plan: start the installer for how rote was
    installed (uv tool install keeping your extras and --with packages,
    pipx inject --force, this interpreter's pip, uv pip install --python for a
    venv without pip, pip install -e for a checkout), contacting pypi.org.
    A person confirms it; --yes applies it; --json or an agent caller
    without --yes gets action_required (exit 2) and nothing is installed.
    uvx, or an install from a path, gets the command to run instead. The
    version is pinned in every install command, so adding an extra never
    moves rote.

  • Behavior change: a dbos or temporal app needs its extra, and on a plain
    install rote run stops to ask for it.
    rote compile, rote emit and
    rote init never stop for it: they write the code, print the install
    hint as a note, and exit as before.

  • rote doctor checks the base dependencies, names the command that
    restores a missing one, and fails its check when one is missing; it used
    to say "ready to compile" without them. Missing extras are listed and
    never fail it. Emitted apps' requirements.txt and READMEs name the
    same MCP client range rote installs (fastmcp-slim[client]>=4.1.3,<5).

  • The mcp extra stays as an empty alias, so rote-cli[mcp] still
    resolves.

  • The paired eval calls a server you did not register read-only, through
    its own readOnlyHint annotations.
    Both sides may call only the tools
    the server marks read-only, the rest are denied, and a pipeline that
    calls a tool the server does not mark stops before anything is spent.
    Your login is copied owner-only into each trial's throwaway state.

Fixed

  • A compiled step sends the argument values a recorded run sent. A
    compile of Atlassian's search skill passed a page's ARI where the tool
    takes the plain id, and every call failed with a 401. The generation card
    now shows an MCP-bound reader's recorded argument values, and the module
    converts to that form in code. Recorded results from read-only tools are
    replayed through rote's own client before the compile reads them, so it
    sees the shape a compiled step receives (Atlassian's text block wraps the
    page; its structured result is the page).

  • A judge whose passthrough covers every output field no longer fails the
    compile
    : the passthrough is dropped and the model answers as before. An
    unused fallback tool on a server the pipeline already calls is a warning,
    not an error.

  • rote setup on a machine with no AI-client config says it found
    nothing to import instead of "0 server(s) registered", and the demo's
    public servers are no longer said to block the compile.

  • rote try refuses before the compile when the comparison could not
    call a server the skill names.
    The eval only calls servers registered
    read-only, and it used to find out after the compile had been paid for.

  • rote skill run starts faster on every block. Measured on docs-digest
    against the real Microsoft Learn and DeepWiki servers, the median start
    went from 3.47 s to 2.46 s on an install without bytecode (uv tool install) and from 2.90 s to 2.40 s with it. The rote command now hands
    skill run straight to its launcher instead of building the whole CLI
    (rote.entry), the CLI's parser no longer imports the anthropic SDK (about
    0.5 s on every command with the api extra), the driver runs without -B
    so fastmcp keeps its bytecode, and it exits with os._exit once its block
    is printed.

  • A Python judge whose IR sets temperature runs again. anthropic 1.0
    removed the sampling keywords from messages.create, so every such judge
    raised TypeError: ... unexpected keyword argument 'temperature' before
    its first request, on every Python runtime and every SDK rote allows.
    The helper now sends sampling in extra_body and pins
    anthropic-version: 2023-06-01 on that request, because anthropic 2.x
    sends API version 2026-10-21, which rejects sampling outright. A judge
    without sampling is unchanged. Emitted apps pick this up on re-emit.

  • The structure pass keeps an order the skill states without a data
    dependency.
    A node can declare after: [earlier nodes], backed by at
    least four words of its own step's text that code checks verbatim, and
    the order becomes an IR edge. ops-report's gate ("After pulling the
    above, ask the duty manager to provide:") was asked in parallel with the
    data pulls; it now waits for them. An unbacked order is dropped with an
    Open question.

  • A table row is cited as a row. A rule a skill states in a markdown
    table (| ... | review | ... | approved / rejected / pending |) could not
    be cited: the citation was read as prose fragments split on its ...
    cells and refused as "under 4 words", which exited a real expense-audit
    compile 1. A quote that starts with | is now matched cell by cell
    against the skill's table rows, ... matching any cell, and must carry
    at least four words between the cells it quotes (with their column
    headers when it quotes two cells or more). A bare cell such as
    | review | still never passes.