Skip to content

v0.50.0: stable prompt-cache prefix, session affinity, codemode

Choose a tag to compare

@nuroctane nuroctane released this 14 Oct 19:32
· 26 commits to main since this release

Prompt cache: a prefix that actually holds

The v0.43.4 trial (#1, #18) measured 16% cache hits where Codex and pi reach 96-98% on the same model. Every request now keeps the cached prefix intact:

  • Static system prompt. Permission mode, the todo list and the session inventories (goal, harness, context variables, routed memory) no longer live in the system prompt. They ride as the last input item of each request, headed [nur turn state], rebuilt per request and never persisted, so the prompt is byte-identical from the first request of a turn to the last.
  • Full tool roster every request, in a stable order. The task-shaped first-round profile that changed the schemas between requests is gone.
  • Skill activation is a conversation item after the user message, not a prompt edit.
  • Anthropic cache breakpoints sit on the prompt and tools and roll along the history; the turn-state block never carries one.
  • Session affinity (#2). One stable id derived from the nur session id (so it survives /model, /resume and failover) is sent as x-opencode-session on the OpenCode Go route (gated on the host, not the provider id, so opencode-go and custom providers pointed at Go work), session_id on the Codex OAuth path and x-session-affinity on Fireworks.
  • A per-request status line reports the share of input read from cache.
  • E2E: the fake provider now records request headers (#16); scenario 31 asserts the stable prefix across a whole turn.

Docs: new "Prompt caching" section in docs/providers.md.

codemode

A new codemode tool, modelled on Command Code's: the model writes a short JavaScript script that calls other tools in a loop, and only what the script prints or returns comes back, so three or more dependent reads cost one model round instead of one each.

  • Runs in Boa, a pure-Rust engine with no filesystem, network, timer or process bindings; the tool bridge is the only way out.
  • The bridge admits a nested call only when the permission layer already classifies it read-only (the same rule the approval gate uses, not a second allowlist), and never agent, question, submit_plan, message, admission, bg, fractal, omp or codemode itself. A script cannot write, edit, run a shell command or spawn work.
  • Globals: tools.<name>(args), ALL_TOOLS, searchTools(q), describeTool(name), CWD, text(...) / console.log(...), store(k, v) / load(k), exit(value); top-level await and return work.
  • Limits: 64 nested calls, 1 MB printed output, 10M loop iterations, 1 s to 10 min deadline.
  • Free in plan mode and approval-free in manual mode (it is read-only); not offered to subagents. Cargo feature codemode, on by default. E2E scenario 32.

Docs: docs/tools.md → codemode.

Also

  • src/api/anthropic.rs: UTF-8 mojibake in comments repaired.
  • Release binary grows from 35 MB to 44 MB for the JS engine; --no-default-features or dropping codemode from the feature list builds without it.