Skip to content

v0.28.0

Choose a tag to compare

@muthuishere muthuishere released this 08 Nov 11:53
· 51 commits to main since this release
32c5c17

Upgrading — read this first.

  • BREAKING (rust only): AcpStreamEvent gains ToolCallArgs and is now #[non_exhaustive]; a match
    on it needs a _ arm once. Rust code that builds ClientOptions or Hooks by setting every field by
    hand must add the new stop_recovery / after_round fields (or end the literal with ..Default::default()).
  • java: Jackson 2 (jackson-databind 2.x) is gone; Jackson 3 is the only JSON library. If your code used
    the Jackson 2 that toolnexus brought in transitively, declare it yourself.
  • Default change (#191, all ports, with stopRecovery off): tool arguments that are valid JSON but not an
    object ([1], "x") now reach the tool as {}. Before, js and python passed the raw value through.
  • rust and java: local stdio MCP servers now run on in-house clients (rmcp / the MCP Java SDK remain for
    remote HTTP servers and serve).
  • Benchmarks: the JavaScript speed-up claimed in the request-path work (#185) did not reproduce in the
    quiet benchmark (3.43 → 3.40 ms). Java after the in-house stdio client: init 137 ms, p50 1.63 ms vs
    LangChain4j 220 ms / 1.84 ms, in a separate 7-round sitting (not blended into the standing page's
    5-round medians).

Java: local MCP servers run on an in-house stdio client (java only). Pre-2026 stdio servers no longer
go through the official MCP Java SDK session. That covers nearly every local server in use today: those
that answer server/discover as an old server, plus the child respawned after a silent discover. The SDK
put a Reactor chain and scheduler hops on every request, and built a JSON tree for every line. The new
client writes each request straight to the child's stdin. One reader thread routes replies by id and
hands their raw bytes to the waiting caller, which parses them straight into the same McpSchema result
types. That is the shape LangChain4j and our Rust port use. Server→client requests (ping, elicitation,
roots) are answered off the reader thread, so a server that floods requests while not reading stdin
cannot deadlock a call. Measured on the mock-LLM benchmark (same machine, runs interleaved, medians of 7):
cold init went from 167 ms to 137 ms, and per-request p50 went from 1.88 ms to 1.63 ms. LangChain4j's
MCP client on the same harness: 220 ms init, 1.84 ms p50. A raw tools/call round trip went from ~40 µs
to ~20 µs. Behaviour is otherwise unchanged: the same protocol versions are offered (2025-11-25, down to
2024-11-05), with the same error texts, idle timeout, cancellation and content mapping. One addition, as
the Rust client already does: when a call is given up on, the server now gets notifications/cancelled
with requestId and a reason ("request timeout" for an idle timeout, "cancelled" for a caller
cancel). A late reply to that id is dropped quietly.
Not changed: remote HTTP/SSE servers and serve still use the SDK. 2026-07-28 servers were already
in-house (Mcp2026). One visible difference: a respawned child's stderr now passes through to the host's
stderr, as the probe child's already did, instead of going to SLF4J. Requests are written by one
writer thread, so a server that stops reading stdin can no longer park a caller past its call timeout,
and close also kills the server's children after the server itself has exited. Tracked in
openspec/changes/java-inhouse-stdio-mcp.

Benchmarks re-measured for 0.28.0 (docs only). A quiet re-run (load < 4, five interleaved rounds of 0.27.0
against main, medians) replaces the standing page's numbers and adds sitting 2026-11-08-quiet-0.28 to
benchmarks/results-2026-11.json. Over MCP, main vs 0.27.0: python p50 1.38 → 0.92 ms; java init 361 → 195 ms
and p50 2.44 → 2.28 ms; csharp init 89 → 61 ms; rust p50 0.376 → 0.342 ms; go unchanged. Against competitors:
Rust now beats AutoAgents on latency (0.342 vs 0.368 ms; AutoAgents still uses less memory), C# now starts faster
than Microsoft Agent Framework, Go is level with Eino, and Java is still 5% behind LangChain4j (was 37%). Not
reproduced:
the request-path entry's JS figure (3.28 → ~0.55 ms); this harness shows JS unchanged (3.43 → 3.40 ms).
Elixir and Clojure were not re-run.

Change an ACP agent's mode or model without restarting it (#197). A running ACP client now has
setMode(mode) and setModel(model) (Go SetMode(ctx, mode) / SetModel(ctx, model), Python
set_mode / set_model, C# SetModeAsync / SetModelAsync, Clojure set-mode! / set-model!).
Before, the only way to switch mode (e.g. a shift+tab plan/act toggle) was to close the agent and
loadACP again, which is slow and loses the session. setMode sends session/set_mode; setModel
does exactly what the load-time model option does — session/set_config_option {configId:"model"},
then session/set_model if the agent rejects that. Same process, same session; values pass through
unvalidated and no vendor is named. If the agent rejects the change, only that call fails and the
client keeps working. All eight ports; SPEC §8C, conformance liveSwitch.

ACP tool rows now show their arguments and full title (#188). Agents such as the Claude ACP
adapter send a tool's rawInput (file path, command) and its full title only on a later
tool_call_update; toolnexus read rawInput only from the first tool_call, so a host saw bare
rows like Write or Terminal. A streaming ACP generate now emits
{type:"tool_call_args", id, name, args, via:"acp"} when an update brings rawInput, and an
update's title is kept, so the later tool_result carries Write /tmp/a.txt rather than Write.
Additive: a consumer that ignores unknown event types sees no change. BREAKING (rust only):
AcpStreamEvent gains a ToolCallArgs variant and is now #[non_exhaustive], so a match
needs a _ arm once (future variants will not break again); Java
(StreamEvent.Kind.TOOL_CALL_ARGS) and C# (StreamKind.ToolCallArgs) gain an enum member. All eight ports; SPEC §8,
conformance stream.acpToolUpdate.

A streamed turn now says when it is retrying (#189)

When the provider answers 429, 529 or a 5xx, stream() now emits
{type: "retry", attempt, max, delayMs, status?, reason} before it waits, so a host can show
"Retrying in 4s · attempt 1/2" instead of a frozen spinner. delayMs is the exact wait the loop
is about to sleep — a server's Retry-After when it sent one, otherwise the backoff actually
drawn. reason is "HTTP 429" or "network"; status is absent on a network error. No event
is emitted for a failure that will not be retried, and run/ask output is unchanged.

The client also obeys the x-should-retry: true|false response header (the one Anthropic
and OpenAI send, and their own SDKs obey): it overrides the default retryable classification
before onError runs, in run, ask and stream. onError still has the final say and
retries still bounds every retry. Without the header nothing changes.

  • Additive in every port: the event type gains a retry kind and fields (Rust: kind == "retry"
    with attempt, max, delay_ms, status: Option<u16>, reason on the StreamEvent struct;
    no exhaustive match breaks). Clojure has no stream entry point, so the event goes to
    :on-event (a run without :on-event emits none).
  • In-process clients have no wire and emit no retry event.

Spec: SPEC.md § Streaming and § Resilience; openspec/changes/add-retry-stream-event.

Round-boundary hook: append a message between rounds, or keep a stopped loop going (#192)

A new optional hook, afterRound({ turn, final, text, messages }), fires at the end of every loop
round — after the round's tool results are in the transcript and before the next request. Return
{ inject: ["…"] } and each string is appended as a user message: todo reminders, sub-agent
notices, budget warnings, without rebuilding the transcript in beforeLLM.

It also fires when the model stops (no tool call) as long as a turn remains: a non-empty
injection there continues the loop ("you still have 2 open todos"). maxTurns still bounds it.
Without the hook nothing changes. Run and stream, OpenAI and Anthropic styles, and the §7D agent
path (hooks are forwarded verbatim). It never fires on a pending halt, an error, translate, or an
ACP model source. Spec: SPEC.md §8 Hooks; change openspec/changes/add-round-boundary-hook.

Model fallback chain on overload (issue #190)

A provider under load answers 529 Overloaded or 503, and until now the client spent its
retries on the one model and then failed the run — even when a sibling model on the same endpoint
would have answered. The new opt-in client option fallbackModels (js fallbackModels, python
fallback_models, Go FallbackModels, java fallbackModels(...), C# FallbackModels, elixir
:fallback_models, clojure :fallback-models, rust fallback_models(...)) is an ordered list of
models to switch to: once the retry budget on the current model is used up and the last answer was
503 or 529
, the same request goes to the next model with a fresh budget. 429, other errors
and an onError "fail" never switch models. stream emits {type:"fallback", from, to} before
switching, and RunResult.model, afterLLM and the metric events report the model that actually
served. Set TOOLNEXUS_DISABLE_MODEL_FALLBACK=1 to turn it off without a redeploy. Without the
option nothing changes. Not done: no cooldown/stickiness across calls (each turn starts at its own
model), and in-process/ACP sources ignore the option. Spec: SPEC.md §8,
openspec/changes/add-model-fallback-chain (the model-only slice of add-llm-upstream-failover).

Classifier: an openai backend on OpenAI's Decisions API (issue #157). createClassifier({backend: "openai"}) sends the same noul / choice / score questions to POST https://api.openai.com/v1/decisions
(gpt-6-luna) and returns the same Decision as every other backend. A front end such as jevx can now
offer an openai profile type without its own adapter. Every shared option applies unchanged (baseUrl,
headers, timeout, httpClient, retries, onError, requestParams, bodyTransform, cache, onMetric).
OPENAI_API_KEY goes only to api.openai.com over https (the #155 rule). Images go as input_image
data URLs. Caller keys are still never sent: questions travel in key order and answers are read back by
position. A refusal answer is an error, never a "no". Three things this wire cannot take fail before
any request: raw questions such as box, a one-option choice, and more than 400 questions.
Not done: decisions from this backend report calibrated: false until calibration is measured, so
thresholds tuned on Jev do not transfer. The Decisions API has no batch call, so states().evaluate
still fans out one request per state. There is no model catalogue (#158). OpenRouter's and Vercel's
Decisions routes have no preset yet; they work today with a baseUrl override and a named apiKeyEnv.
The wire was checked live against gpt-6-luna from Go, Python and Rust (yes/no, pick-one and
score). Every port's mock-kit suite also covers it through tools/mock-llm's /v1/decisions.
Tracked in openspec/changes/add-classifier-openai-decisions. Shared fixture
examples/judge/backends/openai-decisions.json. Docs: an "OpenAI (Decisions API)" section on the judge
backends page and an openai recipe (yes/no, pick-one, score) in the judge cookbook, in all eight languages.

Stop-reason recovery (stopRecovery, opt-in)

A run no longer has to end on a reply that is not an answer. Set stopRecovery: true
(stop_recovery / StopRecovery / :stop-recovery) and the loop recovers from the three stop
shapes Karya (go-agent-cli) hit in real runs (issue #191):

  • Empty answer (a thinking-only end_turn: reasoning, no text) — the empty reply is dropped
    and the model is nudged once to give its answer. Before, text came back "".
  • Truncated (OpenAI finish_reason: "length", Anthropic stop_reason: "max_tokens") — the
    partial stays and the model is asked to continue, up to 3 times; text is the joined answer.
    Before, a half answer came back as if it were final.
  • Malformed tool call (arguments that are not a JSON object) — no tool runs, the reply is
    dropped and the model is told once to call again with valid JSON. Before, the tool silently ran
    with {}.

Each recovery appends one byte-exact user message pinned in SPEC.md §8, so every port sends the
same words. Recoveries are ordinary turns: they fire beforeLLM/afterLLM, count in turns and
usage, and never exceed maxTurns. Applies to run and stream, both styles, resume paths and
agent runs; not to translate or the in-process client. Off by default — with the option
absent a run is byte-identical to before. Not done: no stream event or metric announces a
recovery (watch for the pinned text in beforeLLM), and the budgets and texts are not
configurable. Fixture: conformance/stop_recovery/cases.json; change
openspec/changes/add-stop-reason-recovery.

Small behaviour change with the option off: tool arguments that are valid JSON but not an
object ([1], "x", or an Anthropic input that is not an object) now reach the tool as {}
in every port, as malformed text already did. Before, js and python passed the raw value through
(and the Go Anthropic loop failed the whole response on a non-object input).

Reactive compaction on context overflow (issue #194)

A long agent run no longer dies the first time the provider says the prompt is too long. Set
the new client option compactOnOverflow (a beforeLLM-shaped hook — usually a
compactor(...) with a tighter budget than your proactive one) and the loop answers an
overflow by compacting the transcript and re-sending the same turn. Without the option a run
is byte-identical to before
: the overflow still surfaces as the typed LlmHttpError.

Why: the proactive compactor works from an estimate (ceil(chars/4)); only the provider knows
whether a prompt fits, and one huge tool result can push a turn over between estimates. Karya
(go-agent-cli) asked for this, with a breaker.

  • Detection is pinned and exported as isContextOverflow(status, body): status 400 or
    413 and one of context_length_exceeded, maximum context length,
    prompt is too long, prompt_too_long, request_too_large, context window in the
    lowercased body. Covers OpenAI, Anthropic and OpenAI-compatible gateways; a bare 413 or a
    429 that mentions context length is not an overflow.
  • The breaker: an overflow within fewer than two successful LLM calls of the last reactive
    compaction is a rapid refill; after three compactions in a rapid chain the run fails with the
    typed ContextOverflowError (compactions, status, body; message
    context overflow: <n> reactive compaction(s) did not make the prompt fit). Overflows spaced
    further apart reset the chain. A hook that returns nothing, or a transcript that is not
    shorter, fails at once (compactions 0) instead of looping.
  • The retry does not run beforeLLM again and does not consume a turn. Applies to run and
    stream, both styles, and the resume paths. SPEC §7F.1; fixture
    conformance/reactive_compaction/cases.json; change openspec/changes/add-reactive-compaction.
  • Not done: no per-agent (AgentDef) option; a §7D agent run uses its client's option.
  • Not done: the single-call translate paths do not react to an overflow (they are not
    loops). Resume paths are covered because they run through the same loops, but no port has a
    dedicated resume test. In golang, ContextOverflowError unwraps to the last *ProviderError;
    elixir and rust carry it as a separate error type, with no unwrap equivalent.

Memory hierarchy and dreaming

A persona's memory can now stay small and stay true. Everything is opt-in; a persona built
without the new options is byte-identical to before.

  • memoryIndexCap on fromDir: injects at most 200 lines / 25600 bytes of MEMORY.md
    (whole lines, a fixed notice when cut) so a diligent persona no longer pays for an ever-growing
    index on every turn. Disk is untouched.
  • notes: true: the memory tool gains note:<slug> targets (memory/<slug>.md, never
    injected), read and a sorted list. Bad slugs (../SOUL, uppercase) are refused with no I/O.
  • transcriptSearchTool(store): a read-only tool that searches stored conversations
    (case-insensitive substring), so an agent can check what it believes against what happened.
  • dreamer(dir, { store }) and fromDir(dir, { dream: { everyMs, store } }): a built-in
    dream pass that verifies MEMORY.md against the transcripts, fixes or prunes claims, and
    records every change in memory/log.md. No HEARTBEAT.md to write.
  • A dream cannot delete correct memory without a trace. The prompt removes a claim only when
    the transcripts contradict it; a claim they merely do not mention stays, because transcripts get
    pruned and silence is not evidence. The dreamer's memory tool refuses every action on
    USER.md (dreamer: USER.md is off-limits) and replace/remove on note:log
    (dreamer: memory/log.md is append-only), so a dream never edits the user model and can only
    add to its own audit trail. Not done: no snapshot is taken before a pass — keep the home in git.
  • Transcript search folds case for ASCII letters only, now pinned by a fixture case: É in a
    message is not found by é. A port using Unicode lowercasing fails it.
  • Memory writes are fsynced and serialised per file within a runtime, so concurrent agents no
    longer lose each other's entries.
  • replace/remove now change only the first match in every port. python and java used to
    change every occurrence; a remove "x" there deleted every "x" in the file.
  • The notes-off memory tool's description and schema are now identical in all eight ports.
    python, go, rust, csharp and elixir had drifted ("next session" vs "NEXT session", a missing
    (with), no target description), so the text their models see changes slightly.
  • replace inserts with literally everywhere. js expanded $&, $1, $$ in the
    replacement (so a model writing "costs $&" got the matched text pasted in); all eight ports now
    insert the text exactly, pinned by a fixture case.
  • Hardening from review, all eight ports: a slug with a trailing newline (note:abc\n) is
    refused (elixir and csharp accepted it); list skips symlinks and directories and no longer
    throws on a dangling symlink (js threw; most ports listed links); the write lock keys on the real
    path, so a home reached through a symlink shares one lock with its real path; limit: null
    is an error, not "no limit"; tool_use input keeps U+2028/U+2029 as-is (go and csharp escaped
    them); startAgent throws startAgent: dreamer name collision: <name>-dreamer is already registered instead of letting one agent silently shadow the other, and never leaves the
    persona's heartbeat running when the dreamer fails to start.

All eight ports; fixtures examples/memory-hierarchy/ and examples/dreaming/ (the "Mt. Moon"
case: a dream pass fixes a wrong note and the next session answers differently). Docs: new
"Dreaming" page. Not done: no cross-process write locking; no semantic search; clojure writes
are not atomic (koine has no rename) and its lock is in-process only — SPEC §7E now lists this as a
permitted per-port difference; clojure's note list uses a
real-path check instead of lstat (koine has none), so a FIFO or socket named x.md would be listed; on
a case-insensitive filesystem two spellings of one path may get two locks; a go IndexCap field of 0
means "use the preset" rather than an error; rust custom conversation stores must override
supports_list() to be searchable. Change: openspec/changes/add-memory-hierarchy.

Fix: the persona memory tool no longer truncates a MEMORY.md / USER.md larger than 2 MB.
The tool read the file through the same 2 MB cap used to build the system prompt, then wrote that
capped text back. So once the file went over 2 MB, a single add, replace or remove cut it down
to 2 MB on disk and wrote the truncation notice into it, silently losing everything after the cap.
SPEC §7E always said the cap leaves the on-disk file untouched. Affected: js, java and clojure;
the other five already edited the full file. Those three now do too, and seven ports (all but
clojure) now write atomically (temp file + rename), so a crash mid-write cannot leave a half-written
file; an existing file keeps its permission bits, a new one gets the usual 0666 & ~umask. The cap still applies only when building the system prompt. Pinned by fixture key
H3_memoryWriteOver2MBPreservesDisk in examples/persona-agent/fixture.json. Not done here:
a MEMORY.md that is a symlink is now replaced by a regular file on the first write (the rename
replaces the link), so point the home dir at the real directory rather than linking the file;
clojure's write is not atomic yet (koine has no rename/move, so it needs adding to koine first);
concurrent writers are addressed by the memory hierarchy entry above.

A mock model server: run any toolnexus client, classifier or media job with no key and no cost.
tools/mock-llm/mock_llm.py is one stdlib-only Python file (python3 tools/mock-llm/mock_llm.py --port 8787, base URL http://127.0.0.1:8787/v1, any key). It answers OpenAI chat and Anthropic
messages (tool calls when tools are offered, then a final text; SSE streaming), echoes attachments
back with their mime type, size and sha256 so a test can prove an image arrived, serves the System
One, Cloudflare and OpenAI Decisions classifier wires with deterministic answers (noul 0.95, choice
by hash, score within the rubric), serves a §7G media style covering all six verbs (async jobs with
a poll, tiny real PNG/WAV/MP4 files), and injects 429/5xx with Retry-After on request. --script
overrides replies. Sibling fakes cover every other source: an MCP server (stdio and HTTP, on both
the server/discover path and the frozen one), an A2A agent, HTTP tool endpoints, a one-shot CLI
for createInProcessClient and an ACP agent with permission, tool-call updates and
session/cancel, and a fake Grok/Codex media CLI (fake_media_cli.py; nothing on main drives a
media CLI yet, so it is validated against the unreleased media-clients branch). The in-process recipe fakes moved here from site/recipes/in-process-client/fakes/.
Every port now has an integration suite that runs its real client, classifier
and media engine against it, in its normal test command. Docs: cookbook "Run without a model".

  • Not done: the ports' suites test the System One and Cloudflare classifier wires only; the
    OpenAI Decisions wire is served and self-tested, but the openai backend lives on the #157
    branch, so port-level Decisions cases land with that change. The existing in-process unit fakes
    stay as they are. ACP cancel is not exercised in elixir (#169) or clojure (#170), which lack it;
    clojure has no network streaming, so its suite has no streaming case. The benchmark's own mock (benchmarks/mock_llm.py) is unchanged.

Faster requests and cold starts in python, js, java, csharp and rust (openspec/changes/speed-up-request-path).
Measured against a no-framework floor, most of our per-request time was waiting, not computing: a new
TCP connection per LLM call, a thread hand-off per MCP stdio line, an OS proxy lookup per call, and SDK
code loaded only after the MCP child was up. Fixing those, against the shared benchmark (median p50
per request, before → after, interleaved with a clean HEAD build on one machine): python 1.35 → 0.96 ms,
js 3.28 → ~0.55 ms, java 2.89 → 2.54 ms (0.88 ms once the JIT is warm) with init 358 → 180 ms, csharp
init 93 → 62 ms. These numbers are from a busy machine; the standing page changes only after a quiet re-run.

  • python, js: the default LLM transport keeps the connection open. A connection the server dropped
    while idle is replaced before use; a request that failed after it was sent is never re-sent, so a
    model call cannot be billed twice. js follows redirects from the target and leaves a host-installed
    undici dispatcher in charge. It steps aside, exactly as before, when a proxy env var applies or you pass
    your own transport/fetch. js uses it on Node only; Bun, Deno and browsers keep the platform fetch.
  • python, csharp behaviour change: the proxy decision is cached per scheme and host, so a system
    proxy changed while the process runs is not picked up. Python's urllib already read it once per process.
  • java: drops jackson-databind 2.x; Jackson 3, which the MCP SDK already needs, is the only JSON
    library. JSON output is byte-identical (new golden test). If you used the Jackson 2 that toolnexus
    brought in transitively, declare it yourself.
  • rust: stdio MCP servers now run on our own client instead of rmcp: callers write straight to the
    child's pipe and one reader routes replies by id. A tools/call fell from ~30 to ~17 µs; steady-state
    p50 0.265 → 0.22 ms, ahead of AutoAgents and level with golang (golang still leads by ~10 µs under
    heavy load). Client::run_ctx now runs the turn on a tokio worker thread. Wire changes on the
    pre-2026 stdio path, all bringing rust in line with golang and SPEC §2: initialize offers
    2025-11-25 (rmcp sent 2026-07-28) with clientInfo toolnexus; caller cancellation now sends
    notifications/cancelled; a freshly spawned server's stderr passes through (§2.3); close kills the
    whole process group after 3 s; a tool with a non-object inputSchema no longer fails the whole list.
    rmcp remains for remote (HTTP/SSE) servers and the serve side; the rmcp/transport-child-process
    feature is dropped.
  • Not done: golang needed nothing: it measured within ~8 µs of the floor. elixir and clojure are out
    of scope (#168 tracks clojure).