Rearm the wire tools array after a mid-turn toggle_toolbox - #468
Merged
Conversation
The browser builds the request's tools array once, at envelope-POST time, from the thread's toolboxes_enabled - and the orchestrator's round chain reused that frozen array for every round. A toolbox the model enabled mid-turn via toggle_toolbox therefore shipped no tool schemas until the next user turn. Most serving backends papered over the gap by accepting calls to tools the request never declared (and performToolCall runs any registered name), so toggle-then-write appeared to work; a backend that holds the model to the declared list turned the same flow into a visible failure - the model's intended followup_create came out as followup_list, eight rounds in a row, right after two successful toggles (thread "Baking Mishap with Joy's Loaf", model deepseek-v4-flash-0731). The envelope now also carries the full tool catalog (always-on defs plus every gated toolbox's defs keyed by name, MCP boxes included), and the round loop rebuilds body.tools from it whenever a round's toggle_toolbox succeeds. The browser stays the single source of truth for what tools exist; the server only filters and dedupes what it was shipped, in catalog key order, so a rebuilt array is exactly what buildToolList would have produced had the toggle preceded the turn - tests/tool-catalog-parity.test.ts imports both sides and pins that equivalence, and the new Deno tests cover the envelope boundary coercion. Envelopes without the field (older cached browser builds) degrade to the old frozen-array behavior. Also rewrites the stale chat.md gotcha that described the retired browser-side round loop's in-flight toggle handling (including a toolboxes_enabled_change broadcast that no longer exists), fixes the matching stale comment in chat/loop.ts, and backfills a QA use-case for the single-turn toggle-then-write flow.
The handler (and ChatLoopResult.toolboxesEnabled alongside it) was the browser-driven round loop's way of reacting to a model toggle in-process. Since the round chain moved into the venice function, nothing ever invoked the handler and nothing read the returned snapshot - the composer toolbox flash already keys off the threads realtime UPDATE echo in the drawer-refresh path, which Chat.svelte's own comments called the only remaining signal. Remove the handler declaration, the dead Chat.svelte callback, the unread result field, and the comment/doc references to them.
The prompt-augmentation contributor table called the wire tools array a per-turn snapshot of toolboxes_enabled, and the MCP doc's wire-inflation bullet did not mention that the /stream envelope now also ships the full catalog (disabled MCP boxes included) for the rearm. Both statements were left behind by the rearm change; the catalog note also draws the line that envelope weight is browser-to-function POST cost, never Venice-wire or model-context cost.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SYNOPSIS
Make a toolbox the model enables mid-turn callable in the same turn, by shipping the full tool catalog in the /stream envelope and rebuilding the wire
toolsarray server-side after a successfultoggle_toolbox.PURPOSE
Currently the browser builds the request's
toolsarray once, at envelope-POST time, from the thread'stoolboxes_enabled, and the orchestrator's round chain reuses that frozen array for every round. A toolbox the model enables mid-turn viatoggle_toolboxtherefore ships no tool schemas until the next user turn. Bad because the whole toggle-then-write flow the system prompt teaches ("enable that toolbox FIRST... Then call the write tool") silently depends on the serving backend accepting calls to tools the request never declared. A backend that holds the model to the declared list turns the flow into a visible failure: in production (thread "Baking Mishap with Joy's Loaf", modeldeepseek-v4-flash-0731), the model's intendedfollowup_createcame out asfollowup_listeight rounds in a row - right after two successful toggles - while each call'sactivitytext said "Creating a follow-up...".DESCRIPTION
How the existing code behaves:
buildToolList(browser) filters the catalog to enabled boxes and the result ridesbodyTemplate.tools;getStreamingResponsecopies the body round-to-round mutating onlymessages;performToolCalldispatches any registered name without re-checking the gate - enablement is request-shape only.What this PR changes, at each of those points:
chat/loop.tsnow also shipstoolCatalog: buildToolCatalog(mcpToolboxes): always-on defs plus every gated box's wire defs keyed by name (static boxes inTOOLBOXESorder, then MCP boxes). Browser stays the single source of truth for what tools exist.toggle_toolboxsucceeded, the orchestrator rebuildsbody.toolsfrom the catalog + the toggle's accepted set (buildToolsFromCatalogin the new dependency-freevenice/tool_catalog.ts); next round declares the newly enabled writes. Guarded onArray.isArray(body.tools)so a stripped-tools body stays stripped.tools.mdgotcha documents that posture explicitly.How that fixes PURPOSE: the rebuilt array is exactly what buildToolList would have produced had the toggle preceded the turn -
tests/tool-catalog-parity.test.tsimports both runtimes' halves and pins the equivalence per enabled-set shape; Deno tests cover the envelope boundary coercion. Envelopes without the field (older cached PWA builds) degrade to the frozen-array behavior.Also in the diff:
onToolboxesEnabledChangehandler surface + unreadChatLoopResult.toolboxesEnabled(orphaned by the server-side round-chain migration; nothing invoked or read them - the composer flash keys off the threads realtime UPDATE echo)chat.mdgotcha describing the retired browser-side loop (incl. atoolboxes_enabled_changebroadcast that no longer exists) + the staleloop.tscommenttoolbox-midturn-enable.mdfor the single-turn toggle-then-write seam; notes inprompt-augmentation.md+mcp-integrations.mdNotes:
rearmed toolslog line)Generated by Claude Code