Make agent tool configuration explicit - #69
Merged
Conversation
Contributor
Author
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
rgarcia
force-pushed
the
hypeship/agent-tool-configuration
branch
from
July 28, 2026 17:09
1f10b43 to
198bb0c
Compare
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
There are 3 total unresolved issues (including 2 from previous reviews).
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Failed-turn flag survives prompts
- Added a
before_agent_starthook that resetstoolTurnFailedat the start of every prompt so stale failure state cannot block subsequent tool calls.
- Added a
Or push these changes by commenting:
@cursor push 5ae0b3c70a
Preview (5ae0b3c70a)
diff --git a/packages/agent/src/agent.ts b/packages/agent/src/agent.ts
--- a/packages/agent/src/agent.ts
+++ b/packages/agent/src/agent.ts
@@ -351,6 +351,10 @@
else if (event.type === "tool_execution_end" && event.isError) this.toolTurnFailed = true;
else if (event.type === "queue_update") this.hasPendingQueue = event.steer.length > 0 || event.followUp.length > 0;
});
+ core.on("before_agent_start", () => {
+ this.toolTurnFailed = false;
+ return undefined;
+ });
if (onPayload) {
core.on("before_provider_payload", async ({ model: selectedModel, payload }) => ({
payload: (await onPayload(payload, selectedModel)) ?? payload,You can send follow-ups to the cloud agent here.
rgarcia
force-pushed
the
hypeship/agent-tool-configuration
branch
from
July 28, 2026 17:21
198bb0c to
608ed09
Compare
Contributor
Author
|
bugbot run |
rgarcia
force-pushed
the
hypeship/agent-tool-configuration
branch
from
July 28, 2026 17:24
608ed09 to
9650243
Compare
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Semantic failures keep screenshots
- I now treat semantic batch stops as errors when formatting read results so screenshots are replaced with text markers, and added a regression test for that path.
Or push these changes by commenting:
@cursor push a63771d1a1
Preview (a63771d1a1)
diff --git a/packages/agent/src/resources.ts b/packages/agent/src/resources.ts
--- a/packages/agent/src/resources.ts
+++ b/packages/agent/src/resources.ts
@@ -101,9 +101,9 @@
failure = error;
}
- const formatted = formatReadResults(result.readResults, spec.execution.resultPolicy, failure !== undefined);
const semanticFailure = spec.execution.batch && hasUnsatisfiedSemanticRead(result.readResults);
const isError = failure !== undefined || semanticFailure;
+ const formatted = formatReadResults(result.readResults, spec.execution.resultPolicy, isError);
const browserWriteNeedsGrounding = spec.execution.resultPolicy === "browser" && !actions.some(isExplicitBrowserRead);
if (!isError && (browserWriteNeedsGrounding || formatted.content.length === 0)) {
await this.appendGrounding(formatted.content, formatted.details, spec.execution.resultPolicy);
diff --git a/packages/agent/test/resources.test.ts b/packages/agent/test/resources.test.ts
--- a/packages/agent/test/resources.test.ts
+++ b/packages/agent/test/resources.test.ts
@@ -99,6 +99,38 @@
]);
});
+ it("replaces screenshots when semantic browser reads stop a batch", async () => {
+ const { resources } = setup();
+ vi.spyOn(resources, "computer").mockResolvedValue({
+ readResults: [
+ { type: "screenshot", data: Buffer.from("viewport"), mimeType: "image/png" },
+ {
+ type: "browser_wait_for",
+ result: {
+ status: "timed_out",
+ evidence: "failed",
+ initial: { truth: false, details: ["before"] },
+ final: { truth: false, details: ["after"] },
+ elapsed_ms: 1000,
+ details: ["timed out"],
+ },
+ },
+ ],
+ stoppedActionIndex: 1,
+ skippedActions: 1,
+ });
+ const spec = cua.tools.browser.batch({ actions: ["snapshot"] });
+ const tool = resources.materialize(spec);
+ const result = await tool.execute("browser-batch", { actions: [{ action: "snapshot" }] });
+ expect(result.content).toEqual([
+ { type: "text", text: "[screenshot captured: 8 bytes]" },
+ { type: "text", text: "wait_for: timed_out/failed after 1000ms\ntimed out" },
+ expect.objectContaining({ type: "text", text: expect.stringContaining("stopped at an unsatisfied semantic browser condition") }),
+ ]);
+ expect(result.content.some((item) => item.type === "image")).toBe(false);
+ expect(result.details).toMatchObject({ isError: true, failedActionIndex: 1, skippedActions: 1 });
+ });
+
it("shares one lazy browser executor across independently materialized tools", async () => {
const { resources, createBrowserExecutor } = setup();
await resources.materialize(cua.tools.browser.snapshot()).execute("snapshot", {});You can send follow-ups to the cloud agent here.
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 9650243. Configure here.
rgarcia
force-pushed
the
hypeship/agent-tool-configuration
branch
from
July 28, 2026 17:32
9650243 to
da59660
Compare
Contributor
Author
|
bugbot run |
rgarcia
force-pushed
the
hypeship/agent-tool-configuration
branch
from
July 28, 2026 17:40
da59660 to
27cfee2
Compare
Contributor
Author
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 27cfee2. Configure here.
rgarcia
force-pushed
the
hypeship/agent-tool-configuration
branch
from
July 28, 2026 21:36
27cfee2 to
0de325e
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Stop-failure guard not reinstalled
- CuaAgentHarness now reinstalls its native tool_call stop-on-failure guard after each assistant message_end so caller tool_call handlers cannot permanently override turn stopping after a failure.
Or push these changes by commenting:
@cursor push 191d99c83a
Preview (191d99c83a)
diff --git a/packages/agent/src/agent.ts b/packages/agent/src/agent.ts
--- a/packages/agent/src/agent.ts
+++ b/packages/agent/src/agent.ts
@@ -312,6 +312,7 @@
private emptyResponseRecoveryAttempts = 0;
private hasPendingQueue = false;
private toolTurnFailed = false;
+ private removeToolFailureGuard?: () => void;
constructor(options: CuaAgentHarnessOptions<TSkill, TPromptTemplate>) {
const {
@@ -349,14 +350,24 @@
this.env = core.env;
this.models = core.models;
+ const installToolFailureGuard = () => {
+ this.removeToolFailureGuard?.();
+ this.removeToolFailureGuard = core.on("tool_call", () => {
+ const stopMessage = turnFailureStopMessage(manager);
+ return this.toolTurnFailed && stopMessage ? { block: true, reason: stopMessage } : undefined;
+ });
+ };
core.on("tool_result", (event) => hasExecutionError(event.details) ? { isError: true } : undefined);
- core.on("tool_call", () => this.toolTurnFailed && turnFailureStopMessage(manager)
- ? { block: true, reason: turnFailureStopMessage(manager) }
- : undefined);
+ installToolFailureGuard();
core.subscribe((event) => {
- if (event.type === "message_end" && event.message.role === "assistant") this.toolTurnFailed = false;
- else if (event.type === "tool_execution_end" && event.isError) this.toolTurnFailed = true;
- else if (event.type === "queue_update") this.hasPendingQueue = event.steer.length > 0 || event.followUp.length > 0;
+ if (event.type === "message_end" && event.message.role === "assistant") {
+ this.toolTurnFailed = false;
+ installToolFailureGuard();
+ } else if (event.type === "tool_execution_end" && event.isError) {
+ this.toolTurnFailed = true;
+ } else if (event.type === "queue_update") {
+ this.hasPendingQueue = event.steer.length > 0 || event.followUp.length > 0;
+ }
});
core.on("before_agent_start", () => {
this.toolTurnFailed = false;You can send follow-ups to the cloud agent here.
rgarcia
force-pushed
the
hypeship/agent-tool-configuration
branch
from
July 28, 2026 22:47
0de325e to
eaa35b7
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Semantic failures report success
- Removed the batch-only gate from semantic failure detection and added regression tests so standalone browser_wait_for/browser_act failures now set error status and isError.
Or push these changes by commenting:
@cursor push 329abf5a46
Preview (329abf5a46)
diff --git a/packages/agent/src/resources.ts b/packages/agent/src/resources.ts
--- a/packages/agent/src/resources.ts
+++ b/packages/agent/src/resources.ts
@@ -96,7 +96,7 @@
failure = error;
}
- const semanticFailure = spec.execution.batch && hasUnsatisfiedSemanticRead(result.readResults);
+ const semanticFailure = hasUnsatisfiedSemanticRead(result.readResults);
const isError = failure !== undefined || semanticFailure;
const formatted = formatReadResults(result.readResults, isError);
if (isError) {
diff --git a/packages/agent/test/resources.test.ts b/packages/agent/test/resources.test.ts
--- a/packages/agent/test/resources.test.ts
+++ b/packages/agent/test/resources.test.ts
@@ -52,6 +52,16 @@
details: [],
},
}];
+ if (action.type === "browser_act") return [{
+ type: "browser_act",
+ result: {
+ outcome: "didnt",
+ steps: [],
+ stopped_at: 0,
+ stop_reason: "expectation_failed",
+ successor: { status: "unavailable", error: "missing expected text" },
+ },
+ }];
return [];
},
screenshot: browserScreenshot,
@@ -128,6 +138,38 @@
expect(result.details).toMatchObject({ isError: true, failedActionIndex: 1 });
});
+ it("marks standalone browser_wait_for timeouts as errors", async () => {
+ const { resources } = setup();
+ const result = await resources.materialize(cua.tools.browser.waitFor()).execute("wait-for", {
+ expect: { type: "text", text: "Ready" },
+ });
+ expect(result.content).toEqual([
+ { type: "text", text: "wait_for: timed_out/failed after 20ms" },
+ { type: "text", text: "Action 0 stopped at an unsatisfied semantic browser condition." },
+ ]);
+ expect(result.details).toMatchObject({
+ isError: true,
+ failedActionIndex: 0,
+ statusText: "Actions stopped before completion.",
+ });
+ });
+
+ it("marks standalone browser_act semantic failures as errors", async () => {
+ const { resources } = setup();
+ const result = await resources.materialize(cua.tools.browser.act()).execute("act", {
+ steps: [{ type: "click", ref: "e1" }],
+ });
+ expect(result.content).toEqual([
+ { type: "text", text: "browser_act: didnt\nstopped_at: 0 (expectation_failed)\nsuccessor unavailable: missing expected text" },
+ { type: "text", text: "Action 0 stopped at an unsatisfied semantic browser condition." },
+ ]);
+ expect(result.details).toMatchObject({
+ isError: true,
+ failedActionIndex: 0,
+ statusText: "Actions stopped before completion.",
+ });
+ });
+
it("shares one lazy browser executor across independently materialized tools", async () => {
const { resources, createBrowserExecutor } = setup();
await resources.materialize(cua.tools.browser.snapshot()).execute("snapshot", {});You can send follow-ups to the cloud agent here.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Successful act marked as error
- Updated semantic failure detection so browser_act results are treated as errors only when outcome is not worked, preventing successful navigation stop_reasons from being misclassified.
Or push these changes by commenting:
@cursor push 5912f50c39
Preview (5912f50c39)
diff --git a/packages/agent/src/resources.ts b/packages/agent/src/resources.ts
--- a/packages/agent/src/resources.ts
+++ b/packages/agent/src/resources.ts
@@ -195,7 +195,7 @@
return reads.some((read) =>
read.type === "browser_wait_for"
? read.result.status !== "satisfied"
- : read.type === "browser_act" && (read.result.outcome === "didnt" || read.result.stop_reason !== undefined),
+ : read.type === "browser_act" && read.result.outcome !== "worked",
);
}You can send follow-ups to the cloud agent here.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Harness setModel rollback errors swallow original failure
- Guarded each rollback step in both
setTools()andsetModel()so rollback failures no longer replace the original thrown error and model/tools restoration is attempted independently.
- Guarded each rollback step in both
- ✅ Fixed: CuaPayloadPlan.apply silently ignores its model parameter
- Updated
createPayloadPlansoapply(payload, model)now accepts and uses the passed model (with a safe default) when running payload transforms.
- Updated
Or push these changes by commenting:
@cursor push 42b9cb631c
Preview (42b9cb631c)
diff --git a/packages/agent/src/agent.ts b/packages/agent/src/agent.ts
--- a/packages/agent/src/agent.ts
+++ b/packages/agent/src/agent.ts
@@ -387,7 +387,11 @@
try {
await this.coreHarness.setTools(materialized, materialized.map((tool) => tool.name));
} catch (error) {
- await this.coreHarness.setTools(previousTools, previousTools.map((tool) => tool.name));
+ try {
+ await this.coreHarness.setTools(previousTools, previousTools.map((tool) => tool.name));
+ } catch {
+ // Preserve the original failure that triggered rollback.
+ }
throw error;
}
this.tools.commit(catalog);
@@ -404,8 +408,16 @@
await this.coreHarness.setModel(catalog.model);
await this.coreHarness.setTools(materialized, materialized.map((tool) => tool.name));
} catch (error) {
- await this.coreHarness.setModel(previous.model);
- await this.coreHarness.setTools(previousTools, previousTools.map((tool) => tool.name));
+ try {
+ await this.coreHarness.setModel(previous.model);
+ } catch {
+ // Preserve the original failure that triggered rollback.
+ }
+ try {
+ await this.coreHarness.setTools(previousTools, previousTools.map((tool) => tool.name));
+ } catch {
+ // Preserve the original failure that triggered rollback.
+ }
throw error;
}
this.tools.commit(catalog);
diff --git a/packages/ai/src/tool-catalog.ts b/packages/ai/src/tool-catalog.ts
--- a/packages/ai/src/tool-catalog.ts
+++ b/packages/ai/src/tool-catalog.ts
@@ -593,9 +593,9 @@
const ordered = Object.freeze([...transforms].sort((a, b) => phases[a.phase] - phases[b.phase]));
return Object.freeze({
transforms: ordered,
- async apply(payload: unknown): Promise<unknown> {
+ async apply(payload: unknown, selectedModel: Model<Api> = model): Promise<unknown> {
let current = payload;
- for (const transform of ordered) current = (await transform.apply(current, model, names)) ?? current;
+ for (const transform of ordered) current = (await transform.apply(current, selectedModel, names)) ?? current;
return current;
},
});You can send follow-ups to the cloud agent here.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: setModel skips execution-mode guard
- Added the same mutation-scope check to prepareModel so setModel now rejects in-tool mutations unless the caller tool is sequential.
Or push these changes by commenting:
@cursor push 6f5eadd0cd
Preview (6f5eadd0cd)
diff --git a/packages/agent/src/tool-manager.ts b/packages/agent/src/tool-manager.ts
--- a/packages/agent/src/tool-manager.ts
+++ b/packages/agent/src/tool-manager.ts
@@ -43,6 +43,7 @@
}
prepareModel(model: CuaModelRef | Model<Api>): CuaToolCatalog {
+ this.assertMutationScope();
return compileCuaToolCatalog({ model, requestedTools: this.current.requested, resources: this.resources });
}You can send follow-ups to the cloud agent here.
prepareTools() refused catalog mutation from a tool running in parallel mode, but prepareModel() never called the same guard, so a parallel tool could recompile and replace the active catalog mid-flight via setModel(). Share assertMutationScope() between both entry points and parameterize its message per API. Tests cover the rejection (with model and tool state asserted intact) and the sequential-tool path that is still allowed.
/model with no argument now opens a searchable picker modelled on pi's own model selector: same frame, fuzzy search, centred scroll window, [provider] badges, a check on the active model, wrapping navigation, and single-press cancel. /model <provider:model> still switches directly without any UI; an unresolvable ref reports the error and then opens the picker prefilled. Unlike pi's selector, this one never writes global settings and does no background catalog refresh, because cua's catalog is a static table. /tools opens a session-local menu over exactly the tool list the CLI composed for the active model. It applies a subset of that baseline, in baseline order, through one setTools() call, so it can only remove tools and never add ones the model does not support. Edits are staged and committed with ctrl+s; cancelling leaves live state untouched, and a selection rejected by catalog validation reports the error without mutating the session. Provider-native sets that cannot be partially suppressed (Yutori n1) toggle as one group. Both pickers mount with pi's swap-in-place pattern so the status line and telemetry footer stay visible, and both refuse to open mid-turn. Fixes three bugs found while building this: - ctrl+c and ctrl+d quit the app with a picker open, because input listeners run before the focused component. The listener now yields all input to an open selector. - Bulk-action keys silently did nothing and their hints rendered blank: main.ts constructed a KeybindingsManager it never published, and pi-coding-agent's keyText() reads a different pi-tui module instance than the one we register with. Register cua.tools.* ids explicitly and format hints through our own import. - A /tools apply and a /model switch could interleave, letting a queued setTools() land between a switch's setModel() and its final setTools() and fail to compile against the wrong provider. Both now run through one serialization queue. Also cap picker list height to the viewport so a picker cannot push its own footer off a short terminal, rebuild themed strings on theme change, keep space out of the toggle binding while a search query is active so multi-word queries stay typeable, and honour a rebound tui.select.cancel so the footer hint and the handler cannot disagree.
Record the picker keybindings, the staged-apply semantics, and the subset-of-baseline contract in the CLI README, the CLI skill, the architecture doc, and the tool configuration spec. State the mid-turn refusal accurately: the TUI refuses because recompiling the catalog while a request is streaming is unsafe. The agent's execution-scope guard only covers mutation attempted from inside a tool's execute, so it is not what protects this case.
The live agent-e2e moonshotai cases have failed on every run since browser_act was added to their toolset, while meta (same toolset) and yutori keep passing. Before that commit moonshotai passed in ~6.4s; since it, the turn aborts in ~0.6-1.0s with zero tool calls, which is Moonshot's API rejecting the request rather than the model declining to call a tool. browser_act declares a ~124KB schema, roughly eight times browser_wait_for (~15KB), the largest tool Moonshot demonstrably accepts. So the existing complexSchema allowlist is too coarse: Moonshot takes complex schemas but not one of this scale. Gate schema size separately via largeSchema, drop browser_act from Moonshot's CLI and e2e toolsets, and leave the providers with no contrary evidence untouched. Also assert provider and assistant errors before the tool-call counts in assertStats, since an API rejection also yields zero tool calls and only the error assertions carry a message explaining why.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Moonshot examples include browser_act
- Both provider-matrix examples now handle
moonshotaiseparately and returncua.toolsets.browser()sobrowser_actis no longer sent to Kimi.
- Both provider-matrix examples now handle
Or push these changes by commenting:
@cursor push 471f3f36e5
Preview (471f3f36e5)
diff --git a/packages/agent/examples/agent-provider-matrix.ts b/packages/agent/examples/agent-provider-matrix.ts
--- a/packages/agent/examples/agent-provider-matrix.ts
+++ b/packages/agent/examples/agent-provider-matrix.ts
@@ -34,10 +34,12 @@
return cua.providers.google.toolsets.browser();
case "meta":
case "xai":
- case "moonshotai":
// No first-party native browser surface exists, so use CUA browser primitives
// plus verified dependent plans.
return structuredBrowserTools();
+ case "moonshotai":
+ // Kimi rejects requests once `browser_act` is attached, so keep browser primitives only.
+ return cua.toolsets.browser();
case "tzafon":
// Northstar's documented native computer schema is its supported interaction contract.
return [cua.providers.tzafon.tools.computer()];
diff --git a/packages/agent/examples/harness-provider-matrix.ts b/packages/agent/examples/harness-provider-matrix.ts
--- a/packages/agent/examples/harness-provider-matrix.ts
+++ b/packages/agent/examples/harness-provider-matrix.ts
@@ -34,10 +34,12 @@
return cua.providers.google.toolsets.browser();
case "meta":
case "xai":
- case "moonshotai":
// No first-party native browser surface exists, so use CUA browser primitives
// plus verified dependent plans.
return structuredBrowserTools();
+ case "moonshotai":
+ // Kimi rejects requests once `browser_act` is attached, so keep browser primitives only.
+ return cua.toolsets.browser();
case "tzafon":
// Northstar's documented native computer schema is its supported interaction contract.
return [cua.providers.tzafon.tools.computer()];You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit 6573c8a. Configure here.
BugBot caught that the agent and harness provider matrices still routed moonshotai through structuredBrowserTools(), so both examples would now throw when the catalog rejects browser_act for Moonshot. Both files carried a byte-identical copy of the policy, which is why the same bug landed twice. Move it to examples/shared/tools.ts so there is one copy, and cover it with a test that compiles a catalog for every model the matrices advertise. The examples are excluded from `tsc -b` and never run in CI, so nothing previously caught a policy that could not compile.
Resolves 6 of the 8 advisories `npm audit` reported at HEAD (8 -> 2 full, 4 -> 2 production-only). Production: - packages/agent: sharp ^0.34.5 -> ^0.35.3 for GHSA-f88m-g3jw-g9cj (high; CVE-2026-33327/33328/35590/35591). This is the only advisory in this set that touched attacker-influenced bytes: sharp decodes cloud-browser screenshots in the translator's zoom(). The APIs used (extract/png/ toBuffer) are unchanged in 0.35, so no source changes were needed. Note sharp 0.35 drops the install lifecycle script and the build-from-source fallback; both are documented in the agent changelog. - ws 8.20.0 -> 8.21.1 (high), semver 7.8.5 via lockfile refresh. Dev-only, via the tsx ^4.21.0 -> ^4.23.1 bump and an in-range lockfile refresh: vitest 3.2.4 -> 3.2.7 (critical), esbuild 0.27.7 -> 0.28.1 (tsx 4.23 unpins ~0.27.0), vite 7.3.2 -> 7.3.6, postcss 8.5.13 -> 8.5.25, nanoid 3.3.16. No manifest range was widened beyond the two intended bumps, and no openai/zod/aws-sdk churn. Two advisories remain and are disclosed, not suppressed: brace-expansion 5.0.6 (high) and protobufjs 7.6.4 (moderate). Both are pinned verbatim by @earendil-works/pi-coding-agent 0.80.10's published npm-shrinkwrap.json. Root `overrides` were tested and confirmed to be silently ignored across a dependency shrinkwrap, and `npm audit fix` is a no-op here (0 added, 0 removed, 0 changed) despite reporting fixAvailable: true. No overrides, no ignores, no omission flags, no --force. brace-expansion has no reachable fix: even pi-coding-agent 0.83.0 pins 5.0.7, and GHSA-mh99-v99m-4gvg covers <=5.0.7. Only @onkernel/cua-cli is affected; consumer installs of @onkernel/cua-ai and @onkernel/cua-agent audit clean. Also in this commit, both unrelated to dependencies: - release-cua-agent.yml: the ESM smoke test imported createCuaComputerTools, removed at HEAD by eaa35b7. The cua-agent/v0.8.0 tag would have failed before publish. Now asserts CuaAgent, CuaAgentHarness, formatBrowserActResult, NodeExecutionEnv and cua, verified against a real packed tarball. - packages/ai and packages/agent now declare engines.node >=22.19.0, matching the CLI, the root, and every @earendil-works/pi-* dependency. The floor was previously enforced only transitively.
compileCuaToolCatalog no longer materializes executable AgentTools or carries any pi-agent-core coupling. It accepts CUA declarative specs and sanitized caller Tool declarations plus a viewport, and returns identity entries, pi-ai Tool declarations (catalog.toolDeclarations), and provider plans. Compilation is pure: identical declaration, model, and viewport inputs produce identical catalogs and fingerprints. Remove CuaAgentTool, CuaToolCatalogResources, CuaToolCatalog.requested, catalog.agentTools, entry.requested/agentTool/spec, and all executor fingerprinting (including CuaToolSpec.executorFingerprint and the caller executor WeakMap/counter). Add callerToolIdentity() as the one canonical caller-tool identity scheme shared with cua-agent and cua-cli, and drop the @earendil-works/pi-agent-core dependency from the package manifest.
CuaAgentTool is now defined and exported by cua-agent, the only package that holds both halves of the union. CuaToolManager keeps the caller's requested list as the sole source of truth, projects ordinary AgentTools into fresh declaration-only objects before compilation (execute, label, prepareArguments, and executionMode never cross into cua-ai), and joins compiled entries back strictly by identity with an asserted join — never by position. Implementation identity lives here: keyed on the caller execute function (a new wrapper reusing the same function retains identity; a new execute is a replacement even with identical name/schema) and on the spec object (the same object stays stable across model recompilation; a freshly created spec is conservatively a replacement). Prepared states compose cua-ai declaration fingerprints with that identity for cache-preserving deferred-tool decisions. CuaExecutionResources memoizes materialization so each spec is materialized exactly once per pool. setTools/setModel stay atomic prepare-then-commit, harness rollback reinstalls the exact previously committed executable objects, and any compile, join, or materialization failure leaves all prior state untouched.
toolKey() now uses the exported callerToolIdentity() helper instead of a hand-rolled caller.<name> template, and CuaAgentTool is imported from @onkernel/cua-agent, its new defining package. Picker keys are unchanged.
Update the architecture and agent-tool-configuration docs, package READMEs, and the unreleased 0.8.0 changelog entries for the new ownership boundary: cua-ai compiles deterministic declaration-only catalogs with no pi-agent-core dependency, while cua-agent owns CuaAgentTool, once-per-spec materialization, and implementation identity.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
toolsarray (tools: []is valid)cua.tools,cua.toolsets, andcua.providers.*namespace backed by one stable identity-keyed cataloggetTools()/setTools()and model revalidation without a separateinspectTools()or active-tool APImode,nativeTool,extraTools,playwright, generated prompts,computer_use_extra, result-grounding policy, initial screenshots, and Yutori request-time screenshot injectiongemini-3.6-flash,gemini-3.5-flash, andgemini-3.5-flash-litecua act '<json>'for direct validatedbrowser_actexecution with agent-equivalent bounded feedback and causal exit codes/modeland/toolspickers to the TUI, both session-local, and requireexecutionMode: "sequential"for in-toolsetModel()as well assetTools()@onkernel/cua-ai0.8.0,@onkernel/cua-agent0.8.0, and@onkernel/cua-cli0.6.0Public API
First-party provider surfaces remain explicitly namespaced:
No alpha compatibility shim is retained.
Provider defaults
The CLI and shared examples select one explicit interaction catalog:
browser_actverified plansbrowser_act's larger schemabrowser_actotherwisecomputer_screenshotThe CLI owns and retains its coding-tool list across
/modelchanges rather than classifying compiled tools.Result semantics
isError: trueworkedaction plan remains successful across a terminal navigation boundaryInteractive pickers
/modelwith no argument opens a searchable picker modelled on pi's own modelselector: same frame, fuzzy search over provider/ref/model/name, centred scroll
window,
[provider]badges,✓on the active model, wrapping navigation, andsingle-press
esc/ctrl+ccancel. Two deliberate deviations from pi: thepicker never writes pi's global
settings.json, and there is no backgroundcatalog refresh because cua's catalog is a static synchronous table.
/model <provider:model>is unchanged — it switches directly, without openingany UI, through the same three-step tool transition and rollback as before. An
unresolvable ref now reports the resolver error and then opens the picker
prefilled with what was typed.
/toolsopens a session-local menu over exactly the list the CLI composed forthe active model (interaction tools plus coding tools). It applies a subset of
that baseline, in baseline order, through one
setTools()call, so it canonly remove tools and never add one the model does not support.
↑/↓enterspacectrl+a/ctrl+xctrl+rctrl+sescctrl+cEdits are staged: nothing reaches the harness until
ctrl+s, cancelling leaveslive state untouched, and a selection rejected by catalog validation reports the
error while leaving the session byte-identical. Provider-native sets that cannot
be partially suppressed (Yutori n1) toggle as one atomic group; disabling
everything is allowed and yields a text-only agent. Selections are never
persisted, and
/modelrebuilds the baseline from the new model's defaults withan explicit reset notice — tool identities are provider-specific, so carrying a
selection across providers would be exactly the silent substitution the spec
forbids.
Both pickers mount with pi's swap-in-place pattern so the status line and
telemetry footer stay visible, own all keyboard input while open, and refuse to
open mid-turn.
Bugs fixed while building the pickers
setModel()skipped the execution-mode guard.prepareTools()refusedcatalog mutation from a parallel-mode tool, but
prepareModel()never calledthe same guard, so a parallel tool could recompile and replace the active
catalog mid-flight. Both entry points now share
assertMutationScope().ctrl+c/ctrl+dquit the app with a picker open, because input listenersrun before the focused component. The global listener now yields all input to
an open selector.
main.tsconstructed aKeybindingsManagerit never published, andpi-coding-agent's
keyText()reads a different pi-tui module instance thanthe one we register with. cua's
cua.tools.*ids are now registeredexplicitly and hints formatted through our own import.
/toolsapply and a/modelswitch could interleave, letting a queuedsetTools()land between a switch'ssetModel()and its finalsetTools()and fail to compile against the wrong provider. Both now run through one
serialization queue.
/toolsbaseline could drift. A later switch adopts the exact list itinstalled rather than re-reading
harness.getTools(), which a customizationwould have shrunk — permanently losing the disabled rows.
Also: picker list height is capped to the viewport so a picker cannot push its
own footer off a short terminal; themed strings rebuild on theme change;
spacestays out of the toggle binding while a search query is active; and a rebound
tui.select.cancelis honoured so the footer hint and the handler cannotdisagree.
Validation
Local:
npm run typecheck(all three packages) — cleannpm test --workspace @onkernel/cua-ai— 107 passednpm test --workspace @onkernel/cua-agent— 271 passed, 19 skippednpm test --workspace @onkernel/cua-cli— 148 passed, including all 10ptywright TUI fixtures under
PTYWRIGHT_REQUIRED=1git diff --check@onkernel/cua-ai@0.8.0,@onkernel/cua-agent@0.8.0, and@onkernel/cua-cli@0.6.0packed successfullyPicker-specific coverage:
mutation-queue.test.ts(5) — a queued mutation cannot start until theprevious one settles; a failure surfaces to its own caller only and does not
wedge the queue or reorder its successor.
tools-picker.test.ts(9) — component-level input handling: space togglesonly while the search is empty, spaces type into a non-empty query,
enterstill toggles under an active search,
ctrl+cclears before cancelling,cancel never applies, bulk actions respect the filter, and a rebound
tui.select.cancelis honoured.tool-revalidation.test.ts(5) — against the real harness: a partial Googlesubset is accepted, a partial Yutori n1 subset is rejected with
getTools()provably unchanged, whole-group drops andsetTools([])work,and cross-provider recomposition holds.
tool-selection.test.ts(13),model-picker.test.ts(13),tui-keybindings.test.ts(5),slash-commands.test.ts(+2).cancel with focus restored,
ctrl+ccancel with the process surviving,/model <ref>staying non-interactive plus unknown-ref prefill, tooltoggle/cancel-discards/apply/reopen with
ctrl+a/ctrl+x/ctrl+r, andreset-on-model-change. Stable across repeated runs.
The two agent-side guard tests were verified to fail with the
setModelguardremoved, and the ptywright tool-count assertions derive their expected numbers
from
defaultInteractionTools()+defaultApplicationTools()rather thanhard-coding them.
Live QA:
gemini-3.6-flashcalled the current predefinedtake_screenshotaction successfully through bothCuaAgentandCuaAgentHarnessgemini-3.5-flashandgemini-3.5-flash-liteemitted the current predefined screenshot action in direct provider probescomputer_screenshottool through both agent APIsbrowser_act; an OpenAI live run returnedworkedwith a newly verified ref-value expectation, then the temporary field value was clearedcua actlive QA returnedworkedwith a newly verified transition from a present action button to an absent one and rendered the resulting confirmation successor without an LLM or screenshotEvery live QA run deleted only the Kernel browser sessions it created. No credentials or temporary QA artifacts are committed.
Dependency audit and release hardening (commit
a01265b)npm auditat the previous head reported 8 advisories (1 critical, 5 high, 1 moderate, 1 low);production-only reported 4. This commit resolves 6 of them.
Audit counts
45cf276)a01265b)npm auditnpm audit --omit=devManifest changes
packages/agent/package.jsonsharp^0.34.5→^0.35.3; addengines.node >=22.19.0packages/ai/package.jsonengines.node >=22.19.0package.json(root)tsx^4.21.0→^4.23.1package-lock.jsonpackages/{agent,ai,cli}/CHANGELOG.md.github/workflows/release-cua-agent.ymlAdvisory resolution
vitest3.2.4→3.2.7sharp0.34.5→0.35.3ws8.20.0→8.21.1vite7.3.2→7.3.6postcss8.5.13→8.5.25esbuild0.27.7→0.28.1tsx4.23, which unpins~0.27.0brace-expansion5.0.6protobufjs7.6.4sharpis the only advisory here on attacker-influenced bytes: it decodes cloud-browserscreenshots at
packages/agent/src/translator/translator.ts:216(zoom()). The only APIs usedare
extract().png().toBuffer(), unchanged in 0.35, so no source changes were needed.Confirmed against the GitHub advisory API:
sharp < 0.35.0, patched0.35.0, high, same four CVEs(CVE-2026-33327/33328/35590/35591). One installed copy resolves at 0.35.3, all 26
@img/sharp-*prebuilds at 0.35.3 / libvips 1.3.2; runtime load reports libvips 8.18.3.
Lockfile blast radius is contained: the only production deltas are
sharp+@img/*,ws8.20.0→8.21.1,semver7.8.4→7.8.5, and an optional@emnapi/runtime. Everything else isdev-only and advisory-driven. No manifest range was widened beyond the two intended bumps; no
openai/zod/aws-sdk churn. Every newresolvedURL points atregistry.npmjs.org.🚫 Blocker:
npm auditcannot reach zero from this repositorybrace-expansion5.0.6 (high) has no reachable fixed version. This is not suppressed.It is flagged by two advisories: GHSA-mh99-v99m-4gvg (
<=5.0.7, CVSS 7.5, OOM crash) andGHSA-3jxr-9vmj-r5cp (
>=3.0.0 <5.0.7, CVSS 5.3, exponential-time expansion). It is pinnedverbatim by
@earendil-works/pi-coding-agent@0.80.10's publishednpm-shrinkwrap.json.Every escape route was tested rather than assumed:
overridesare silently ignored across a dependency's shrinkwrap. Verified in a cleanfixture on npm 11.17.0: with
overridesforbrace-expansion/minimatch/protobufjs, npm stillinstalled 5.0.6 / 10.2.5 / 7.6.4 with no warning and an unchanged audit count. Adding them would be
dead configuration and false assurance, so none were added.
npm audit fixis a no-op here —0 added, 0 removed, 0 changed— despite reportingfixAvailable: true. That flag reflects npm proposing to remove/downgradepi-coding-agent, not areal fix.
pi-coding-agentclears it. Shrinkwraps read directly from the registry tarballs:0.80.10 pins 5.0.6; 0.82.0, 0.82.1 and 0.83.0 all pin 5.0.7, still inside
GHSA-mh99-v99m-4gvg's
<=5.0.7range. Installing 0.82.1 or 0.83.0 standalone still audits to1 high.
The fix must happen upstream:
@earendil-works/pi-coding-agentneeds to refresh its shrinkwrap tominimatch10.2.6, which is the first release to depend onbrace-expansion^5.0.8(10.2.5 depends on
^5.0.5). Nothing in this repository can clear it. No ignores, no--omitflags, no--force, no.npmrcaudit config was used anywhere.Scope is narrower than the raw count suggests. Both residuals enter only through
pi-coding-agent, which is a@onkernel/cua-cli-only dependency. Verified by installing the realpacked tarballs into clean consumer projects:
@onkernel/cua-ai0.8.0 +@onkernel/cua-agent0.8.0 →found 0 vulnerabilities@onkernel/cua-cli0.6.0 → 2 (1 high, 1 moderate)protobufjs: fixable, deliberately deferredpi-coding-agent0.82.0+ pinsprotobufjs7.6.5 and would clear this moderate. I implemented thatbump (all four
@earendil-works/pi-*packages 0.80.10 → 0.82.1), measured it, and reverted it:brace-expansionstill leaves 1 high.packages/ai/test/models.test.ts:115, and not as a stale assertion. pi-ai 0.82 revisesKimi K3's catalog entry:
supportsReasoningEffortfalse→true,thinkingFormat"deepseek"→"openai", andthinkingLevelMap.highnull→"high". That changes the requestpayload the CLI sends to Moonshot and cannot be validated offline.
@google/genai's opt-in local tokenizerand untrusted
.protosource text. There are zero references to@google/genai,countTokens,or
tokenizeranywhere inpackages/*/src,test, orexamples.Trading an untestable customer-visible provider change for an unreachable moderate — while still not
reaching zero — is the wrong call for a security patch. It is documented in the CLI changelog with the
exact upgrade path so it can be taken deliberately.
Non-dependency fixes in this commit
.github/workflows/release-cua-agent.yml. The ESM smoke test importedcreateCuaComputerTools, whicheaa35b7removed from the public API. Thecua-agent/v0.8.0tagwould have failed at the smoke step before publish. It now asserts
CuaAgent,CuaAgentHarness,formatBrowserActResult,NodeExecutionEnvandcua, all confirmed exported and verified byrunning the exact smoke script against a real packed tarball.
engines.node >=22.19.0onpackages/aiandpackages/agent. Both were missing it while theCLI, the root, and every
@earendil-works/pi-*dependency declare it, so the floor was enforcedonly transitively. Not a new requirement.
Validation
All run on the committed tree:
npm cinpm run typechecknpm run build(all 4 workspaces)@onkernel/cua-aitests@onkernel/cua-agenttests@onkernel/cua-clitests (PTYWRIGHT_REQUIRED=1)git diff --checknpm pack --dry-run×3Package graph and tarballs inspected: versions
cua-ai 0.8.0,cua-agent 0.8.0,cua-cli 0.6.0areall unpublished (npm latest is 0.7.0 / 0.7.0 / 0.5.0). Internal edges exact — agent →
cua-ai 0.8.0;cli →
cua-ai 0.8.0+cua-agent 0.8.0;@onkernel/sdkidentically0.49.0in agent and cli, whichthe CLI prerelease merge script requires.
publishConfig.access: publicon all three. Tarballs containonly
dist, docs, examples and manifests — no stray files. Agentdistkeepssharpexternal.Consumer install resolves
sharp 0.35.3andws 8.21.1, andcua --helpworks from the packed CLI.No secrets, credentials, or untracked artifacts are present; no live browser or payment actions were
taken. Left unmerged and untagged.
Note
Medium Risk
Documentation and packaging only in this diff, but it documents a breaking public API and ships dependency upgrades (notably sharp for screenshot decoding); release workflow smoke-test fix prevents a failed 0.8.0 agent publish.
Overview
This PR’s diff is mostly documentation, release metadata, and dependency lockfile updates that align the repo with the already-implemented explicit agent tool configuration (the behavioral change lives in prior commits; this slice records and ships it).
Docs and specs are rewritten around a single required
toolsarray and the frozencua.tools/cua.toolsets/cua.providers.*namespace instead ofmode,nativeTool,extraTools,playwright, and implicit helpers likecomputer_use_extra.docs/architecture.mdand the root README describe identity-keyed catalog compilation, shared execution resources, explicit screenshot semantics, mechanical batches, and CLI composition (including model-specific default catalogs and TUI/model//toolspickers).docs/agent-tool-configuration-spec.mdis marked implemented; the harness migration doc is trimmed to a pointer at the new architecture.Release and agent skills:
.agents/skills/release/SKILL.mdadds@onkernel/cua-cli(tagcua-cli/v, workflowrelease-cua-cli.yml) and dependency-order publish steps; changelogs document 0.8.0 (ai/agent) and 0.6.0 (cli) breaking API and behavior.CI and dependencies:
release-cua-agent.ymlsmoke test drops removedcreateCuaComputerToolsand asserts current exports (cua,formatBrowserActResult, etc.).sharp^0.35.3 on cua-agent (image decode path),tsx^4.23.1 at root, lockfile refreshes (vitest, ws, vite, esbuild, etc.), andengines.node >=22.19.0on ai/agent packages.Reviewed by Cursor Bugbot for commit e85e6af. Bugbot is set up for automated code reviews on this repo. Configure here.