Skip to content

feat(sdk): emit tool-output-denied so the UI can tell a denial from a failure#5381

Merged
mmabrouk merged 1 commit into
release/v0.105.6from
feat/deny-frame-egress
Jul 18, 2026
Merged

feat(sdk): emit tool-output-denied so the UI can tell a denial from a failure#5381
mmabrouk merged 1 commit into
release/v0.105.6from
feat/deny-frame-egress

Conversation

@mmabrouk

Copy link
Copy Markdown
Member

The problem

When a user denies a gated tool call in the agent playground, the UI showed it as a
tool failure — the same red error box a real breakage produces. A decline ("you chose not
to run this") and a breakage ("the tool crashed") are different things and should read
differently.

The cause is on the wire. A denial reaches the SDK as an isError tool result: the runner
replies reject to the approval gate, and the harness (Claude / Pi) then closes the call as
a failed tool call — indistinguishable, byte for byte, from a genuine tool error. The
Vercel egress had no way to tell them apart, so every denied call became a
tool-output-error frame.

The fix

Add a structural marker at the runner's deny mapping, never a guess from the error text
(which varies by harness and version):

  • The runner flags the denied tool-call id (otel.markToolCallDenied) at the point it
    decides to reject the gate, and the closing failed tool_result carries denied: true.
    This covers the cold decision-map deny (both the Claude ACP gate and the Pi ACP gate) and
    the live keep-alive resume-reject.
  • The SDK egress (both projection sites in stream.py) projects the AI SDK's dedicated
    tool-output-denied chunk for a denied result, and keeps tool-output-error for a real
    failure.

The frontend already renders output-denied as a distinct decline (ToolActivity.tsx), so
no UI change was needed — the missing piece was purely the egress emitting the frame.

The wire, before and after

A user denies a gated bash call, then the turn resumes:

before:  … tool-input-available  tool-output-error   text-delta …   (UI: red error box)
after:   … tool-input-available  tool-output-denied  text-delta …   (UI: "declined")

tool-output-denied is a strict {type, toolCallId} chunk in ai@6's
uiMessageChunkSchema — no errorText/output — so the denied frame carries no leaked
error payload.

Design record

D-022 in docs/design/agent-workflows/projects/qa/final-sweep-decisions.md — the deny path
was designed to emit tool-output-denied; this PR is the egress half that was still missing.

Testing

  • Runner unit (pnpm test, 1209 green): the otel marker stamps denied on a failed
    close only when flagged (and a genuine failure is left untouched); the ACP responder flags
    the gated id on a deny verdict and not on an allow. pnpm run typecheck clean.
  • SDK unit (agents suite): both egress sites project tool-output-denied for a denied
    result and keep tool-output-error for an unmarked failure; the conformance test's
    vendored ai@6 schema mirror now pins tool-output-denied as {type, toolCallId}. The
    pre-existing test that asserted "a deny rides isError, never a denied field" was updated to
    the new contract.
  • Live (EE dev stack, claude / local sandbox / vault key): drove a real gated bash
    call through the product /services/agent/v0/invoke endpoint, denied it inline, and
    captured the resumed turn's frames:
    … tool-input-available, tool-output-denied, text-start, … finish. The release-gate
    deny journey — which asserts outcome == "denied", a state only the new frame
    produces — went green (denied: the gated command never executed (outcome=denied)).

https://claude.ai/code/session_01DnWRxU3dCJ11hgDidm26vW

… failure

When a user denies a gated tool call, the runner replied reject and the harness
closed the call as a failed tool result, so the Vercel egress projected the
generic tool-output-error frame -- the UI rendered a decline as a breakage.

Add a structural denied marker at the runner deny mapping (never string-match the
error text): otel.markToolCallDenied flags the gated tool-call id, and the closing
failed tool_result carries denied: true. The SDK egress (both projection sites)
projects the AI SDK tool-output-denied chunk for a denied result and keeps
tool-output-error for a genuine failure. Covers the cold decision-map deny (Claude
and Pi gates) and the live keep-alive resume-reject.

Design record: D-022 (final-sweep-decisions.md).

Claude-Session: https://claude.ai/code/session_01DnWRxU3dCJ11hgDidm26vW
@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Jul 18, 2026
@vercel

vercel Bot commented Jul 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agenta-documentation Ready Ready Preview, Comment Jul 18, 2026 7:36pm

Request Review

@dosubot dosubot Bot added the Feature Request New feature or request label Jul 18, 2026
@mmabrouk

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 18, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Jul 18, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Denied gated tool calls now carry a structural denied marker from the runner through tracing and protocol events. Vercel adapters project these events as tool-output-denied, while genuine tool failures remain tool-output-error.

Changes

Denied tool-call projection

Layer / File(s) Summary
Protocol and tracing classification
services/runner/src/protocol.ts, services/runner/src/tracing/otel.ts, services/runner/tests/unit/stream-events.test.ts
The tool_result contract accepts denied, tracing records denied tool-call IDs and emits the marker on failed closures, and tests cover denied versus genuine failures.
Permission denial wiring
services/runner/src/engines/sandbox_agent/acp-interactions.ts, services/runner/src/engines/sandbox_agent/run-turn.ts, services/runner/tests/unit/*
ACP deny responses and keep-alive rejection resumes mark the associated tool call, with allow and deny behavior covered by unit-test harnesses.
Vercel denied-output projection
sdks/python/agenta/sdk/agents/adapters/vercel/stream.py, sdks/python/oss/tests/pytest/unit/agents/adapters/test_vercel_stream_conformance.py, sdks/python/oss/tests/pytest/unit/agents/test_ui_messages.py
Live and dev-twin adapters emit strict tool-output-denied chunks for marked results; conformance tests validate the chunk and preserve generic error output for unmarked failures.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: emitting tool-output-denied to distinguish denials from failures.
Description check ✅ Passed The description is detailed and directly describes the denial-signaling changes in the runner and SDK egress.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/deny-frame-egress

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Railway Preview Environment

Status Destroyed (PR closed)

Updated at 2026-07-18T23:39:46.044Z

@mmabrouk mmabrouk left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mmabrouk mmabrouk added the needs-review Agent updated; awaiting Mahmoud's review label Jul 18, 2026
@mmabrouk
mmabrouk changed the base branch from main to release/v0.105.6 July 18, 2026 23:39
@mmabrouk
mmabrouk merged commit d916c3d into release/v0.105.6 Jul 18, 2026
63 of 64 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature Request New feature or request needs-review Agent updated; awaiting Mahmoud's review size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant