[Bug] ask_user_question inside run_code: the answer reaches the host but is silently dropped from the conversation

[xinhochen]

ask_user_question inside run_code: the answer reaches the host but is silently dropped from the conversation

Environment

• DeepSeek Harness 0.1.0-rc.6 (packages @deepseek-ai/dsh-* 0.1.0-rc.6), npx checkout at /Users/lynk/.npm/_npx/1e7f6d9597241db0
• Web GUI (dsh web, profile web), macOS
• Model: deepseek-v4-pro (deepseek-official)

TL;DR

When the model calls ask_user_question from inside a run_code program, the question popup works and the user's selection does arrive at the host (the sub-call settle event carries the complete answer). But if the program doesn't return/print the ask result, the outer run_code completes with "(run_code completed with no output)", the answer never re-enters the conversation, and the model wrongly concludes the user never answered. The information is in the transcript (the tool/code-dispatch event), but it is discarded at the outer tool-result boundary. This is a footgun for interactive tools; suggesting a harness-level guard below.

Steps to reproduce

1. In a web session, give the agent a task that requires a user decision (e.g. "confirm the logging-backend choice with the user").
2. The agent wraps the interactive ask in run_code:

await tools.ask_user_question({ questions: [{ id: "backend_choice", ... options ... }] });
// no return, no console.log

3. The question popup renders normally; the user picks an option.
4. Observed: the page shows (run_code completed with no output), and the model's next step reasons "The ask_user_question returned no output (maybe no answerer available or empty)" and proceeds with a wrong assumption (暂缓决策 / "defer the decision") even though the user explicitly selected an option.

Actual behavior vs expected

• Expected: the user's selection is the tool result; the model reads it and acts on it.
• Actual: the selection is recorded correctly in the sub-call dispatch event, but the outer run_code result is empty, so the model never sees the selection.

Evidence from the session log (session.jsonl, turn 59)

seq	event	time offset	content
864008	tool/call (run_code)	+0ms	program calls ask_user_question, no return/log
864009	tool/code-dispatch-start	+34ms	sub-call ...:code:1 ask_user_question
864010	tool/code-dispatch (settle)	+13.07s	isError: false, content = {"answers":[{"id":"backend_choice","selected":["采纳：统一 zerolog（Recommended）"]}]} — the user's real selection
864011	tool/result (run_code)	+13.07s	"(run_code completed with no output)"

So the question/answer pipeline (userQuestions service → apiproxy provider → web UI → answer back) works end-to-end. The answer is present in event 864010. Only the final hop — the run_code program propagating the value — is missing, and nothing downstream recovers it.

Root cause

The code-mode dispatch bridge correctly executes the nested ask_user_question through the full tool pipeline and settles the sub-call with the canonical answer. The worker program received the value but discarded it (no return/console.log), so the outer run_code result is empty by contract ("absent result means the program returned undefined"). The agent then sees an empty tool result and misinterprets it as "no answerer available".

Discussion points / proposed improvements

1. Surface interactive-tool sub-call results automatically. When a run_code program calls an interactive/human-answer tool (ask_user_question, or anything whose result originates from the user), the harness could append that sub-call's answer to the outer run_code result (or emit it as an additional visible message), so a program that forgets to propagate it cannot silently drop a human decision. The tool/code-dispatch event already carries the data — it just needs to cross the outer-result boundary.
2. Code-mode SDK prompt guidance. The TypeScript/Python SDK text could carry a hard rule: "if you call an interactive tool (ask_user_question), you MUST return or print its result; the answer otherwise never reaches the conversation."
3. Consider rejecting/redirecting interactive tools in code mode. Either forbid ask_user_question inside run_code (force a direct top-level call, whose result reaches the conversation automatically), or make the bridge auto-print the answer for such tools.

Related observation: the model also reported the failure as "ask_user_question returned no output", which led it to conclude the user never answered — a misleading failure signature for a case where the answer actually arrived.

Impact

Low severity (no data loss — the answer is recoverable from the log), but high confusion cost: the agent proceeds against an explicitly stated user decision. This happened in a long-running multi-round review session (turn 59) and silently derailed the plan until the user noticed.

[Electricitysheep]

ask_user_question 在 run_code 里答案到达 host 却被静默丢弃——工具链的"交互反馈回路"断裂（工具结果没进会话流）。

这类工具返回/上下文注入问题第 8 章有排查思路（产物追踪/上下文注入）：https://github.com/Electricitysheep/dsh-handbook/blob/main/docs/08-tools-context.md

[xinhochen]

看了下第8章，我遇到的情况和他们的不一样。我这个是run_code里调用了ask_user_question插件，然后没等返回，4毫秒就返回了空结果。而内层的ask_user_question在12秒后返回的结果被直接丢弃，dsh建议我在AGENTS.md里加约定来规避，不过我还是认为这个应该是一个run_code处理的问题。

[Electricitysheep]

补充一个更明确的复现（我这边也踩到了）：run_code 里调 ask_user_question，4ms 就返回空结果（同步执行不等异步），而内层 ask_user_question 12 秒后返回的结果被直接丢弃——答案到了但没进会话流。

这基本可以确认是 run_code 的执行模型问题，不是模型/文档约定能根解的：

1. run_code 是同步沙箱执行，内部发的异步工具调用（ask_user_question 这类交互式工具）不阻塞，代码立即跑完返回空
2. 异步结果 12s 后回来时，run_code 的会话帧已结束，结果无处安放被丢弃

AGENTS.md 约定只能"让模型别这么写"（治标），治本应该在 run_code 侧：要么支持异步工具回调（把结果挂回会话），要么对未完成的交互式工具返回 pending 状态让上层等待。

这个案例（run_code 内交互式工具的回调丢失）会记录进手册第 8 章工具链的坑位——欢迎在帖子里补充更多复现细节，方便给官方一个完整的 bug 报告。

[xinhochen]

基本上能提供的细节dsh已经帮我整理好了提交在这里了，我看了下也不能补充更多了

[Electricitysheep]

@xinhochen 你这个复现更精确：run_code 里调 ask_user_question，4ms 就返回空结果（同步执行不等异步），内层 12s 后返回的结果被直接丢弃（会话帧已结束）。

这和原帖的"答案到达 host 但被丢弃"是同一执行模型问题的两个面：

1. run_code 是同步沙箱执行，交互式工具的异步回调不阻塞，代码立即跑完返回空
2. 异步结果回来时 run_code 会话帧已结束，结果无处安放

同意你的判断：这是 run_code 处理的问题（应支持异步工具回调挂回会话，或对未完成交互工具返回 pending），AGENTS.md 约定只能让模型别这么写，治标不治本。

这个案例我们已记录进手册第 8 章工具链坑位（run_code 内交互式工具异步回调丢失），也欢迎你补充更多复现细节，给官方一个完整 bug 报告。https://github.com/Electricitysheep/dsh-handbook/blob/main/docs/08-tools-context.md