You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug] After a mid-turn crash + model continuation, the tool registry degrades 40→13 (bash/read → UNKNOWN_TOOL) and never rebinds
#9998
dsh serve (server + web GUI), profile with several live sessions
Summary
After the active model (glm/glm-5.3) crashed mid-turn and the session was continued on a different model (model continuation), the tool-binding layer was rebuilt only partially: out of 40 bound tools, only 13 remained. All 27 lost tools are the core filesystem/shell/web/goal/job/subagent tools (bash, read, write, edit, glob, grep, web_search, subagent, create_goal, job_*, schedule_*, present, skill, todo_write, workflow, …). The continuing model still holds the pre-crash 40-tool schema in context, so it calls bash/read → the harness returns ToolNotFoundError / code: UNKNOWN_TOOL (unknown tool "bash", unknown tool "read").
The degradation is persistent: the identical 13-tool registry survived three further model switches (MiniMax → qwen3.8-max → back to glm-5.3) and was never restored to the full 40 on any turn. The session is effectively "blinded" — no shell, no file read/write — and only a new session recovers.
A regular (non-crash) model switch does NOT reproduce this: every request/header before the crash carries the full 40 tools while alternating providers (MiniMax and glm-5.3) — i.e. ordinary rotation/switch rebinds correctly.
Steps to reproduce
Start a long-running dsh serve session with an active model (e.g. glm/glm-5.3) that actively uses core tools (bash, read, …). Confirm request/header.tools == 40.
Cause a crash/interruption mid-turn on the active model (ours: the GLM provider went down mid-turn).
Let the session continue on a different model (model continuation / auto-rotation). A system reminder appears: [model changed: assistant turns above this point were generated by glm/glm-5.3; the session continues with deepseek-official/MiniMax-M3.1-Flash-Preview].
On the next turn, inspect request/header.tools and try to call bash or read.
Expected
request/header.tools again contains the full set (40), including bash, read, write, edit, glob, grep, etc. Calling bash/read executes normally. Tool-binding rebuild is independent of whether the session was started fresh or continued after a crash/model switch.
Actual
request/header.tools == 13 (subset). bash/read/write/edit/glob/grep/web_*/subagent*/*goal*/job_*/schedule_*/present/skill/todo_write/workflow/read_image/ask_user_question/exit_plan_mode are all absent.
Calling bash → tool/result with Error: unknown tool "bash", error.name = ToolNotFoundError, error.code = UNKNOWN_TOOL. Same for read.
The subset does not recover on subsequent turns or across further model switches (qwen, back to glm) — still 13.
Evidence (from the durable session log)
request/header.tools: 40 → 13 at the crash, persistent afterwards
3920 request/header provider=deepseek-official model=MiniMax-M3.1-Flash-Preview tools=13 ← already degraded
3924 assistant/message "The turn was interrupted by the restart — checking whether v1.2 landed…"
3925 tool/call name="bash"
3926 tool/result content=[{type:text,text:'Error: unknown tool "bash"'}] isError=true
error={name:"ToolNotFoundError", code:"UNKNOWN_TOOL"}
3930 assistant/message "Tools are limited on this turn (no shell — file reads only)."
3931 tool/call name="read"
3932 tool/result content=[{type:text,text:'Error: unknown tool "read"'}] isError=true
error={name:"ToolNotFoundError", code:"UNKNOWN_TOOL"}
Model-change system reminders (all four directions around the crash)
[model changed: … generated by glm/glm-5.3; the session continues with deepseek-official/MiniMax-M3.1-Flash-Preview]
[model changed: … generated by deepseek-official/MiniMax-M3.1-Flash-Preview; the session continues with qwen/qwen3.8-max]
[model changed: … generated by deepseek-official/MiniMax-M3.1-Flash-Preview; the session continues with glm/glm-5.3]
[model changed: … generated by qwen/qwen3.8-max; the session continues with glm/glm-5.3]
Root-cause hypothesis
The tool registry is assembled from several contributors. On the crash-recovery / resume + model-continuation path, only a subset of contributors re-registers — judging by the surviving subset, these are the Agent Teams layer (list_agents, send_message, spawn_teammate, interrupt_agent, wait_agent, team_task_*), memento/memory (memory, memory_recall), and worktree/cwd (create_worktree, working_directory). The contributor for the core tools (bash/read/write/edit/glob/grep/web_*/subagent*/*goal*/job_*/schedule_*/present/skill/todo_write/workflow) is not rebuilt on recovery. A regular model switch goes through a different path (full rebuild → 40), which is why the bug reproduces only on the "crash mid-turn → continuation" combination.
Secondary clue: the model's introspection referenced names like CompatReadFilea3996, CompatMemory11c898 (the compat middleware registers wrappers with a generation suffix) — i.e. after recovery the compat layer re-registered some tools under new suffixes, but core bash/read never made it into the public header. Needs confirmation against upstream code (dsh-agent-loop / tool assembly / crash-recovery path).
Impact
Severity: high for long-running server (dsh serve) sessions. A provider crash mid-turn + auto-rotation is not rare; the result is a fully "blinded" session (no FS/shell) that cannot be healed without a restart.
The failure mode is visible (explicit UNKNOWN_TOOL), not silent — but the model keeps receiving turns without tools and degrades.
The workaround is cheap but manual: start a new session (losing the in-turn progress).
Suggested direction
On the crash-recovery/resume path, guarantee a full tool-registry rebuild (all contributors), not only the team/memory/worktree layer; or reuse the same full-rebuild path used for a regular model switch.
Regression test: simulate a mid-turn interruption + continuation and assert that request/header.tools after recovery == the full set (40), and that bash/read execute.
Defensive measure: if the model calls a tool that is absent from the current registry but was present in the last valid header, treat it as a binding desync and trigger a full rebuild (instead of just returning UNKNOWN_TOOL).
Report assembled deterministically from the affected session's durable log (session.v4.jsonl.zstd, 4176 JSONL events) by parsing request/header.data.header.tools and tool/result.data.error. Happy to share additional log excerpts if useful.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Environment
@deepseek-ai/dsh 0.2.1-alpha.4dsh serve(server + web GUI), profile with several live sessionsSummary
After the active model (
glm/glm-5.3) crashed mid-turn and the session was continued on a different model (model continuation), the tool-binding layer was rebuilt only partially: out of 40 bound tools, only 13 remained. All 27 lost tools are the core filesystem/shell/web/goal/job/subagent tools (bash,read,write,edit,glob,grep,web_search,subagent,create_goal,job_*,schedule_*,present,skill,todo_write,workflow, …). The continuing model still holds the pre-crash 40-tool schema in context, so it callsbash/read→ the harness returnsToolNotFoundError/code: UNKNOWN_TOOL(unknown tool "bash",unknown tool "read").The degradation is persistent: the identical 13-tool registry survived three further model switches (MiniMax → qwen3.8-max → back to glm-5.3) and was never restored to the full 40 on any turn. The session is effectively "blinded" — no shell, no file read/write — and only a new session recovers.
A regular (non-crash) model switch does NOT reproduce this: every
request/headerbefore the crash carries the full 40 tools while alternating providers (MiniMax and glm-5.3) — i.e. ordinary rotation/switch rebinds correctly.Steps to reproduce
dsh servesession with an active model (e.g.glm/glm-5.3) that actively uses core tools (bash,read, …). Confirmrequest/header.tools== 40.[model changed: assistant turns above this point were generated by glm/glm-5.3; the session continues with deepseek-official/MiniMax-M3.1-Flash-Preview].request/header.toolsand try to callbashorread.Expected
request/header.toolsagain contains the full set (40), includingbash,read,write,edit,glob,grep, etc. Callingbash/readexecutes normally. Tool-binding rebuild is independent of whether the session was started fresh or continued after a crash/model switch.Actual
request/header.tools== 13 (subset).bash/read/write/edit/glob/grep/web_*/subagent*/*goal*/job_*/schedule_*/present/skill/todo_write/workflow/read_image/ask_user_question/exit_plan_modeare all absent.bash→tool/resultwithError: unknown tool "bash",error.name = ToolNotFoundError,error.code = UNKNOWN_TOOL. Same forread.Evidence (from the durable session log)
request/header.tools: 40 → 13 at the crash, persistent afterwardsbash?read?13 survivors (team/agent coordination + memory + worktree + cwd):
create_worktree, interrupt_agent, list_agents, memory, memory_recall, send_message, spawn_teammate, team_task_create, team_task_get, team_task_list, team_task_update, wait_agent, working_directory.27 lost (core FS/shell/web/goal/job/subagent/skill/schedule/…):
ask_user_question, bash, create_goal, edit, exit_plan_mode, get_goal, glob, grep, job_kill, job_list, job_output, present, read, read_image, schedule_create, schedule_delete, schedule_list, schedule_update, skill, subagent, subagent_fork, todo_write, update_goal, web_fetch, web_search, workflow, write.Interruption timeline (seq → event)
Model-change system reminders (all four directions around the crash)
Root-cause hypothesis
The tool registry is assembled from several contributors. On the crash-recovery / resume + model-continuation path, only a subset of contributors re-registers — judging by the surviving subset, these are the Agent Teams layer (
list_agents,send_message,spawn_teammate,interrupt_agent,wait_agent,team_task_*), memento/memory (memory,memory_recall), and worktree/cwd (create_worktree,working_directory). The contributor for the core tools (bash/read/write/edit/glob/grep/web_*/subagent*/*goal*/job_*/schedule_*/present/skill/todo_write/workflow) is not rebuilt on recovery. A regular model switch goes through a different path (full rebuild → 40), which is why the bug reproduces only on the "crash mid-turn → continuation" combination.Secondary clue: the model's introspection referenced names like
CompatReadFilea3996,CompatMemory11c898(the compat middleware registers wrappers with a generation suffix) — i.e. after recovery the compat layer re-registered some tools under new suffixes, but corebash/readnever made it into the public header. Needs confirmation against upstream code (dsh-agent-loop/ tool assembly / crash-recovery path).Impact
dsh serve) sessions. A provider crash mid-turn + auto-rotation is not rare; the result is a fully "blinded" session (no FS/shell) that cannot be healed without a restart.UNKNOWN_TOOL), not silent — but the model keeps receiving turns without tools and degrades.Suggested direction
request/header.toolsafter recovery == the full set (40), and thatbash/readexecute.UNKNOWN_TOOL).Report assembled deterministically from the affected session's durable log (
session.v4.jsonl.zstd, 4176 JSONL events) by parsingrequest/header.data.header.toolsandtool/result.data.error. Happy to share additional log excerpts if useful.All reactions