Bug: Failed tool call causing session dead lock

[marshfolx]

Bug: failed tool image result hard-aborts the turn and is replayed on every "continue", so the session can never recover

Summary

When a read_image tool result carries an image that the LLM provider rejects
(in this case LM Studio returns 400 "'url' field must be a base64 encoded image."
for a WebP data URI), DSH:

1. Converts the provider error into a hard turn-level error (turn/end reason: "error") instead of returning it to the model as a failed tool result
   (is_error: true), so the model never gets a chance to recover (e.g. re-encode
   to PNG and retry).
2. On a subsequent "继续" (continue), replays the interrupted state verbatim —
   including the rejected image still sitting in the tool-result context — so the
   very first model call of the resumed turn fails with the identical error
   again. DSH does not even issue a fresh LLM request; it errors before producing
   any tokens. This repeats forever; the conversation is effectively dead.

Environment

• DSH v0.1.1-rc.2, Windows.
• Provider: local LM Studio, model ornith1.5-9b (a VLM).
• Trigger: read_image on an image DSH re-encodes to WebP (e.g. a PNG carrying a
  text comment chunk). LM Studio's OpenAI-compat endpoint rejects image/webp.

Reproduced

Session session-c62bdc04-464e-46f2-967e-f78d47076aa4 (user title
"上个会话本来是要画个……").

Turn 2 — the failure (session event seqs)

seq 3601  step 4  finish kind="tool-calls"   # model calls read_image
seq 3603  tool/call                           # read_image issued
seq 3604  tool/result                          # returns an image (WebP) into context
seq 3607  step 5  usage inputTokens=0 outputTokens=0
seq 3608  step 5  finish kind="error" failure={
             "message": "400 \"'url' field must be a base64 encoded image.\"",
             "code": "INVALID_REQUEST" }
seq 3610  turn/end reason="error"             # HARD turn abort, error NOT returned to model

The provider rejected the image the moment the model's next step tried to send it.
Instead of feeding this back as a failed tool_result, DSH aborts the whole turn.

Turns 3/4/5 — "继续" replays the dead state (each ~87 minutes apart)

seq 3612  turn/start (turn 3)
seq 3616  step 1 usage inputTokens=0 outputTokens=0
seq 3617  step 1 finish kind="error"  failure={ same 400 INVALID_REQUEST }
seq 3619  turn/end reason="error"

seq 3621  turn/start (turn 4)  -> seq 3626 error (same 400) -> seq 3628 turn/end error
seq 3630  turn/start (turn 5)  -> seq 3635 error (same 400) -> seq 3637 turn/end error

User messages between turns: 继续，去读_im2.png (turn 3), 继续 (turn 4),
继续 (turn 5). Note that none of turns 3-5 produced a request/header event:
DSH never makes a fresh model call. It reconstructs the conversation (which still
contains the rejected WebP image in the tool-result context), attempts the first
model step, and fails with the exact same provider error before generating any
tokens.

Root cause

Two coupled problems:

1. Provider/tool errors are escalated to a hard turn abort instead of a failed
   tool result. The model never sees "read_image/that image failed with
   ", so it cannot adapt (e.g. strip metadata, re-save as PNG, or skip the
   image and proceed).
2. "继续" replays the failed turn's full message history unchanged. The
   rejected image stays in context, so every resumed first step reproduces the
   exact same provider 400. Because the error is a turn-level abort (not a
   recorded tool error), there is no "recover from the failure" path — the state
   is re-sent as-is and dies at the same point.

Expected behaviour

• A failed tool call (provider rejection of tool output) should be returned to the
  model as a tool_result with is_error: true containing the error message,
  so the model can decide how to recover. Only genuinely unrecoverable conditions
  should abort the turn.
• On "继续"/resume after an interrupted failed tool call, DSH should not blindly
  re-send the failed tool output that the provider already rejected. It should
  either drop/neutralize the failed tool result, surface the previous error to the
  model, or otherwise allow the resumed turn to make a fresh decision instead of
  looping on the identical provider rejection.

Impact

• Any provider rejection of a tool result (image format, oversized content, tool
  schema) turns into a permanent session hang: the model can never recover, and
  "继续" is useless because the failing input is replayed forever.
• In this case the underlying WebP rejection is an LM Studio limitation, but the
  session-death behaviour above is provider-agnostic and is the real bug worth
  fixing independently of the WebP issue.

Suggested area

• Tool-result error propagation (feed provider/tool errors back to the model).
• Resume/replay logic for interrupted turns (drop or neutralize failed tool
  results instead of re-sending them).

[argszero]

Verified the full chain against v0.1.1-rc.2 source (your session-seq evidence is excellent — it made the mechanism unambiguous). The bug is real; I would decompose it into three independent layers, because each has a different fix and the deepest one is a systemic pattern.

L1 — trigger: DSH re-encodes to WebP without checking provider support
The attachment pipeline prefers WebP for alpha/low-colour sources: attachment-local/src/normalization.ts:142-148 (alpha → webp) and attachment-local/src/request-image.ts:91-93 (order ['low-colour:png-webp', 'alpha:webp', 'opaque:jpeg']). Your "PNG with a text comment chunk → WebP" matches the alpha branch. The pi-ai adapter has a rich capability-catalog machinery (reasoning, maxTokens — llm-pi-ai/src/catalog.ts:544-597) but no image media-type capability, and llm-pi-ai/src/context.ts:64 passes the mimeType through verbatim. So the stack emits image/webp with no way for a provider that rejects WebP to be discovered before the request.

L2 — escalation: a provider request error becomes a hard turn abort, not a failed tool result
In agent-loop/src/agent.ts:373-389, finish.kind === 'error' goes through the agent/request-error waterfall; with no registered retry action it throws LlmError (line 387), and the outer catch (lines 302-315) records turn/end reason="error". The model is never handed is_error: true — it gets no chance to react (e.g. re-encode to PNG). The hook exists; the default is "abort".

L3 — replay: "continue" rebuilds the identical poisoned context, so the failure is sticky forever
Every turn rebuilds the request from session.deriveMessages() (agent.ts:340-341), which still contains the rejected WebP tool result. Your turns 3/4/5 (each ~87 minutes apart, identical 400 before any token) are the replay loop.

Family: this is the third member of the sticky turn-end family — #3685 (sticky max-tokens turn-end, agent.ts:285-290/:410), #2821 (unbounded step loop with no max-steps), and now #4048 (sticky error replay). Common gap: a turn-end reason that repeats identically on resume with no context repair. My "snapshot vs live" analysis (#3552/#3565) has the same lesson: when a persisted artifact becomes poisonous, inject a live/fixed source instead of replaying the snapshot.

Fix directions (in increasing depth)

1. L1 root fix: adapter-declared supported image media types; normalization selects from the intersection (WebP only when the provider accepts it, else PNG). Automatic de-poison: on a 400 naming the image part, re-encode that part to PNG and retry once before surfacing the error.
2. L2: convert request-time provider errors attributable to a tool-result part into an injected failed tool result (is_error: true) so the model can self-correct.
3. L3: poison-context eviction — when a turn ends in error with an identifiable offending event, exclude/shrink that event from resume context so "continue" cannot dead-replay.

Immediate workaround: re-encode the image to PNG before read_image (or drop the attachment), so the WebP path never triggers — a new session on the same conversation avoids the replay only if the image is gone from the input.

If you can share whether the 400 arrives at request start (before any chunk, as your inputTokens=0 suggests) vs mid-stream, it would confirm whether L2's fix belongs in the stream-rejection path (agent.ts:354-370) or the finish-classification path — my read is request-start, but your event sequence is authoritative.

[marshfolx]

I'll paste some log from lmstudio so you can spot two behavior:

1. dsh silently convert png to webp.
2. the error 400 from lmstudio ('url' field must be a base64 encoded image)

I'm currently not sure what makes dsh decided to convert format. The sample png was deleted. Perhaps ImageMagick converted ill-formed svg to ill-formed png. Directly call the openai endpoint with the png, bypassing dsh, no error occured. And inpecting the raw data, the file header was truly PNG.

The problem can still be reproduced by just let the model read a webp.

      "role": "assistant",
      "content": "Chrome is blocked by an OS-level sandbox restricti... <Truncated in logs> ...ls composition/centering issues. Let me view it.\n\n",
      "reasoning_content": "Chrome still fails on the same OpenProcess access-... <Truncated in logs> ...then decide.\n\nLet me read the ImageMagick render.\n",
      "tool_calls": [
        {
          "id": "H4FU7fVVaAwYYwko8xKp0O7Q7aWkaezd",
          "type": "function",
          "function": {
            "name": "read_image",
            "arguments": "{\"file_path\":\"C:\\\\Users\\\\chris\\\\source\\\\DS\\\\eval\\\\_im.png\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "content": "<path>C:\\Users\\chris\\source\\DS\\eval\\_im.png</path>... <Truncated in logs> ...c8bd01e60fa84de09f58e43; request image 1600x900px.",
      "tool_call_id": "H4FU7fVVaAwYYwko8xKp0O7Q7aWkaezd"
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "Attached image(s) from tool result:"
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/webp;base64,UklGRi6UAABXRUJQVlA4ICKUAAA... <Truncated in logs> ...AAAAAAAABh76qeqFjm5rEGTR9yb2T15kL/LVUAAAAAAAAAAA=="
          }
        }
      ]
    }
  ],
  "stream": true,
  "stream_options": {
    "include_usage": true
  },
  "store": false,
  "max_completion_tokens": 32000,
  "tools": [
    {
      "type": "function",
      
      ... <Truncated in logs> ...
      
    {
      "type": "function",
      "function": {
        "name": "write",
        "description": "Create or fully replace a UTF-8 text file.",
        "parameters": {
          "type": "object",
          "properties": {
            "file_path": {
              "type": "string",
              "description": "Path to write, resolved by the filesystem backend."
            },
            "content": {
              "type": "string",
              "description": "Full UTF-8 text content to write."
            },
            "sandbox_permissions": {
              "type": "string",
              "description": "The wider sandbox mode this file operation needs. ... <Truncated in logs> ... denied; requires justification and user approval.",
              "enum": [
                "workspace-write",
                "danger-full-access"
              ]
            },
            "justification": {
              "type": "string",
              "description": "Required with sandbox_permissions: one sentence fo... <Truncated in logs> ... this exact file operation needs the wider access."
            }
          },
          "required": [
            "file_path",
            "content"
          ]
        },
        "strict": false
      }
    }
  ]
}
[2026-08-22 23:06:38][ERROR][ornith1.5-9b] 'url' field must be a base64 encoded image

[2026-08-22 23:07:16][DEBUG] Received request: POST to /v1/chat/completions with body {
  "model": "ornith1.5-9b",
  "messages": [
    {
      "role": "system",
      "content": "You are an AI agent powered by DeepSeek Harness.\n\n... <Truncated in logs> ... when unique among the files changed in that turn."
    },
    {
      "role": "user",
      "content": "写个网页，里面用SVG 画个鹈鹕骑自行车的动画，顺带构思一下场景"
    },

... <Truncated in logs> ...

    {
      "role": "tool",
      "content": "returncode: 4294930433\r\nSTDERR: [0822/230626.784:E... <Truncated in logs> ...aunch, self-terminating\r\n\r\nexists: False size: 0\r\n",
      "tool_call_id": "RYfMevKwOvcpC8F71JmSQqYEFDMQKeBy"
    },
    {
      "role": "assistant",
      "content": "Chrome is blocked by an OS-level sandbox restricti... <Truncated in logs> ...ls composition/centering issues. Let me view it.\n\n",
      "reasoning_content": "Chrome still fails on the same OpenProcess access-... <Truncated in logs> ...then decide.\n\nLet me read the ImageMagick render.\n",
      "tool_calls": [
        {
          "id": "H4FU7fVVaAwYYwko8xKp0O7Q7aWkaezd",
          "type": "function",
          "function": {
            "name": "read_image",
            "arguments": "{\"file_path\":\"C:\\\\Users\\\\chris\\\\source\\\\DS\\\\eval\\\\_im.png\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
      "content": "<path>C:\\Users\\chris\\source\\DS\\eval\\_im.png</path>... <Truncated in logs> ...c8bd01e60fa84de09f58e43; request image 1600x900px.",
      "tool_call_id": "H4FU7fVVaAwYYwko8xKp0O7Q7aWkaezd"
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "Attached image(s) from tool result:"
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/webp;base64,UklGRi6UAABXRUJQVlA4ICKUAAA... <Truncated in logs> ...AAAAAAAABh76qeqFjm5rEGTR9yb2T15kL/LVUAAAAAAAAAAA=="
          }
        }
      ]
    },
    {
      "role": "user",
      "content": "读图失败了，继续"
    }
  ],
  "stream": true,
  "stream_options": {
    "include_usage": true
  },
  "store": false,
  "max_completion_tokens": 32000,
  "tools": [
    {

... <Truncated in logs> ...

              ]
            },
            "justification": {
              "type": "string",
              "description": "Required with sandbox_permissions: one sentence fo... <Truncated in logs> ... this exact file operation needs the wider access."
            }
          },
          "required": [
            "file_path",
            "content"
          ]
        },
        "strict": false
      }
    }
  ]
}
[2026-08-22 23:07:16][ERROR][ornith1.5-9b] 'url' field must be a base64 encoded image

[2026-08-22 23:08:46][DEBUG] Received request: POST to /v1/chat/completions with body {
  "model": "ornith1.5-9b",
  "messages": [
    {
      "role": "system",
      "content": "You are an AI agent powered by DeepSeek Harness.\n\n... <Truncated in logs> ... when unique among the files changed in that turn."
    },
    {
      "role": "user",
      "content": "写个网页，里面用SVG 画个鹈鹕骑自行车的动画，顺带构思一下场景"
    },
    
... <Truncated in logs> ...

    {
      "role": "tool",
      "content": "<path>C:\\Users\\chris\\source\\DS\\eval\\_im.png</path>... <Truncated in logs> ...c8bd01e60fa84de09f58e43; request image 1600x900px.",
      "tool_call_id": "H4FU7fVVaAwYYwko8xKp0O7Q7aWkaezd"
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "Attached image(s) from tool result:"
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "data:image/webp;base64,UklGRi6UAABXRUJQVlA4ICKUAAA... <Truncated in logs> ...AAAAAAAABh76qeqFjm5rEGTR9yb2T15kL/LVUAAAAAAAAAAA=="
          }
        }
      ]
    },
    {
      "role": "user",
      "content": "读图失败了，继续"
    },
    {
      "role": "user",
      "content": "继续"
    }
  ],
  "stream": true,
  "stream_options": {
    "include_usage": true
  },
  "store": false,
  "max_completion_tokens": 32000,
  "tools": [
    {
    
... <Truncated in logs> ...

            "justification": {
              "type": "string",
              "description": "Required with sandbox_permissions: one sentence fo... <Truncated in logs> ... this exact file operation needs the wider access."
            }
          },
          "required": [
            "file_path",
            "content"
          ]
        },
        "strict": false
      }
    }
  ]
}
[2026-08-22 23:08:46][ERROR][ornith1.5-9b] 'url' field must be a base64 encoded image