opus-4.8 / sonnet-4.6 intermittently omit a required tool-call input field when streaming via native Anthropic /v1/messages

[hooyao]

🏷️ Discussion Type

Bug

💬 Feature/Topic Area

Copilot in GitHub

Body

Summary

When streaming a tool call from claude-opus-4.8 through Copilot's native
Anthropic endpoint (POST /v1/messages, Accept: text/event-stream), the
reassembled tool_use.input intermittently omits a field that the tool's
input_schema declares as required. The input_json_delta.partial_json
fragments that Copilot streams concatenate into JSON that is syntactically
valid but schema-invalid — a required property is simply absent.

For a tool like Anthropic's AskUserQuestion (each questions[] item requires
question, header, options, multiSelect), the streamed input comes back
with the question field missing from every question object, so any strict
schema validator on the client rejects the tool call ("Invalid tool
parameters") and the turn fails.

This is non-deterministic: the same request shape succeeds most of the time and
fails a minority of the time, which points at the model/gateway streaming path
rather than a client bug.

Affected surface

• Endpoint: POST https://api.<enterprise|business|individual>.githubcopilot.com/v1/messages
• Header: Accept: text/event-stream (streaming); also reproduced via the standard VS Code editor headers
• Models observed: claude-opus-4.8 (primary). The same family streams tool calls the same way, so claude-sonnet-4.6 is likely also affected.
• Not observed on non-streaming (stream:false) responses, where tool_use.input arrives as a single complete object.

What I expected

The concatenation of all input_json_delta.partial_json fragments for a
tool_use content block should reproduce a JSON object that satisfies the
tool's declared input_schema, including every required property — exactly as
the first-party Anthropic API does.

What actually happens

The concatenated fragments are valid JSON but missing a required field.
Example (real capture, reassembled from the streamed partial_json fragments;
the tool was AskUserQuestion):

{"questions":[
  {"header":"Database","multiSelect":false,"options":[ ... ]},
  {"header":"Deploy target","multiSelect":false,"options":[ ... ]}
]}

Each questions[] item is missing its question property (required by the
tool's schema). The fragment boundaries show the gap directly — two consecutive
input_json_delta events were:

partial_json = "{\"questi"
partial_json = "ons\": [{\"he"     ←  jumps straight from `[{"`  to  `"he`(ader)

i.e. the "question":"…", key/value pair that should sit between [{ and
"header" is absent from the stream itself.

Frequency (measured)

Across 6 consecutive AskUserQuestion tool calls in one session:

call	questions in call	questions WITH required question field
1	3	2 (one missing)
2	3	3
3	1	1
4	1	1
5	2	0 (all missing → turn rejected)
6	2	2

So roughly 1–2 in 6 calls were affected. A tighter, isolated reproduction (below)
came back clean for 8/8 on one batch — consistent with a low-frequency,
non-deterministic omission rather than a deterministic failure.

Reproduction

Force a tool call with tool_choice so the model must emit AskUserQuestion,
stream the response, and concatenate the partial_json fragments. Repeat ~10–20
times; a fraction will be missing the question field.

curl -N https://api.githubcopilot.com/v1/messages \
  -H "Authorization: Bearer $COPILOT_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -H "Editor-Version: vscode/1.99.0" \
  -H "Editor-Plugin-Version: copilot-chat/0.26.0" \
  -H "Copilot-Integration-Id: vscode-chat" \
  -d '{
    "model": "claude-opus-4.8",
    "max_tokens": 4096,
    "stream": true,
    "tool_choice": {"type":"tool","name":"AskUserQuestion"},
    "tools": [{
      "name": "AskUserQuestion",
      "description": "Ask the user one or more multiple-choice questions.",
      "input_schema": {
        "type":"object",
        "properties": {
          "questions": {
            "type":"array",
            "items": {
              "type":"object",
              "properties": {
                "question": {"type":"string"},
                "header": {"type":"string"},
                "multiSelect": {"type":"boolean"},
                "options": {"type":"array","items":{"type":"object","properties":{"label":{"type":"string"},"description":{"type":"string"}}}}
              },
              "required": ["question","header","options","multiSelect"]
            }
          }
        },
        "required": ["questions"]
      }
    }],
    "messages": [{"role":"user","content":"I am choosing a database and a deploy target for a new web app. Ask me two questions to narrow it down, each with 3 options."}]
  }'

Then, for each response, concatenate every
data: event whose delta.type == "input_json_delta" by their partial_json,
JSON.parse the result, and check that each questions[] item contains a
question key. A subset of runs will be missing it.

Why this is server-side, not a client bug

I isolated the bridge/client out of the loop two ways:

1. Round-trip parser test. Feeding well-formed Anthropic tool-call SSE
   (including fragment boundaries that split mid-key right before question,
   CRLF line endings, and an empty first fragment) through a standard SSE parse +
   re-serialize preserves every field byte-for-byte. The client never drops it
   for well-formed input.

2. Raw byte read. Reading Copilot's response with a raw stream reader
   (no SSE event framing / no reassembly logic at all) and concatenating the
   partial_json fragments straight off the wire still shows the field missing
   on the affected runs. The omission is present in the bytes Copilot emits,
   before any client-side parsing.

Impact

Any client that validates tool_use.input against the tool's declared
input_schema (the Anthropic SDKs and Claude Code do) will reject the entire
tool call when a required field is missing, failing the turn. Because it is
intermittent, it manifests as flaky "Invalid tool parameters" / "tool input did
not match schema" errors that are hard to attribute.

Environment

• Copilot Enterprise, native Anthropic /v1/messages endpoint
• claude-opus-4.8, streaming, tool_choice forcing a specific tool
• Observed 2026-06-05

Suggested fix

Ensure the streaming tool-call generation path emits the complete tool input —
specifically that every property the model produces (and at minimum every
required property per the tool's input_schema) is present in the
concatenated input_json_delta fragments, matching first-party Anthropic API
behavior. If the model occasionally produces incomplete tool JSON, consider
validating/repairing against the declared input_schema before the stream is
finalized, or surfacing a clear stop_reason rather than emitting a silently
incomplete object.

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[hooyao]

Update — more data, and a second (more diagnostic) failure shape

Following up on my own report with a fresh session and full request/response traces. Same tool (AskUserQuestion), same streaming native-Anthropic /v1/messages path, but a distinct corruption morphology that points at the same root cause: the model intermittently mis-serializes the tool input while streaming.

Setup: claude-opus-4.8, streamed, relayed through a byte-level passthrough proxy. The proxy does not parse, re-encode, or reassemble content_block_delta — input_json_delta fragments are forwarded verbatim. Non-streaming is unaffected, consistent with before.

Frequency this session: 3 of 17 AskUserQuestion calls corrupted (all on opus-4.8). Every other tool-input block in the same capture — several hundred calls across ~30 tools (Bash/Read/Edit/Write/Grep/…) — was well-formed. Only AskUserQuestion, the one tool whose input is a nested array of objects, was hit.

Schema: AskUserQuestion.questions is type array.

Healthy output (14/17) — questions is an array, CJK as raw UTF-8:
{"questions":[{"header":"...","question":"...经 Azure...","options":[...]}]}

Corrupted output (3/17) — the whole array is serialized into a string, and inside that string the model hand-encodes CJK as \uXXXX, with exactly one broken escape per response:
{"questions":"[{"header":"...","question":"...\u7 ecf Azure...","options":[...]}]"}

Two independent defects, both server-side:

1. Type: questions arrives as a JSON string, not an array → schema-invalid.
2. Escape: a space is injected into the middle of a \uXXXX escape — \u7ecf ("经") emitted as \u7 ecf — so even a lenient client that tries to re-parse the string fails.

The corruption is inside a single SSE fragment, not a concatenation seam. The on-the-wire input_json_delta that carries it:
partial_json = "...2f'\u7 ecf "
So client-side reassembly is ruled out — the bad bytes are what the endpoint emits. (After manually repairing that one escape, the inner array decodes to fully coherent text, confirming this is a serialization defect, not random garbage.)

Signature: all three breaks land at a CJK→ASCII word boundary, immediately before the ASCII token Azure:

┌─────┬───────────────┬───────────────────┐
│ # │ on the wire │ should be │
├─────┼───────────────┼───────────────────┤
│ 1 │ \u7 ecf Azure │ \u7ecf (经) Azure │
├─────┼───────────────┼───────────────────┤
│ 2 │ \u7 ed5 Azure │ \u7ed5 (绕) Azure │
├─────┼───────────────┼───────────────────┤
│ 3 │ \u7 ed5 Azure │ \u7ed5 (绕) Azure │
└─────┴───────────────┴───────────────────┘

Net effect for the client (Claude Code here): the outer object parses, fails schema (string ≠ array), and surfaces as "Invalid tool parameters."

This is consistent with the original report (intermittent, streaming-only, schema-invalid AskUserQuestion input) and adds a precise trigger: mixed CJK/ASCII content in a nested tool-input field, streamed, occasionally makes the model (a) stringify the nested structure and (b) split a \uXXXX escape at the multibyte→ASCII boundary. The earlier "dropped field" shape and this "stringify + broken escape" shape are two manifestations of the same malformed serialization on the streaming path.

[hooyao]

We've managed to consistently reproduce the "Invalid tool parameters" (stringified array / dropped fields) issue with Claude Opus 4.8 when streaming via the Copilot /v1/messages endpoint.

The issue can be observed using a pure HTTP call. The bug triggers intermittently (but very frequently) when the model needs to generate an array of complex objects inside a tool call during a streaming request.

Reproduction Steps

You can reproduce this by sending a POST request with a complex nested array schema (AskUserQuestion) and forcing the tool call.

1. The curl command:

curl -X POST https://api.githubcopilot.com/chat/completions \
  -H "Authorization: Bearer <YOUR_COPILOT_TOKEN>" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -H "Editor-Version: vscode/1.90.0" \
  -H "Copilot-Integration-Id: vscode-chat" \
  -d '{
    "model": "claude-opus-4.8",
    "max_tokens": 4096,
    "stream": true,
    "tool_choice": {"type": "tool", "name": "AskUserQuestion"},
    "tools": [{
      "name": "AskUserQuestion",
      "description": "Ask the user one or more multiple-choice questions.",
      "input_schema": {
        "type": "object",
        "properties": {
          "questions": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "question": {"type": "string"},
                "header": {"type": "string"},
                "multiSelect": {"type": "boolean"},
                "options": {
                  "type": "array",
                  "items": {
                    "type": "object",
                    "properties": {
                      "label": {"type": "string"},
                      "description": {"type": "string"}
                    }
                  }
                }
              },
              "required": ["question", "header", "options", "multiSelect"]
            }
          }
        },
        "required": ["questions"]
      }
    }],
    "messages": [
      {
        "role": "user",
        "content": "I am choosing a database and a deploy target for a new web app. Ask me two questions to narrow it down, each with 3 options."
      }
    ]
  }'

(Note: Replace the endpoint with the Anthropic-compatible /v1/messages Copilot proxy endpoint if you are testing the native Anthropic shape, the payload remains identical).

2. Expected vs. Actual Observed Output

When watching the input_json_delta events in the SSE stream, instead of incrementally receiving a valid JSON array, the Copilot backend proxy occasionally (about 1 in every 5-10 requests) wraps the entire array inside a JSON string.

Buggy SSE Stream Output (Stringified Array):

data: {"type": "content_block_delta", "index": 0, "delta": {"type": "input_json_delta", "partial_json": "{\"questions\": \"[{\\\"question\\\":\\\"What kind of data structure best fits your web app's needs?\\\",\\\"header\\\":\\\"Data Model\\\"}]\"}"}}

Notice \"questions\": \"[{\\\"question\\\":... — the questions array has been incorrectly serialized as an escaped string.

Because most downstream SDKs and clients strictly validate the incoming JSON against the tool's schema (which expects an array, not a string), this immediately throws an Invalid tool parameters or JSON schema validation error on the client side. It seems the streaming translation layer in the Copilot proxy is occasionally failing to handle deeply nested arrays correctly and falls back to stringifying them.