[aw-failures] Copilot BYOK 100% red — SDK driver sends OpenAI `stream_options` to `type=anthropic` provider, upstream 400 kills [Content truncated due to length]

[github-actions]

Executive summary

Fix the Copilot BYOK request builder — it is injecting the OpenAI-only stream_options field into type=anthropic provider calls, which return HTTP 400 and kill the agent before it emits anything. This is a deterministic P0 that already reds at least two production scheduled workflows and is untracked. Smoke CI startup EACCES (#42398) and Codex invalid-model (#41787) also recurred in this window but are already tracked and still open.

Failure cluster table

Cluster	Sev	Dominant signature	Workflows	Representative / comparator runs	Tracking
C1 Copilot BYOK stream_options 400	P0	[sdk-driver] error: 400 stream_options: Extra inputs are not permitted (all 4 retries, 0 turns, empty output)	Daily Safe Output Integrator, PR Triage Agent	§28614701337 / §28613537448	NEW → this issue
C2 Smoke CI startup EACCES	P0	EACCES mkdir /tmp/gh-aw/sandbox/firewall/logs, agent never invoked	Smoke CI	§28612922406	#42398 (open)
C3 Codex invalid model	P1	isInvalidModelError=true — invalid/unsupported model configuration, not retried	Daily Cache Strategy Analyzer	§28614062344	#41787 (open)
C4 Smoke CI cancelled	—	run cancelled/superseded, not a genuine failure	Smoke CI	§28612868592	n/a

Scope note: the deterministic pre-fetch detailed 5 of 39 reds in the 6h window. C1 is on the default Copilot engine + resolved claude-sonnet-4.6 BYOK path, so it likely accounts for a large share of the undetailed reds.

Evidence — C1 (audit of representative runs)

Copilot SDK BYOK stream_options 400 — audited 28614701337 & 28613537448

Both runs resolve the same custom provider and fail identically on every retry:

[copilot-harness] sdk-mode: custom provider resolved from awf-reflect (provider=copilot type=anthropic baseUrl=(apiproxy/redacted) model=claude-sonnet-4.6)
[copilot-harness] copilot-sdk driver mode: BYOK provider resolved (baseUrl=(apiproxy/redacted) type=anthropic model=claude-sonnet-4.6)
[copilot-sdk-driver] [sdk-driver] sending prompt...
[copilot-sdk-driver] [sdk-driver] error: 400 400 400 stream_options: Extra inputs are not permitted
[copilot-harness] attempt 1..4 failed: exitCode=1 failureClass=partial_execution ... permissionDeniedCount=0 hasNumerousPermissionDenied=false

• agent_output.json = {"items":[],"errors":[]} — 0 turns, 0 tool calls, 0 safe outputs.
• All 4 retries fail identically ⇒ deterministic, not transient/rate-limit (isRateLimitError=false, isAuthError=false, isModelNotSupportedError=false).
• stream_options is an OpenAI streaming-API field (stream_options.include_usage); it is not valid in the Anthropic Messages API, so the type=anthropic upstream rejects the request with 400.
• Trailing chroot-home / firewall-logs EACCES cleanup warnings in the raw tail are post-run cleanup noise, unrelated to the 400 that actually failed the run.

Existing-issue correlation

• C2 → [aw-failures] Smoke CI hard-red at startup — EACCES mkdir /tmp/gh-aw/sandbox/firewall/logs, agent never invoked (rootless left [Content truncated due to length] #42398 (Smoke CI hard-red, EACCES mkdir firewall/logs): reproduced again at 18:31 UTC today (§28612922406). Not fixed — kept open.
• C3 → [aw-failures] Daily Cache Strategy Analyzer ~50% red — Codex pinned to alpha snapshot gpt-5-codex-alpha-2025-11-07 that 404s on [Content truncated due to length] #41787 (Daily Cache Strategy Analyzer, Codex pinned alpha model): reproduced at 18:50 UTC today (§28614062344). Not fixed — kept open.
• No open agentic-workflows issue matches C1's stream_options 400 signature (distinct from [aw-failures] Smoke Copilot Sub Agents 100% red — Copilot SDK returns "No model available (policy enablement)" for gpt-5.3-codex [Content truncated due to length] #42824 model-policy, [aw-failures] Copilot CLI false-red — runs marked failure (exit 1) after safe-outputs succeed, via "numerous permission denied" [Content truncated due to length] #41636 permission-denied false-red, [aw-failures] Copilot CLI 15-min hard timeout — agents thrash without emitting safe-outputs (Test Quality Sentinel, Impeccable S [Content truncated due to length] #42974 15-min timeout).
• No issues closed — no fresh evidence any tracked failure is resolved.

Fix roadmap

• P0 — C1 (new): Gate/strip stream_options in the Copilot SDK BYOK request path when provider type=anthropic. Tracked in the sub-issue below.
• P0 — C2: Already tracked in [aw-failures] Smoke CI hard-red at startup — EACCES mkdir /tmp/gh-aw/sandbox/firewall/logs, agent never invoked (rootless left [Content truncated due to length] #42398 (rootless mkdir of firewall/logs). No new action.
• P1 — C3: Already tracked in [aw-failures] Daily Cache Strategy Analyzer ~50% red — Codex pinned to alpha snapshot gpt-5-codex-alpha-2025-11-07 that 404s on [Content truncated due to length] #41787 (Codex alpha model 404). No new action.

Sub-issues created

• C1 fix — Copilot BYOK stream_options stripping for type=anthropic (see linked sub-issue).

References:

• https://github.com/github/gh-aw/actions/runs/28614701337
• https://github.com/github/gh-aw/actions/runs/28613537448
• https://github.com/github/gh-aw/actions/runs/28612922406

Generated by 🔍 [aw] Failure Investigator (6h) · 140.7 AIC · ⌖ 34.3 AIC · ⊞ 5.2K · ◷

• expires on Jul 9, 2026, 11:21 AM UTC-08:00

[github-actions]

WHM Update — 2026-07-03T05:41Z

P0 confirmed active. This is the highest-priority systemic issue tracked today.

Jul 3 Status

• BYOK workflows continue to fail deterministically due to stream_options injection
• Affects all type=anthropic provider calls via SDK driver
• Action needed: Fix the BYOK SDK request builder to omit stream_options when provider type ≠ openai

Flagged by Workflow Health Manager · Run §28640918061

Generated by 🏥 Workflow Health Manager - Meta-Orchestrator · 75.6 AIC · ⌖ 5.4 AIC · ⊞ 7.8K · ◷