Skip to content

fix(copilot): tolerate SDK metadata parsing errors when listing models#193

Merged
jrob5756 merged 1 commit into
mainfrom
fix/copilot-sdk-modelbilling-error
May 15, 2026
Merged

fix(copilot): tolerate SDK metadata parsing errors when listing models#193
jrob5756 merged 1 commit into
mainfrom
fix/copilot-sdk-modelbilling-error

Conversation

@jrob5756
Copy link
Copy Markdown
Collaborator

@jrob5756 jrob5756 commented May 15, 2026

Summary

Workflows that configure reasoning.effort (or workflow-wide
runtime.default_reasoning_effort) on the Copilot provider were failing with:

Dialog turn failed: Missing required field 'multiplier' in ModelBilling

after exhausting the retry loop. The bug affects every named Copilot model
(see "Blast radius" below) — it just wasn't caught earlier because few
workflows exercise the reasoning-effort path on Copilot.

Root cause

github-copilot-sdk 0.3.0 eagerly parses every entry in its
models.list response with dataclass from_dict helpers. The
ModelBilling.from_dict parser
(client.py:496-507)
treats multiplier as required and raises
ValueError("Missing required field 'multiplier' in ModelBilling") if it's
absent — and none of the currently-served models include it. Because the
parsing is done in a list-comprehension, a single malformed entry kills the
whole list_models() call, and the SDK's response cache is only populated
after a successful parse, so every call from a fresh session fails the same way.

Both _validate_reasoning_effort_for_model (called before every
create_session when reasoning.effort is set) and get_max_prompt_tokens
caught only (TimeoutError, ProviderError, OSError, RuntimeError) at the
SDK boundary, so the ValueError leaked through. In the reasoning-effort
path it then poisoned the retry loop and surfaced as the user-visible
Dialog turn failed: ... error. (get_max_prompt_tokens was rescued by
the engine's outer except Exception at
engine/workflow.py:1071,
so context-window metadata was silently unavailable rather than fatal.)

Blast radius

Inspected the live response on this machine — 18 of 19 returned models ship
a billing object without multiplier:

claude-sonnet-4.6, claude-sonnet-4.5, claude-haiku-4.5,
claude-opus-4.7, claude-opus-4.7-1m-internal, claude-opus-4.7-high,
claude-opus-4.7-xhigh, claude-opus-4.6, claude-opus-4.6-1m,
claude-opus-4.5, gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.3-codex,
gpt-5.2-codex, gpt-5.2, gpt-5-mini, gpt-4.1

The only "clean" entry is auto, which has billing: None so the parser
short-circuits.

Fix

Broaden both catches to Exception (still excludes
asyncio.CancelledError / KeyboardInterrupt / SystemExit, all of
which are BaseException subclasses) and document the SDK schema-failure
case at both call sites. This matches the existing docstring contract:
"capability metadata must never block a workflow that the SDK might
otherwise accept."

The narrow tuple was written before anyone realized the SDK would raise
non-OSError exceptions in normal operation.

Tests

  • Replaced test_unexpected_exception_propagates (which asserted the
    opposite of the docstring contract) with
    test_value_error_from_sdk_parser_returns_none, using the exact
    upstream error message for clarity.
  • Added test_value_error_from_sdk_parser_skips_validation covering the
    reasoning-effort path: confirms the workflow proceeds normally and
    that reasoning_effort is still forwarded to create_session.
  • Full suite green: 2579 passed, 15 skipped, 29 deselected in 7m36s.

Note

This is an internal-model regression on the Copilot SDK side, not worth
filing upstream — the missing field is a temporary backend quirk and the
SDK will likely make multiplier optional in a future release. Our fix
hardens the code path regardless.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Copilot SDK >=0.3.0 eagerly parses every model in its `models.list`
response with dataclass `from_dict` helpers. When any required field is
missing the parser raises `ValueError` and the entire listing fails.
`claude-opus-4.7-1m-internal` ships a `billing` object without the
required `multiplier` field, which makes `ModelBilling.from_dict`
raise `ValueError("Missing required field 'multiplier' in
ModelBilling")`.

Both `get_max_prompt_tokens` and `_validate_reasoning_effort_for_model`
caught only `(TimeoutError, ProviderError, OSError, RuntimeError)` at
the SDK boundary, so the `ValueError` leaked through and surfaced as
`Dialog turn failed: ...` after exhausting the retry loop. This was
particularly disruptive for workflows that configure
`reasoning.effort` on the Copilot provider, since the validation step
runs before `create_session` and blocks every agent invocation.

Broaden both catches to `Exception` (still excludes
`asyncio.CancelledError`/`KeyboardInterrupt`/`SystemExit`) and
document the rationale at both call sites. The behavior matches the
existing docstring contract: "capability metadata must never block a
workflow that the SDK might otherwise accept."

Updated `test_unexpected_exception_propagates` (which asserted the
opposite contract) into `test_value_error_from_sdk_parser_returns_none`
and added `test_value_error_from_sdk_parser_skips_validation` covering
the reasoning-effort path with the exact upstream error message.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jrob5756 jrob5756 force-pushed the fix/copilot-sdk-modelbilling-error branch from aae8854 to 9b11908 Compare May 15, 2026 13:45
@jrob5756 jrob5756 merged commit 46d1a14 into main May 15, 2026
9 checks passed
@jrob5756 jrob5756 deleted the fix/copilot-sdk-modelbilling-error branch May 15, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant