Skip to content

fix(python): isolate per-entry parse failures in list_models() (#1302)#1303

Open
jrob5756 wants to merge 2 commits into
github:mainfrom
jrob5756:fix/list-models-per-entry-resilience
Open

fix(python): isolate per-entry parse failures in list_models() (#1302)#1303
jrob5756 wants to merge 2 commits into
github:mainfrom
jrob5756:fix/list-models-per-entry-resilience

Conversation

@jrob5756
Copy link
Copy Markdown

Fixes #1302 (option 2 — per-entry isolation hardening).

Summary

CopilotClient.list_models() parses the models.list response with a
list comprehension:

models = [ModelInfo.from_dict(model) for model in models_data]

A single malformed entry raises and the whole call fails — the cache stays
empty, so every retry from a fresh session fails identically. The original
multiplier-required ValueError from #1302 is already fixed in
main, but the bot that triaged the issue endorsed implementing
option 2
as a follow-up so future schema drift on individual models can't take down
list_models() for every consumer.

What changed

python/copilot/client.py — wrap each ModelInfo.from_dict(model) in
try/except inside the RPC path:

  • Log a WARNING (with model id when available) for each malformed entry.
  • Skip and continue parsing the rest.
  • Custom on_list_models handlers are unaffected.

Docstring updated to document the new resilience behavior.

Why Python only

  • Node.js casts the response (as { models: ModelInfo[] }) — no eager parse.
  • Go uses json.Unmarshal — missing fields become Go zero values.
  • .NET / Rust rely on their own deserializers with similar semantics.

Only the Python SDK has a hand-rolled per-entry parser that raises on
missing fields, so per-entry isolation is the Python-specific gap.

Tests

New TestListModelsParserResilience class in python/test_client.py with 4 unit tests:

  1. test_skips_malformed_entry_and_returns_valid_ones — mixed payload returns only the valid models, with one warning per skipped entry.
  2. test_all_malformed_returns_empty_list_without_raising — all-broken payload returns [] instead of raising.
  3. test_empty_models_payload_returns_empty_list — empty payload still returns [].
  4. test_non_dict_entry_is_skipped_with_warning — non-dict entries (string, int) are skipped without crashing.

Tests use a small _FakeRpcClient stub so they don't need to spawn the CLI subprocess.

Validation

$ pytest test_client.py
56 passed
$ ruff check copilot/client.py test_client.py
All checks passed!
$ ruff format --check copilot/client.py test_client.py
2 files already formatted

ty check reports the same 37 pre-existing diagnostics on
main and on this branch — no new type errors introduced.

Compatibility

  • ✅ Backwards compatible — return type is still list[ModelInfo].
  • ✅ No public API changes.
  • ✅ Custom on_list_models handlers behave identically.
  • ✅ Cache semantics unchanged: a successful (possibly partial) RPC response
    is still cached; the cache is still cleared on disconnect.

Wrap each ModelInfo.from_dict(model) call in a try/except so a single
malformed entry in the models.list response (e.g. backwards-incompatible
schema drift on one model) is logged at WARNING level and skipped,
instead of raising and taking down list_models() for every consumer.

Custom on_list_models handlers are unaffected; only the RPC path is
hardened.

Adds unit tests covering: skip-and-return-valid, all-malformed-empty-list,
empty-payload, and non-dict entries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 15, 2026 14:02
@jrob5756 jrob5756 requested a review from a team as a code owner May 15, 2026 14:02
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Hardens the Python SDK’s CopilotClient.list_models() parsing so a single malformed model entry in the models.list JSON-RPC response won’t fail the entire call (and leave the cache empty), addressing the resilience gap described in #1302.

Changes:

  • Update CopilotClient.list_models() to parse model entries one-by-one, logging a warning and skipping malformed entries instead of failing the whole call.
  • Update the list_models() docstring to document the new “skip malformed entries” behavior.
  • Add unit tests validating partial/malformed payload handling (including non-dict entries) without spawning the CLI.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
python/copilot/client.py Implements per-entry try/except parsing in list_models() and documents the new resilience behavior.
python/test_client.py Adds targeted unit tests using a fake RPC client to verify malformed entries are skipped with warnings.

Comment thread python/copilot/client.py Outdated
Per review on PR github#1303 — `except Exception` is too broad; it masks
unexpected programmer errors in ModelInfo.from_dict (or its helpers)
by silently skipping entries.

- Pre-check isinstance(model_data, dict) before parsing, with its own
  warning for non-dict entries.
- Narrow the parse-failure handler to (ValueError, TypeError,
  AssertionError) — the shape-related exceptions ModelInfo.from_dict
  and its helpers actually raise. Other exceptions propagate.
- Add test_unexpected_exceptions_propagate covering the new behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

list_models() fails for ALL consumers when any model omits ModelBilling.multiplier (currently every named model)

2 participants