Skip to content

fix(setup): configure and validate the backend model selector - #44

Merged
lbx154 merged 2 commits into
lbx154:mainfrom
xiuyunMarx:fix/backend-model-selector
Aug 14, 2026
Merged

fix(setup): configure and validate the backend model selector#44
lbx154 merged 2 commits into
lbx154:mainfrom
xiuyunMarx:fix/backend-model-selector

Conversation

@xiuyunMarx

Copy link
Copy Markdown
Contributor

The bug

argus --setup --non-interactive --backend claude completes, argus --doctor prints all blocking checks passed, and then every single message comes back as:

[not dispatched] Manager could not classify this message. No task was queued; please retry.

The model name never reaches the operator. It is only visible in ~/.argus-skill/cost-control.jsonl:

{"provider":"claude","model":"gpt-5.5","run_label":"manager-frontdoor-classify",
 "error":"There's an issue with the selected model (gpt-5.5). It may not exist or you may not have access to it."}

Why it happens

Three independent gaps line up:

  1. Setup never persists a model for agent-CLI backends. persist_validated_profile writes ARGUS_SKILL_RUNNER_BACKEND, ARGUS_SKILL_BACKEND_AUTH_MODE, ARGUS_SKILL_BACKEND_VALIDATED_VERSION — and nothing else. Only the Pi path (_persist_pi_profile) ever wrote a model. Resolution therefore falls through to the shared default gpt-5.5 (core/knobs.py, capability_vault._DEFAULT_TEXT_MODEL), an OpenAI-catalog id. Per docs/backend-providers.md, Argus sends it to claude verbatim.

  2. Readiness never validates the selector. It checks binary, version and auth. The only model validation was _check_pi_model_routing, gated on profile.backend == "pi". Reproduced on a working install:

    $ ARGUS_SKILL_MODEL=gpt-5.5 argus --doctor
    ✓ ARGUS-BACKEND-001 [backend/ready] claude 2.1.232 ... (subscription_cli; configuration checked; ...)
    all blocking checks passed          # exit 0, on a machine where every call fails
    
  3. docs/agent-install.md has no step that configures a model. An agent following it end to end hits every checkpoint the doc defines and reports success truthfully.

The generic error text comes from life/router.py, which collapses any non-zero backend exit into failure_sink("classifier backend failed"); webapi/manager_dispatch.py then fails closed.

The change

  • persist_validated_profile(report, *, model="") — persists the model adopted for the backend. New default_model_for_backend() returns an id only when the backend has a verified default and the operator chose nothing, so re-running setup never retunes a hand-configured machine.
  • Both setup paths seed the adopted model into the environment before readiness runs — mirroring how the existing Pi branch already seeds ARGUS_SKILL_MODEL — then persist it.
  • _check_backend_model_catalog() rejects a model id that positively matches a foreign vendor catalog. Severity splits on who chose it, matching the Pi precedent: an id nobody chose is a hard problem (Argus's own default, deterministic failure); an id the operator set by hand is a warning (a private gateway may really serve it). An id matching no known catalog is never second-guessed.
  • The doctor prints readiness warnings instead of discarding them, so an advisory is visible before the first call rather than after.
  • docs/agent-install.md gains a "Confirm the model selector" step using argus --config-help (verified to exist and to print each knob's resolved value and source).

Exempt by design: copilot and qoder (their CLIs front several catalogs — Copilot resells Anthropic ids), pi/opencode (provider-agnostic; Pi keeps its real catalog probe), and dsh (its selector goes through the ARGUS_DSH_* overlay, so judging the bare id would misfire).

Only claude → claude-opus-5 is seeded, because that is the one id verified against a real CLI here. Other backends keep current behaviour and now get an actionable failure instead of a silent one; adding a default later is a one-line table entry.

Tests

tests/core/test_backend_readiness.py — the gpt-5.5-on-claude regression, an accepted Anthropic id, operator-chosen foreign id warns instead of failing, unknown ids stay silent, multi-catalog backends exempt, and the adopted-model persistence.

Two pre-existing test doubles stubbed persist_validated_profile positionally and were widened to accept the new keyword. test_grok_readiness_accepts_api_key_without_spending_a_turn configured no model at all, so under a hermetic home it resolved to the OpenAI default — exactly what this change now rejects; it sets a grok- id, which is what a real Grok install carries.

tests/core/test_backend_readiness.py tests/tools/test_setup_readiness.py tests/test_doctor.py   → all pass
python -m ruff check argus_skill tests                                                          → All checks passed

ruff also caught that the new interactive-setup line used a backslash escape inside an f-string, which is a syntax error on the project's 3.11 floor; it now uses literal glyphs like the surrounding code.

Note for the maintainer

tests/core/test_release.py::test_release_manifest_matches_current_shipped_source fails, because compute_source_digest covers argus_skill/**/*.py and generate_manifest refuses to update without build_release rebuilding the shipped frontends. I did not commit a frontend rebuild to keep this diff reviewable — please regenerate the manifest as part of the release flow.

Pre-existing failures unrelated to this change, confirmed by running them on a clean main: tests/tools/test_ppt_master.py (5) and tests/webapi/test_pairing.py (5).

🤖 Generated with Claude Code

`argus --setup --backend claude` persisted only backend, auth_mode and
validated_version, so model resolution fell through to Argus's shared
default `gpt-5.5` — an OpenAI-catalog id the Claude CLI rejects. Readiness
checked that the CLI existed, ran and was authenticated, but never that the
model id was one that CLI could serve, so `argus --doctor` reported "all
blocking checks passed" on a machine where every call failed. The front
door surfaces any non-zero backend exit as "[not dispatched] Manager could
not classify this message", so the model name never reached the operator.

Only the Pi path ever persisted a model; every other backend persisted none.

- setup adopts a backend-appropriate model when the operator chose none,
  seeding it before readiness runs (mirroring the existing Pi branch) and
  persisting it with the validated profile
- readiness rejects a model id that positively matches a foreign vendor
  catalog, hard-failing an id nobody chose and warning on one the operator
  set by hand (a private gateway may really serve it)
- the doctor now prints readiness warnings instead of discarding them
- docs/agent-install.md gains a model-selector step; an installing agent
  following it previously had no step that configured a model at all

Backends whose CLI fronts several catalogs (copilot, qoder, pi, opencode)
and dsh (whose selector goes through the ARGUS_DSH_* overlay) are exempt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Configures backend-specific model defaults during setup and validates model/backend catalog compatibility during readiness checks.

Changes:

  • Persists adopted models and validates catalog compatibility.
  • Displays readiness warnings through doctor diagnostics.
  • Documents model-selector verification and adds regression tests.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
argus_skill/core/backend_readiness.py Adds model adoption, validation, and persistence.
argus_skill/tools/setup.py Seeds adopted models before readiness checks.
argus_skill/webapi/diagnostics.py Exposes readiness warnings.
docs/agent-install.md Documents model verification.
tests/core/test_backend_readiness.py Tests model validation and persistence.
tests/tools/test_setup_readiness.py Updates setup test doubles.
Suppressed comments (1)

argus_skill/core/backend_readiness.py:404

  • A persisted value is no longer evidence that a human chose it: persist_validated_profile() now writes setup's adopted model. After setting up Claude, then setting up Codex, claude-opus-5 is read here as chosen, the foreign-catalog mismatch is downgraded to a warning, and setup succeeds while retaining that model because persisted writes merge. Codex calls then fail. Track whether setup owns the persisted model (and replace or hard-fail it on a backend switch) separately from genuinely operator-set values.
        chosen = _explicit_model_selection(
            role_env, env=env_map, persisted=persisted_map
        )
        if chosen:

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +256 to 261
_MODEL_ROLES: tuple[tuple[str, str], ...] = (
("manager", "ARGUS_SKILL_MANAGER_MODEL"),
("planner", "ARGUS_SKILL_PLAN_MODEL"),
("engineer", "ARGUS_SKILL_ENGINEER_MODEL"),
("reviewer", "ARGUS_SKILL_REVIEWER_MODEL"),
)
Comment thread docs/agent-install.md Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@lbx154
lbx154 merged commit 93c904b into lbx154:main Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants