ENG-736 - Delete model_prefixes: route models off the harness's own fetched-or-shipped list

[druks-operator]

Linear ticket: ENG-736

Plan

ENG-736 — Delete model_prefixes; route models off HarnessSettings.allowed_models

Problem

Model→harness routing still runs off Harness.model_prefixes — a hand-maintained tuple that has to be edited every time a provider invents a naming scheme (e.g. o1/o3/o4). Since ENG-701 landed, HarnessSettings.models_fetched / allowed_models already holds the provider's own selectable list (fetched on connect and refreshed by the RefreshModels cron), and HarnessSettings.allowed_models falls back to each harness's shipped models tuple when the fetch hasn't landed. That row is already the single source of truth — the prefix tuple is a redundant, drifting second one.

Approach

Route directly off HarnessSettings.allowed_models, delete Harness.model_prefixes and Harness.has_model outright, and collapse both user_settings/routes.py validators onto get_harness_for_model so they stop calling has_model themselves. Bare harness names as model strings ("claude", "codex") keep working via the model == row.name arm. The pricing-only startswith inside codex.py::_rate_for (line ~232) is deliberately left alone — it maps a model to a _Rate, not to a harness. The consequence — a brand-new provider model is unroutable (HarnessError) until the next RefreshModels (0 6 * * *) or a reconnect — is accepted per the ticket; the manual-refresh endpoint / picker button follow-up is explicitly out of scope.

Files to change

backend/druks/harnesses/base.py

• Remove the model_prefixes: ClassVar[tuple[str, ...]] declaration (currently line 76) and the classmethod has_model(cls, model) (lines 90–94).
• Trim the surrounding comment block above the classvars (lines 68–83) so it no longer references model_prefixes or "namespace" — the identity/config seeds now enumerated are provider, models, default_model, model_discovery_url, plus the effort/timeout defaults.

backend/druks/harnesses/claude.py

• Remove model_prefixes = ("claude-",) on line 39. No other change to this file.

backend/druks/harnesses/codex.py

• Remove model_prefixes = ("gpt-", "o1", "o3", "o4") on line 274. No other change to this file.
• Do not touch the startswith fallback inside _rate_for (~lines 230–233); it is pricing, not routing.

backend/druks/harnesses/registry.py

• Rewrite get_harness_for_model verbatim to the shape the ticket pins:

  from druks.user_settings.models import HarnessSettings
  
  
  def get_harness_for_model(model: str) -> type[Harness]:
      for row in HarnessSettings.all():
          if model == row.name or any(m["id"] == model for m in row.allowed_models):
              return row.harness
      raise HarnessError(f"no harness runs model {model!r}")

• Keep get_harnesses and get_harness unchanged (both are still used and don't need routing state).
• Import HarnessSettings inside the function (or at module top level — either is fine here since user_settings.models already imports druks.harnesses.registry.get_harness at method-call time via a property, so a module-level import creates no cycle at import time; verify by running the test suite).

backend/druks/user_settings/routes.py

• update_harness_settings (line 65): replace the harness.has_model(updates["model"]) check with a get_harness_for_model call. Reject with the existing 422 shape when the model does not resolve to this specific harness:

  if "model" in updates:
      try:
          resolved = get_harness_for_model(updates["model"])
      except HarnessError:
          resolved = None
      if resolved is not harness:
          raise HTTPException(
              status_code=422,
              detail=f"{updates['model']!r} is not a {harness.name} model.",
          )

  Preserve the exact detail string (f"{updates['model']!r} is not a {harness.name} model.") so callers relying on the 422 shape don't churn.
• _validate_model (line 119): replace the any(harness.has_model(value) for harness in get_harnesses()) check with a get_harness_for_model call, keeping the existing 422 detail (f"No installed harness runs model {value!r}."):

  def _validate_model(value: str | None) -> None:
      if value is None:
          return
      try:
          get_harness_for_model(value)
      except HarnessError:
          raise HTTPException(
              status_code=422,
              detail=f"No installed harness runs model {value!r}.",
          ) from None

• Add the HarnessError import from druks.harnesses.exceptions; drop the now-unused get_harnesses import if nothing else in the module needs it (currently only these two validators call it).

backend/tests/test_harness_model_routing.py

Rework end-to-end against the seeded HarnessSettings rows the session-scoped _test_schema fixture already provides (see backend/tests/conftest.py and backend/druks/bootstrap.py::seed_harnesses). The rework must cover, at minimum:

• Fetched list wins. Mutate the seeded claude row's models_fetched to a list like [{"id": "claude-fable-5", "label": "Claude Fable 5"}], then assert get_harness_for_model("claude-fable-5") is ClaudeHarness. Also assert that a namespace-shaped string absent from that fetched list (e.g. "claude-opus-5") no longer routes — this locks in the accepted consequence (namespace pass-through dies).
• Shipped-tuple fallback. Leave the seeded codex row's models_fetched unset (or explicitly None) and assert get_harness_for_model(CodexHarness.models[0]) returns CodexHarness — that is, allowed_models falls back to the shipped tuple when nothing has been fetched. Do the same for a claude row that has never been fetched (freshly reset for this test).
• Bare harness names still resolve. Assert get_harness_for_model("claude") is ClaudeHarness and get_harness_for_model("codex") is CodexHarness regardless of models_fetched.
• Miss raises. Assert get_harness_for_model("llama-3-70b") raises HarnessError, and specifically that an old-style namespace guess ("gpt-6", "o4-mini", "claude-opus-5") not present in either the fetched list or the shipped tuple also raises — the whole point of the change.

The old test_new_model_in_known_namespace_routes_without_a_release case is removed as part of the rework (its premise — pass-through by namespace — is exactly what this ticket kills). Use the transaction-rollback fixture already in conftest.py; the mutations don't need to be committed.

Verification

Run the full verification profile against the changed backend:

• uv run ruff check backend
• uv run ruff format --check backend
• uv run pyright
• uv run pytest backend/

The frontend commands in the profile aren't touched by this change but stay in the profile for completeness. GitHub CI runs the same commands.

Acceptance Criteria

• ac-no-model-prefixes: The model_prefixes class variable is removed from Harness and from every subclass (no model_prefixes remains in the backend).
  • Verification: grep -rn "model_prefixes" backend/ returns no matches.
• ac-no-has-model: Harness.has_model and all references to it are removed from the backend.
  • Verification: grep -rn "has_model" backend/ returns no matches.
• ac-registry-shape: backend/druks/harnesses/registry.py::get_harness_for_model resolves via HarnessSettings.all(), matching model == row.name or an entry in row.allowed_models by m["id"], and raises HarnessError on miss.
  • Verification: Read backend/druks/harnesses/registry.py and confirm the body iterates HarnessSettings.all() with the specified match arms and the HarnessError(f"no harness runs model {model!r}") raise; no startswith and no has_model call remains in this file.
• ac-update-harness-validator: In backend/druks/user_settings/routes.py, update_harness_settings rejects a model that does not resolve (via get_harness_for_model) to the harness whose row is being patched, returning HTTP 422 with detail f"{model!r} is not a {harness.name} model." — the existing shape.
  • Verification: Read backend/druks/user_settings/routes.py::update_harness_settings; confirm it calls get_harness_for_model (not has_model), rejects when the resolved harness is not harness, and preserves the 422 detail string. Backend tests around the settings PATCH endpoint (backend/tests/test_api_settings.py, or a new/updated case in test_harness_model_routing.py if none is present) pass.
• ac-validate-model-collapse: In backend/druks/user_settings/routes.py, _validate_model no longer calls has_model; it wraps get_harness_for_model in a HarnessError guard and returns HTTP 422 with detail f"No installed harness runs model {value!r}." when the model is unroutable.
  • Verification: Read backend/druks/user_settings/routes.py::_validate_model and confirm it uses get_harness_for_model inside a try/except HarnessError and raises the specified 422; grep -n "has_model" backend/druks/user_settings/routes.py returns no matches.
• ac-pricing-untouched: The pricing prefix match in backend/druks/harnesses/codex.py::_rate_for (around line 232) still contains a startswith fallback over _DEFAULT_RATES — pricing routing is not touched by this PR.
  • Verification: grep -n "startswith" backend/druks/harnesses/codex.py shows the _rate_for startswith(known) line still present; the surrounding function body is unchanged.
• ac-tests-reworked: backend/tests/test_harness_model_routing.py is rewritten to drive get_harness_for_model through seeded HarnessSettings rows, and covers: (a) a model listed via a mutated models_fetched on the claude row routes to ClaudeHarness; (b) a fresh row whose models_fetched is unset falls back to the shipped tuple (a CodexHarness.models / ClaudeHarness.models entry still routes); (c) bare harness names "claude" and "codex" still resolve; (d) an unlisted model (e.g. "llama-3-70b") and an old-style namespace guess absent from both fetched and shipped lists (e.g. "claude-opus-5", "gpt-6", "o4-mini") raise HarnessError.
  • Verification: Read backend/tests/test_harness_model_routing.py and confirm each of the four cases is present; uv run pytest backend/tests/test_harness_model_routing.py passes.
• ac-lint: Lint passes.
  • Verification: uv run ruff check backend exits 0 and uv run ruff format --check backend exits 0.
• ac-typecheck: Type check passes.
  • Verification: uv run pyright exits 0.
• ac-tests: Backend test suite passes.
  • Verification: uv run pytest backend/ exits 0.

[czpython]

Superseded by #82, which delivered ENG-736 and is merged (520425b). This was the first attempt; its run ended before the work gate, leaving the PR as a draft.