add --model flag and ucode models pin/list/unpin#207
Draft
MarkAngler wants to merge 5 commits into
Draft
Conversation
Model discovery bucketed `system.ai.*` ids into claude/codex/gemini/oss by
matching substrings in the model name. Names don't say which API dialect a
model speaks, so the guess misfired three ways:
- `gpt-oss-120b`/`gpt-oss-20b` matched `"gpt-" in id` and were bucketed as
OpenAI Responses models. They only speak `mlflow/v1/chat/completions`, so pi
registered them under `databricks-openai` pointing at /ai-gateway/codex/v1 —
a route that rejects them. Claude's web_search MCP aimed at the same route.
- `claude-fable-5` matched none of `claude-{opus,sonnet,haiku}-` and vanished.
- `claude_models` is a `{family: id}` map, so only the newest model per family
reached agents that render a full model list. A workspace serving 11 Claude
models showed pi 3.
Every model-service entry already carries `supported_api_types`; the listing
fetched it and threw it away. Bucket by that instead, with precedence — a
Claude model also advertises mlflow chat-completions and would otherwise land
in two buckets. Workspaces whose payload predates the field fall back to the
name rules, with the codex rule tightened to require a version digit after
`gpt-` so gpt-oss is excluded there too.
Also:
- Sort every bucket with `model_version_sort_key`, replacing a lexicographic
reverse sort that ranked `opus-4-8` above `opus-4-10`, and leaving
`codex_models[0]` as the newest rather than the alphabetically first.
- Return `claude_ids` (every Anthropic-dialect id) alongside the family map;
persist it as `claude_model_ids` and register it in pi's models.json.
opencode deliberately keeps reading the family map — it takes `anthropic[0]`
as its default, which widening would silently move onto claude-fable-5.
- `check_gateway_endpoint("codex")` now delegates to `codex.default_model`.
A non-empty list wasn't enough: codex drops ids that don't parse as
`gpt-<version>`, so a gpt-oss-only workspace reported codex available and
then failed at launch with "No models available for codex".
State keys are additive; no STATE_VERSION bump (a mismatch discards every
workspace's state, including managed_configs and provider_services).
Refs databricks#204
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pi persists the model picked in its own selector to settings.json (via `setDefaultModelAndProvider`). ucode overwrote `defaultProvider`/`defaultModel` on every launch *and* every 30 minutes from the token-refresh thread, so the choice could never survive a relaunch — the "ucode just overwrites them" half of databricks#204. `_write_settings` now only writes when there is nothing to preserve or when what is there is ucode-authored residue that no longer resolves: - nothing pinned yet — seed it, so Pi's `findInitialModel` can't fall through to an env-key-backed provider (e.g. HF_TOKEN exposing huggingface). This is the behavior the pin existed for, and it is unchanged. - pinned to a ucode provider we no longer register (`databricks-openai` on a workspace with no Responses models, or a legacy provider name) — repair. - pinned to a ucode provider we do register, but to a model it no longer offers — repair. A pin naming a provider ucode does not manage is left alone: Pi writes defaultProvider only on an explicit user selection, so it is a deliberate choice, not fall-through. The refresh thread now passes `update_settings=False`. It still rewrites models.json every 30 minutes — that rotates the bearer token baked into the file, which is load-bearing — but no longer resets the model. Refs databricks#204 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes the "is there any way for a user to override the models?" half of databricks#204. `resolve_launch_model` already took an `explicit_model`, but both callers passed None, and a flag alone would not have worked: pi/gemini/opencode/copilot all recomputed `default_model(state)` inside `_refresh_token_once`, overwriting the chosen model at launch and again every 30 minutes from the refresh thread. Precedence is explicit `--model` > persisted pin > agent default. - `state.model_overrides` mirrors `provider_services`: additive, per-workspace, no STATE_VERSION bump. `build_agent_state` reports the pin. - `available_models_for_tool` / `ensure_model_available` validate against the freshly discovered lists and name the alternatives in the error. Pi and OpenCode also accept provider-qualified selectors. - `launch(state, tool_args, model)` threads the resolved model through `_refresh_token_once` and `_refresh_forever` for the four refresh-thread agents. codex/claude accept and ignore it so the dispatcher stays uniform. The model is passed as a parameter, never stashed on `state` — `save_state` persists the whole dict and would leak a session-only choice to disk. - `_launch_tool` passes a model to the agent only when the user named one, so a plain `ucode pi` still preserves the model picked in Pi's own selector; a `--model`/pinned launch overrides it. - A stale pin is a hard error at launch — never silently run a different model than the one asked for — but degrades to a warning during configure, since re-running configure is how a user recovers from a rotated-out model. - `ucode models list|pin|unpin`, and `ucode status` now shows the active model, finally delivering its own docstring's promise of "selected models". `--model` is not added to `ucode claude`: Claude Code selects by family env var and ships `/model`, and `ucode claude -- --model X` still passes through. BREAKING: declaring `--model` on codex/gemini/opencode/copilot/pi intercepts a flag that previously reached the agent binary verbatim (asserted by tests/test_cli.py). The escape hatch is unchanged: `ucode opencode -- --model X`. Fixes databricks#204 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
External models (e.g. Azure OpenAI) added as classic serving endpoints are invisible to ucode: they live outside UC model-services and the AI Gateway v2 listing, and the unified gateway 404s on them. They are reachable only via the OpenAI-compatible /serving-endpoints/chat/completions route. Discover them via /api/2.0/serving-endpoints (EXTERNAL_MODEL + llm/v1/chat) into a new external_models bucket, and register a databricks-external provider (openai chat-completions) for pi and opencode; copilot switches its single provider base URL to /serving-endpoints when the selected model is external. Wire the bucket into model availability, pinning, default selection, and discovery diagnostics.
…de, and testing protocols
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #204.
What
Adds a user-facing model override — the "is there any way for a user to override the models?" ask in #204.
Precedence: explicit
--model> persisted pin > agent default.Why a flag alone wasn't enough
resolve_launch_modelalready took anexplicit_model, but both callers passedNone, and a flag by itself would still have been discarded: pi/gemini/opencode/copilot all recomputeddefault_model(state)inside_refresh_token_once, overwriting the chosen model at launch and again every 30 minutes from the refresh thread. This threads the resolved model throughlaunch → _refresh_token_once → _refresh_foreverfor those four agents. The model is passed as a parameter, never stashed onstate(whichsave_statewould persist to disk).Details
state.model_overridesmirrorsprovider_services: additive, per-workspace, noSTATE_VERSIONbump.build_agent_stateanducode statusreport the pin.ensure_model_availablevalidates against the freshly discovered lists and names the alternatives in the error. Pi/opencode also accept provider-qualified selectors.configure(re-running configure is how a user recovers from a rotated-out model).ucode claude: Claude Code selects by family env var and ships/model;ucode claude -- --model Xstill passes through.Declaring
--modelon codex/gemini/opencode/copilot/pi intercepts a flag that previously reached the agent binary verbatim. The escape hatch is unchanged:ucode opencode -- --model X. The passthrough test is updated accordingly.🤖 Generated with Claude Code