Skip to content

fix(*): keep migration notices off stdout, and stop planting defaults - #346

Merged
arelchan merged 4 commits into
mainfrom
fix/config_notices_and_doctor_fix
Aug 18, 2026
Merged

fix(*): keep migration notices off stdout, and stop planting defaults#346
arelchan merged 4 commits into
mainfrom
fix/config_notices_and_doctor_fix

Conversation

@arelchan

Copy link
Copy Markdown
Contributor

Summary

Three follow-ups from #341, which fixed a retired contextWindowTokens: 65536 capping every upgraded install at 64k. Two are its loose ends; the third is the reason it happened at all.

The notice no longer prints on stdout. #341 drains config-migration notices in the run() wrapper so every command tells the user, which caught the two whose stdout is a machine-readable document: raven doctor --json and raven import --json, both advertised for automation. Reproduced -- a config still carrying the retired pin makes doctor --json emit JSON with a sentence after it, and jq refuses the result. Moved to stderr, where the rest of this class of message already goes (the ConfigReadError branch a few lines up in the same wrapper, and the loader's malformed-config warning). That fixes any future --json command without it having to know this exists, which allow-listing the two would not. The window is narrow -- the migration fires once per install -- but the run it can spoil is a first run after upgrading, and a CI job is exactly the caller not watching.

raven doctor gains a Config section and a --fix. The premise changed while building it: the migrations run at load, so by the time this command looks, nothing is pending -- they already happened, silently, one launch ago. What is left for a person to ask about is what Raven deliberately will not decide for them. So it reports a contextWindowTokens pinned below what the configured model holds, and names what rides on it: the per-turn history budget, when the Curator starts paying for a slow path, and when memory consolidation archives. --fix removes the pin.

Reported without --fix, never written: that pin is a real configuration for a local model or an endpoint served smaller than the catalogue thinks, and the flag is the user saying which case theirs is. The write carries the file mode across (config.json holds providers.*.apiKey) and uses a per-PID temp name -- the same rules the loader's migration writer follows.

One check rather than two. A config naming no provider is the other thing worth asking about, but it reads differently on either side of the explicit-provider rule in #284, so it belongs to that change; a model that routes to nothing is already reported above it.

The bootstrap stops writing every default into the user's config. This is the cause of the bug #341 cured. save_config dumped the whole Config(), so a new install started life with eight kilobytes of settings its owner had never heard of and, worse, with that day's defaults frozen into their file -- after which a default we improve never reaches anyone who already has one. contextWindowTokens: 65536 got onto disk exactly this way.

It now writes only what differs from the defaults. Lossless on reload, since a value equal to its default reloads as that default. A fresh config drops from 7,985 bytes to 2, and to 634 once the extension blocks are seeded -- what is left is the handful of lines someone actually chose, which is also a config they can read.

That third one is the only change here with a broad blast radius, and it has one visible consequence, which is an improvement: the onboarding picker used to lead with the schema's default model, because the bootstrap had written it into the config and the wizard read it back as "the current model" -- a value nobody had chosen, offered as though somebody had. With nothing written, the wizard leads with the provider's own recommendation instead. The test that pinned the old list now pins the new one and says why.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

pytest tests/                       6650 passed, 1 failed, 33 skipped
                                    (tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare,
                                    which fails identically on github/main when its file runs
                                    together and passes alone on both)
ruff check raven/ tests/            All checks passed
ruff format                         clean

Driven by hand as well as by tests, each against a throwaway home:

doctor --json on a fossil config   stdout parses as JSON again; the notice is on stderr
doctor    (pin 65536, model 1M)    reports the pin and what is sized against it
doctor --fix                       pin removed, file still 0600, a second run reports nothing
save_config on a fresh Config()    2 bytes; 634 after the extension blocks are seeded

New cases cover the stderr split, both doctor paths including the mode preservation, and the save round-trip keeping every value the user chose.

  • Relevant tests pass locally
  • Relevant lint / type checks pass locally
  • User-facing docs or screenshots are updated when needed

Risk

The bootstrap change alters what every new install's config file looks like: nearly empty instead of a full dump. Existing configs are untouched -- nothing rewrites them -- and both shapes load identically, so the risk is confined to anyone reading config.json as documentation of what can be set. raven doctor and the schema remain the answer to that question.

The onboarding picker's default moves from the schema default to the provider's recommendation, as described above.

doctor --fix writes only when asked, and only removes a key. Everything else here is output routing.

Rollback: revert the commits. Nothing persists a new format, so a reverted build reads every config written under this one.

Related Issues

N/A

arelchan and others added 3 commits August 18, 2026 14:34
Review on #341 caught this after it landed. The notice is drained in the
`run()` wrapper, which means it prints after every command -- including
the two whose stdout is a machine-readable document: `raven doctor --json`
and `raven import --json`, both advertised for automation. Reproduced: a
config still carrying the retired pin makes `raven doctor --json` emit
JSON with a sentence after it, and `jq` refuses the result.

stderr is where the rest of this class of message already goes -- the
`ConfigReadError` branch a few lines up in the same wrapper, and the
loader's malformed-config warning. Moving it there fixes both commands and
any future `--json` one without either having to know this exists, which
allow-listing the two would not.

Narrow but sharp: the migration fires exactly once per install, so the run
it can spoil is a first run after upgrading -- and a CI job is precisely
the caller that will not be watching.

The stale section comment the same review flagged is corrected too: the
watermark moved to a sidecar before that PR merged, and the comment still
described it living in the config.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
The context-window migration only runs at load, and only once. That leaves
no way to ask "does my config still carry something that caps me?" -- by
the time anyone wonders, the migration has either already run or decided
not to, silently, one launch ago.

So `raven doctor` grows a Config section. It reports a
`contextWindowTokens` pinned below what the configured model actually
holds, and says what rides on it: the per-turn history budget, when the
Curator starts paying for a slow path, and when memory consolidation
archives. `--fix` removes the pin, so the window follows each model again.

Reported without `--fix`, never written. That pin is a real configuration
for a local model or an endpoint served smaller than the catalogue thinks;
the flag is the user saying which of the two cases theirs is.

The write carries the file mode across (config.json holds
providers.*.apiKey) and uses a per-PID temp name, the same rules the
loader's migration writer follows.

One check, not two: a config naming no provider is the other thing worth
asking about, but it reads differently on either side of the
explicit-provider rule, so it belongs to that change. A model that routes
to nothing is already reported above it.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
The bootstrap dumped the whole `Config()` to disk, so a new install
started life with eight kilobytes of settings its owner had never heard
of -- and, worse, with today's defaults frozen into their file. A default
we improve later then never reaches anyone who already has one.

That is not a hypothetical cost. `contextWindowTokens: 65536` got onto
disk exactly this way, and every upgraded install stayed capped at 64k on
a 1M-token model until #341 went and took it back out. The migration was
the cure; this is the cause.

`save_config` now writes only what differs from the defaults. It stays
lossless on reload -- a value equal to its default reloads as that default
-- and a fresh config drops from 7,985 bytes to 2, with the seeded
extension blocks bringing it to 634. What is left is the handful of lines
a person actually chose, which is also a config they can read.

One visible consequence, and it is an improvement: the onboarding picker
used to lead with the schema's default model, because the bootstrap had
written it into the config and the wizard read it back as "the current
model" -- a value nobody had chosen, offered as though somebody had. With
nothing written, the wizard leads with the provider's own recommendation.
The test that pinned the old list now pins the new one and says why.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
@claude

claude Bot commented Aug 18, 2026

Copy link
Copy Markdown

Claude finished @arelchan's task in 4m 36s —— View job


Review complete

  • Read existing comments/reviews on this PR
  • Review diff against origin/main
  • Check correctness (bugs, edge cases, error handling, back-compat)
  • Check repo conventions (commits, branch naming, uv, test naming, CONTEXT-MAP)
  • Check tests for weakening/skipping
  • Post final findings

Blocking: the network-call-in-a-zero-network-path issue flagged in the earlier review on this PR is still unresolved. The latest commit (9617194, pushed after that review) only adds test coverage and mocks resolve_context_window in every new test — it doesn't pass allow_fetch=False at raven/cli/doctor_commands.py:187, so plain raven doctor / raven doctor --json on a config with an explicit contextWindowTokens pin against a model LiteLLM's static table doesn't know (the exact case this feature targets — self-hosted/custom endpoints) still falls through to a live OpenRouter HTTP call, contradicting this module's documented "zero-network, millisecond-fast" default mode. Not re-filing it as a new inline comment since it's already on the thread, but it stands as-is.

Everything else I checked is solid:

  • save_config's move to exclude_defaults=True is safe — confirmed no other caller (onboard_commands.py, tui_rpc/methods/config.py) depends on a full dump; the extension-block seeding uses its own setdefault-based raw-JSON writer, untouched by this change.
  • The stderr move for migration notices reuses the file's existing Console(stderr=True) pattern (already used a few lines up) and is well-tested (stdout/stderr split, no-notice case).
  • _inspect_config_health / --fix / _write_config_preserving_mode: the raw-dict pop, mode-preservation, and per-PID temp file all check out against the loader's established rules; the read-only-home and unstat-able-mode failure paths are exercised and degrade correctly (report-and-continue, never crash).
  • Commit messages are Conventional-Commits compliant, ASCII-only, with Co-authored-by trailers; branch name matches <type>/<snake_desc>; all test edits are on pre-existing files (test_cli_doctor_commands.py, test_cli_helpers.py, test_config_loader.py, test_cli_onboard_commands.py), no new test files. No large/asset files added.
  • No test was weakened to go green — the onboarding default-model assertion changed because the underlying wizard behavior legitimately changed (documented in the commit body), and it still asserts a specific list/default rather than loosening to something permissive.

One new nit posted inline (dead/untested branch in the Config-section renderer).
· branch fix/config_notices_and_doctor_fix

model = defaults.model

if pinned and model:
real = resolve_context_window(model)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocker] This breaks the module's own "zero-network" contract for plain raven doctor (see the docstring at the top of this file: "Default mode is zero-network, millisecond-fast"; _gather_static_checks is explicitly "Strictly zero-network").

resolve_context_window(model) defaults to allow_fetch=True, and unlike resolve_max_output_tokens (used a few lines up in _gather_static_checks, which only ever consults LiteLLM's static table), resolve_context_window falls through to _lookup_openrouter_entry_fetch_openrouter_models, which does a live httpx GET against OpenRouter (10s timeout) whenever the model isn't in LiteLLM's static table and no fresh disk/in-process cache exists.

_inspect_config_health is called unconditionally in register() whenever report.config_loaded is true — not gated behind --probe — so any config with an explicit contextWindowTokens pin now makes plain raven doctor (and raven doctor --json, the automation path this PR is otherwise protecting) attempt a network call. Worse, the case this feature is built for — a self-hosted/local model or custom endpoint "served smaller than the catalogue thinks" — is exactly the kind of model id LiteLLM's static table won't have, so it's the most likely to hit the network fallback. In a sandboxed/no-egress CI runner this adds a multi-second stall to what's documented as a millisecond-fast, offline check before silently degrading.

Every other synchronous/latency-sensitive caller in the repo passes allow_fetch=False for this reason (e.g. raven/agent/loop/main.py:369,738,1048). Suggest the same here: resolve_context_window(model, allow_fetch=False).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirming this independently -- it reproduces, and the failure is not only the theoretical contract break.

With agents.defaults.model = "openrouter/acme/nonexistent-model-xyz" and contextWindowTokens: 32768 in a throwaway home, _inspect_config_health(load_config(cfg), fix=False) with httpx.Client swapped for a recorder:

network calls attempted: [('GET', 'https://openrouter.ai/api/v1/models')]

So plain raven doctor -- no --probe -- reaches _fetch_openrouter_models, and that client is built with timeout=10.0. Offline or behind a firewall, the documented millisecond-fast default mode now blocks for up to ten seconds, and --json is the flag advertised for CI.

Reachable whenever the pin is set and the LiteLLM tier misses, which is exactly the self-hosted / newer-than-the-table case the finding text is written for. Not every model: _lookup_openrouter_entry returns early unless the id starts with openrouter/.

Worth adding for whoever fixes it: allow_fetch=False costs nothing in coverage here. It also turns off the LiteLLM import, but _gather_static_checks has already called resolve_max_output_tokens(defaults.model) by this point, so LiteLLM is in sys.modules and _try_litellm_context_window(allow_import=False) proceeds exactly as it would with True. The only tier lost is the one that should not be running.

CI's diff-coverage gate came back at 79% against a 90% threshold, naming
thirteen lines: both failure branches of the `--fix` writer, and the
renderer that prints the Config section.

The renderer being untested is the one worth calling out. Every assertion
so far was on `_inspect_config_health`, which decides what to say -- so
the whole block that actually says it could have been deleted with the
suite still green, and a finding nothing prints is a finding the user
never gets. The new cases drive it both ways: with a fix pending (the
section names it and advertises `--fix`) and with one applied (it says
what it did and stops advertising).

The two error branches are covered as the situations they are: a
read-only home leaves the pin in place, says so, and lets the rest of the
report print; a `stat` that will not answer does not stop the fix from
being written.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
health = report.config_health
if health and (health.findings or health.applied):
console.print("\n[bold]Config[/bold]")
for line in health.findings:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] line.startswith(" ") (and the else branch that strips/re-indents it) has no producer: every finding built by _inspect_config_health — the pin message and f"could not write the fix: {exc}" — is a plain, unindented string, so this branch is dead and untested. Either drop the conditional or wire an actual indented-finding case if one is coming.

@arelchan
arelchan requested a review from 0xKT August 18, 2026 07:51

@gloryfromca gloryfromca left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: the zero-network break already flagged at doctor_commands.py:187 reproduces -- plain raven doctor issues a live GET to openrouter.ai. Repro and a note on why the fix is free are in that thread. Everything else here I would merge as it stands.

Covered: the full diff against origin/main, the callers of what it changes (save_config's only real caller, read_raw_or_raise, the loader's own migration writer, tui_rpc.methods.config), backward compatibility of the on-disk config, AGENTS.md conventions, and the tests for weakening.

What I checked and am satisfied with

  • save_config(exclude_defaults=True): the blast radius really is one caller. onboard_commands._bootstrap_empty_config is the only one, and only when the file is absent; every other config write goes through the surgical update_* helpers. The round trip is lossless in the way that matters -- the schema's only model_fields_set consumer is should_warn_deprecated_memory_window, and memory_window is Field(exclude=True), so it never survived a dump anyway. tui_rpc reads raw keys and falls back to its own _DEFAULTS, but its keys (agent.temperature, tui.*) are not schema paths, so a thinner file changes nothing there either.
  • The onboarding picker change is a genuine behaviour change, and the test was repinned with a reason rather than loosened. anthropic/claude-sonnet-5 is the registry's own default_model for that provider (providers/registry.py:322), so leading with it is the right answer, not an accident of the diff.
  • The stderr move follows the pattern already in the file (_helpers.py:311 writes the config-path notice to stderr), and the two new cases assert both directions -- stderr carries it, stdout is empty.
  • Conventions: commits are conventional, ASCII-clean, and carry Co-authored-by with a real model id; the branch name fits section 2.1; no new test files, and the touched ones are the right three per section 5.4.
  • Tests, run here rather than taken on faith: tests/test_cli_doctor_commands.py tests/test_cli_helpers.py tests/test_config_loader.py tests/test_cli_onboard_commands.py -> 360 passed, 0 skipped. The other four files that touch save_config (status / agent / gateway / cli_config_precedence) -> 59 passed. Nothing skipped, xfailed, or env-gated.

I also agree with the dead-branch nit at line 582: _inspect_config_health is the only producer of findings, and neither of its two strings is indented, so that else cannot fire.

Two nits of my own inline. And one forward-looking note that is not a finding: a user who legitimately pins a smaller window now gets the ! on every raven doctor run with no way to say "yes, I meant it". Acceptable while the section holds one check, but if it grows, an acknowledge path will be wanted before the section starts being ignored.

import os as _os

tmp = path.with_name(f"{path.name}.doctorfix.{_os.getpid()}")
tmp.write_text(_json.dumps(raw, indent=2, ensure_ascii=False), encoding="utf-8")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Unlike the loader's writer this says it mirrors, this one does not unlink the temp file when the write fails. That one ends except OSError: ... tmp.unlink(missing_ok=True) (loader.py:258-261); here nothing cleans up.

Concrete: on ENOSPC or a quota, tmp.write_text creates the file, fails partway, and the caller's except Exception turns it into a "could not write the fix" finding -- leaving ~/.raven/config.json.doctorfix.<pid> behind holding a partial copy of the config. The mode is the part that stings: chmod runs after the write, so the orphan keeps the default umask (0644) while containing providers.*.apiKey, sitting next to a config.json the user may have tightened to 0600 -- which is the exact exposure the docstring above exists to prevent. A failing os.replace leaves the same thing, fully written.

tmp.unlink(missing_ok=True) in an except OSError: that re-raises closes it, and makes the "same rule the loader's migration writer follows" claim true.

Comment thread raven/config/loader.py
"""
Save configuration to file.

Only what differs from the defaults is written. A dump of everything is

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] This makes a sibling comment false. _persist_migrations still justifies itself with "save_config is no use here either -- it dumps every default (~8 KB), re-planting the very kind of fossil this migration exists to pull out" (lines 224-226), and after this change it dumps no defaults at all.

The paragraph's other reason still holds and is now the whole justification: the mapping load_config migrated has the extension blocks popped, and a config that needed no edit must stay byte for byte. So this is a one-line trim, not a rethink. Left as is, the next reader takes a present-tense comment as documenting current behaviour and walks away with the wrong picture of what save_config writes -- the same drift the previous commit in this branch fixed for the watermark comment.

@arelchan
arelchan merged commit 5a0c950 into main Aug 18, 2026
18 checks passed
@arelchan
arelchan deleted the fix/config_notices_and_doctor_fix branch August 18, 2026 10:55
arelchan added a commit that referenced this pull request Aug 18, 2026
Resolving the conflict with the merged #346 by keeping both sides left the
two test groups touching. Formatting only.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
arelchan added a commit that referenced this pull request Aug 18, 2026
Resolving the conflict with the merged #346 by keeping both sides left the
two test groups touching. Formatting only.

Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
arelchan added a commit that referenced this pull request Aug 19, 2026
## Summary

The model was two attributes on a process-wide `AgentLoop`, so there was
one answer for everyone: two sessions could not run different models, a
switch in one moved all of them, and the "default" was whatever the last
switch happened to write to `agents.defaults`. This makes the model a
property of the conversation, and -- following the same thread down --
makes the credential that serves it something the user says rather than
something Raven infers.

A `ModelBinding` is a model id, the credential that serves it, and how
much that model can hold, as one value. `ProviderPool` is the single
place deciding which credential a model id pairs with, cached per
(vendor, model). `run_turn` resolves the binding for the turn's session
and holds it in a `ContextVar` for the whole turn tree; the loop's
`provider` / `model` / `context_window_tokens`, the context engine's
LLM-backed segments, the skill gate and rewriter, and the consolidator
all read that instead of a reference of their own.

| Rule | How it lands |
|---|---|
| Different sessions, different models | A dict of overrides, read once
at turn entry |
| A switch moves only the session that asked | Nothing else reads that
session's entry |
| A new session starts on the configured default | A session-scoped
switch does not write `agents.defaults` |
| A configured subsystem uses its own model+credentials; otherwise it
follows the conversation | The factory resolves each pin through the
pool, so a holder has either a complete pair or nothing |
| A switch mid-turn takes effect next turn | Free -- the turn holds the
binding it entered on |

Detached work inherits the context copy `asyncio` makes at task
creation, so a subagent finishes on the model it was spawned under.

### The window belongs to the binding

Sizing came to a head during the rebase. `refresh_context_window` and
its cascade assumed the loop has one current model -- which this PR
makes false, since two sessions can be on a 200k and a 1M model at once,
and one int on the loop cannot answer for both. That is the same
mis-sizing #341 removed, arriving through another door.

So the window moved onto `ModelBinding`, beside the credential, and
every holder reads the binding of the turn it is running under.
`refresh_context_window` is gone: a turn now enters on a binding that
already knows its own size, so there is nothing to refresh.

It resolves on first read rather than at construction, because building
a provider is what imports LiteLLM -- a lazily built one has not done it
yet, so an eager window is the catalogue-miss default for every model,
which is the defect `on_built` existed to paper over. A miss is
deliberately not cached, so a cold read cannot make the process wrong
for its lifetime. An explicit `contextWindowTokens` travels on every
binding, so a pinned number survives whatever model a session switches
to.

### The provider is a word the user says

A model id does not name whose credential serves it. `openrouter`
serving `anthropic/claude-haiku-4-5` and `anthropic` serving
`claude-haiku-4-5` are both real, name different keys, bill different
accounts, and look identical on the wire. Everything that used to fill
in the blank was guessing which account pays.

`agents.defaults.provider: "auto"` was the same guess at the config
layer, and it did not detect anything: a prefixed id was answered by the
provider it names, but a bare id fell to keyword matching in `PROVIDERS`
order and took the first configured claimant -- so with anthropic and
openrouter both keyed, `gpt-4.1` went to openrouter over openai because
openrouter sits third in that list and openai eighth.

- `/model <id>` is refused, and the refusal names the two ways out:
`/model` for the picker, or `/model <provider> <id>`. A prefixed id is
refused too -- a prefix is LiteLLM routing syntax, not evidence about a
credential.
- `config.set model` refuses without a provider, which is the boundary
where the rule can be enforced rather than asked politely.
- `raven provider use` requires `--provider`, and names it alongside
`raven provider list` in the error.
- `--default` is not an exception: changing what new sessions start on
is the same choice about the same thing.
- `agents.defaults.provider` carries no default, and a one-shot
migration resolves each pre-rule config through the old derivation and
writes the answer into it. Behaviour is unchanged by construction -- the
value written is what that config was already getting -- but it stops
being inferred. It reuses the sidecar watermark from #341, with a floor
per migration rather than one shared mark, so raising the generation for
this one does not re-open the previous one. A config the derivation
cannot answer for is left blank rather than filled with a vendor chosen
to have something there.

The picker already asked in this order -- provider, then model, with a
free-text row for an id no catalogue lists -- so it needed nothing. Both
help strings now carry the usage, because a rule the user meets first as
an error is a rule we chose not to tell them.

With nothing left deriving, `providers/pin.py` is deleted: its whole
subject was "which provider to pin when the user changed the model
without naming one", and its last caller was the onboarding wizard,
which asks for the provider before the model and therefore always had
one in hand.

### Config defaults

No subsystem ships a vendor default. `context.curator_model`,
`token_wise.tool_result_lifecycle.summary_model` and
`skill_forge.detect_model` all hardcoded the same Gemini id, and
`token_wise.smart_routing.tiers` shipped six models across three vendors
-- for users who may hold no key for any of them. All are unset now,
which is what "not configured" has to mean for the subsystem rule to be
expressible.

`context.curator_model` and `skill_forge.llm_gate_model` took a model id
and nothing else, so the pool had to guess which credential served it.
Each pin now takes a provider alongside the model (`curator_provider`,
`llm_gate_provider`). With the provider named, a vendor whose
credentials are unusable is logged and dropped rather than silently
borrowing the conversation's key. With it unset the pin still binds: a
configured gateway takes it, because a gateway serves whatever id it is
handed under its own credential, and only without one is the vendor
derived from the id. That branch is unchanged from main and is
documented as it behaves rather than as the rule would prefer -- an
earlier revision of this description and of `CONTEXT.md` claimed the
drop applied there too, which was wrong.

**Release note.** With `context.curator_model` unset, the Curator's slow
path runs on the conversation's model instead of failing on a Gemini id
nobody had a key for and dropping to the deterministic plan. That is the
rule working as asked, but on a long conversation it is up to 12
tool-calling requests per turn of context housekeeping that previously
cost nothing. Set `context.curator_model` (and `curator_provider`) to a
small model to keep it cheap.

### Review findings

**Round 1 (pre-rebase), five findings.** A pin naming a vendor with no
spec crashed the agent at startup -- fixed on main, which now passes the
provider name separately instead of reading it off a spec that may be
None; verified with the reporter's own repro. `/model` before the first
message no longer manufactures a zero-message session record
(`session.create` is lazy, and `session.title` guards the identical case
the same way). Every rpc call in the TUI's session commands has a
`.catch` -- without it, `/model` typed before the first `session.create`
resolves wrote a raw stderr line over the Ink render while the
transcript showed nothing. The remaining two are the picker's provider
row and the gateway pin; see below.

**Round 2, three before-merge findings, all reproduced here before being
fixed.**

- The migration watermark was one high-water mark shared by every
stamped migration, so raising it to 2 for the provider migration re-ran
the context-window one on every config already stamped at 1 -- deleting
a `contextWindowTokens: 65536` a user had put back by hand after our own
notice invited them to, and printing that invitation again. Each
migration now has its own floor. The test that should have caught it
built its precondition from `CURRENT_CONFIG_VERSION`, so it moved with
the bump; it now pins the literal a shipped build wrote, and two tests
cover the upgrade-from-an-older-generation path in both directions.
- Deleting `_adopt_provider` took its `_image_tool_result_ok.clear()`
with it. Both capability caches key on a model id but are computed from
the provider, so an `apiBase` repointed at a box with different
capabilities, or a re-authenticated provider, kept the old endpoint's
verdict for the life of the process -- images silently dropped from tool
results with nothing in the log. Both binding setters now clear them.
- `--default` had no working path through the TUI. The picker footer
taught `/model <name> --default`, which is the bare-id spelling the
parser refuses; the refusal dropped the flag from the suggestion it told
the user to follow; and `/model --default` opened the picker, whose
selection sent a session-scoped switch that looked like it had changed
the default. The flag now rides into the overlay state and back out
through the picker's callback, the refusal keeps it, and the footer
writes the spelling that works.

**The two carried from round 1 are fixed here rather than dissolved.**
An earlier revision of this description said the work had dissolved
them; it had not, and both were still reproducible.

- `model.options` re-derived the provider from the id. For a vendor with
a spec that now reads the prefix `stored_model_id` writes, but
`find_by_model` answers None for a passthrough vendor (mistral, xai), so
the picker starred `agents.defaults.provider` for a session running on
someone else's key -- and the starred row is exactly what a user reads
to answer whose key is paying. It now reads the head of the stored id,
and stars nothing when that names no configured provider.
- The gateway pin is documented as it behaves; see the Config defaults
section above.

**Round 3, after the rebase onto a main that gained #346.** The conflict
was additive and is
described in the thread; two follow-on commits closed the two
nonblocking notes that survived it.
`modelPicker.test.tsx` was rendering `<ModelPicker>` without the `scope`
prop this branch made
required, so the `scope === 'default'` footer had no coverage -- and
`tsconfig.json` excludes
`src/__tests__`, which is why `tsc --noEmit` was clean and silent about
it. And `onModelSelect`, the
callback that carries `--default` back out of the picker, had no test;
the rule it applies is now its
own function, `modelSelectCommand`, because what a selection becomes on
the command line has to
satisfy three separate constraints in `parseModelArg` at once.

**Also taken, each a defect in what this PR itself added or removed:**
`clear_session_binding` was gated on `SessionManager.delete` returning
True, which it does not for a session that switched model before its
first save, so the entry leaked for the life of the process;
`_migrate_auto_provider` validated its probe against a strict `Config`
before the shims that relocate legacy top-level blocks, so a config
still carrying `skillRouter` failed the probe, was skipped silently, and
was stamped anyway -- never retried, on exactly the oldest configs most
likely to still say `auto`; `providers/lazy.py` and
`AgentLoop.set_provider` carried docstrings naming methods and callers
this PR deletes; onboarding still wrote `provider="auto"` when clearing
a removed provider's default; and `ui-tui/package-lock.json` is reverted
to main, its only change being npm-version drift in `libc` metadata.

## Type

- [ ] Fix
- [x] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
pytest tests/                     6700 passed, 33 skipped, 13 deselected   (TERM=dumb)
ruff check / format --check       clean, 832 files formatted
npx tsc --noEmit (ui-tui)         clean
npm test (ui-tui, vitest run)     998 passed (86 files)
npm run lint:rpc                  generated.ts in sync
commitlint + check_commit_messages.py   clean over github/main..HEAD
```

`tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare` is
`TERM`-dependent, not
ordering-dependent: it fails under an interactive `TERM` and passes
under `TERM=dumb`, which is what
the `coverage` target pins and what CI runs. That file is not touched by
this PR.

Every fix in rounds 2 and 3 was mutation-checked rather than assumed:
reverting the migration floor,
the cache clear, the flag pass-through, the passthrough-vendor fallback,
the picker's `scope` prop
and the `--default` command-building each turns the corresponding new
test red.

Driven by hand in an isolated home against the built TUI bundle, not
only by tests: `/model
claude-opus-4-5` refused with the two ways out; `/model
openrouter/anthropic/claude-opus-4-5` refused
as well; `/model openrouter claude-opus-4-5` applied; `/model` opening
the picker at the provider
level. A real config carrying `provider: auto` and a bare
`claude-opus-4-5` migrated to `provider:
openrouter` -- the vendor it was already resolving to -- with the notice
printed and the watermark
written to the sidecar.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

User-visible behaviour changes, all deliberate:

`/model <id>`, `config.set model` without a provider, and `raven
provider use` without `--provider` now fail where they used to guess.
That is the point, but it is a workflow change for anyone who typed a
bare id -- the errors name the fix, and the picker needs nothing new
learned.

`agents.defaults.provider` is written into every config that did not
have one, on the first launch after this lands. The value is what that
config was already resolving to, so no request changes vendor; what
changes is that it is now visible and editable.

With `curator_model` unset out of the box, the Curator's slow path
starts actually running (see the release note above). Cost, not
correctness.

One contract tightening is not backward compatible on its own:
`config.set key="model"` now rejects a call with no `provider`, and the
`-32009 model_switch_in_turn` code is removed from the error table. The
in-repo TUI moves with it, but any client outside this repo that sends a
bare model id starts failing on the first request after this lands.

Rollback: revert the commits. The config keeps an explicit `provider`,
which every published build accepts and treats exactly as it treats a
hand-written one. The `config.migrations.json` sidecar is left behind
and every build since #341 does read it, so a reverted build sees
`{"version": 2}` and treats the config as fully migrated -- which is
correct, since it is. Delete the sidecar to force the migrations to be
reconsidered.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A

---------

Co-authored-by: arelchan <204152633+arelchan@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
arelchan added a commit that referenced this pull request Aug 19, 2026
## Summary

Six follow-ups from #284's review rounds. Five are defects in what that
PR itself added or removed and were deliberately deferred rather than
rushed into a diff already under review; the sixth is a check #346 wrote
down as belonging here, because it only becomes a real state once the
provider is explicit.

**A session's model now survives a restart on every surface, not only in
the TUI.** The read half of the persistence hung off `session.resume`,
so only the surface that calls that handler got it: a conversation
arriving from a channel came back on the configured default with its own
choice sitting unread in its own session record. The read moves onto the
loop, where every entry point already goes, and the handler's duplicate
copy is gone -- one reader instead of two. `has_session_binding`
restores first as well, or a session that switched before a restart
reports as having inherited the default and the picker stars the wrong
row. `clear_session_binding` counts as having consulted the record, so
returning a session to the default is not undone by the next read.

**`raven provider use` refuses an unroutable provider before it
writes.** A typo exited 0 with the config already changed, then printed
a credential warning suggesting `raven provider set <typo> --api-key
...` -- advice that would have created a section for a vendor nothing
routes to. `provider set` already applied the right test before writing
a section; it is exposed as `ensure_routable_provider` and called first,
which is the order the TUI's `/model` already used. Only an unroutable
*name* is refused: a known provider with no credentials yet is still
written and reported, because picking a model before configuring its key
is a normal order to do things in. "No spec of ours" cannot be the typo
test either -- Raven carries none for mistral or xai, and refusing on it
would block every passthrough vendor the config documents as supported.

**The guard that keeps a model and its provider paired has a home
again.** A whole-repo AST scan failing any `set_default_model` without
`provider=`, and any raw `agents.defaults.model` write not paired with a
provider write, lived in `tests/test_provider_pin.py` and was deleted
with that file. Only half its message was obsolete; the rule it enforces
became more central, and it is the only place a machine checks it.
`test_provider_resolution_invariants.py` is where the other source
guards of this kind already live.

**The binding cache is keyed on everything it caches.** The fingerprint
covered `config.providers`, but `bind` also copies `contextWindowTokens`
onto the binding, so editing that number left a stale binding answering
with the old one until a restart -- the "I changed it and nothing
happened" shape the window ladder exists to prevent, arriving through
the cache instead.

**A gateway call is priced from the gateway's own table.** Who routes a
request is not who bills for it. LiteLLM answered first for every id,
including one naming a gateway, and the two disagree: measured on the
pinned LiteLLM, `openrouter/z-ai/glm-4.6` is filed at 0.40/1.75 per
million where OpenRouter's own current table says 0.50/2.00, so every
such call was reported a fifth under its real cost. OpenRouter's
catalogue now answers ahead of LiteLLM for ids that name it, fresh-only
and never fetching: an expired copy is not better evidence than the
router's table and would suppress the refetch the expiry exists to
trigger, while a blocking fetch would be paid by the turn this runs
after. Unchanged for a direct id, which is what once priced a
self-hosted deployment at a hosted model's rate.

**`raven doctor` can ask about a provider nothing resolved.** #346
deferred this with its reason: before the explicit-provider rule, the
field defaulted to `auto` and blank never happened. Now blank means the
load-time migration tried the derivation and had no answer. Two findings
-- blank, and a name nothing routes to -- both report-only. Neither is
offered to `--fix`, and that is not an omission: only the user knows
which vendor they meant to pay, and the migration has already tried.
Guessing again under the name `--fix` would be the same guess wearing
more confidence.

## Type

- [x] Fix
- [ ] Feature
- [ ] Docs
- [ ] CI / tooling
- [ ] Refactor
- [ ] Other

## Verification

```
pytest tests/                     6477 passed, 43 skipped, 13 deselected   (TERM=dumb)
ruff check / format --check       clean, 832 files formatted
npx tsc --noEmit (ui-tui)         clean
npm test (ui-tui)                 998 passed (87 files)
commitlint + check_commit_messages.py    clean over github/main..HEAD
npm run lint:rpc                  generated.ts in sync
commitlint + check_commit_messages.py   clean over github/main..HEAD
```

Every fix is mutation-checked rather than assumed -- reverting it turns
the new test red. That is worth naming for two of them, where writing
the test taught something the fix alone did not:

- The doctor check's scope is narrower than it looks. With a configured
vendor that serves the model, the migration fills the blank during
`load_config` and there is nothing left to report, so the fixture has to
be genuinely unresolvable -- which is exactly when the finding is real.
- The first attempt at the pricing tier read the any-age cache and made
a stale disk copy outrank a refetch.
`test_expired_disk_triggers_refetch` caught it, which is why the tier is
fresh-only.

Three review rounds landed on this PR after it opened, each fixed and
each verified rather than
taken from the report:

- **Blocking:** the doctor provider check read a leftover `auto` as an
unroutable vendor name. That
is the legacy state the check exists for -- `_migrate_auto_provider`
leaves the literal in place
when it cannot resolve one -- so the user was told to fix a typo they
did not make. It now matches
`Config._match_provider`'s own `forced != "auto"` test, and says the
word is retired rather than
merely ignoring it. Neither neighbouring test used a literal `auto`,
which is how the branch
  between them went unentered.
- `_restore_attempted.add` in `clear_session_binding` was inert, and its
stated reason did not hold
for the only caller: `session.delete` unlinks the record before the
clear runs. Dropped rather than
relocated -- marking in `set_session_binding` would claim the record had
been consulted when it had
not. The test that was supposed to guard it is renamed and its docstring
now scopes what it does
  and does not pin.
- `_routes_anywhere` caught `Exception` where the sibling check catches
`KeyError`, and the new
refusal printed a different failure marker from the eleven others in its
file.

- [x] Relevant tests pass locally
- [x] Relevant lint / type checks pass locally
- [ ] User-facing docs or screenshots are updated when needed

## Risk

Two user-visible changes, both deliberate:

`raven provider use` now exits 1 on a provider name nothing routes to,
where it used to exit 0 with the config already written. A script
passing a misspelled `--provider` starts failing; a script passing a
real one is unaffected, including for a passthrough vendor.

Cost figures change for calls made through OpenRouter on ids where its
catalogue and LiteLLM's copy disagree. The new number is the one the
gateway bills, so this corrects a systematic under-report rather than
introducing a new figure -- but a user comparing today's total to
yesterday's will see the step.

Rollback: revert the commits. Nothing here writes a new field or changes
a stored shape, so a revert needs no migration.

- [x] Security impact considered
- [x] Backward compatibility considered
- [x] Rollback path is clear for risky changes

## Related Issues

N/A -- follow-ups from the review of #284, which is merged.

---------

Co-authored-by: arelchan <204152633+arelchan@users.noreply.github.com>
Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants