Skip to content

Add model discovery layer with OpenRouter as the seed provider#717

Merged
dcellison merged 3 commits into
mainfrom
feature/model-discovery
Jun 22, 2026
Merged

Add model discovery layer with OpenRouter as the seed provider#717
dcellison merged 3 commits into
mainfrom
feature/model-discovery

Conversation

@dcellison

Copy link
Copy Markdown
Owner

Summary

Phase 1 of the model-discovery work. Adds a per-provider discovery layer in src/kai/discovery.py that wraps the existing curated lists in src/kai/config.py. Provider registration stays explicit, defaults stay curated, validation stays canonical. The Telegram /models keyboard now consults the layer via a new model_source_for_backend(...) and renders an indirect-callback paginated keyboard for discovered sources; /model <id>, /settings model, and /workspace config model all migrate to the same surface and collapse to a short usage sentence for discovered or large catalogs.

OpenRouter is the seed discoverable provider; cache TTL 24h with on-demand background refresh, atomic writes, dedupe, and graceful fallback. Later phases (filed separately after this lands) bring Ollama, the direct API providers, and OpenCode into the same layer.

What changed

  • src/kai/discovery.py (new): ProviderModelSource and RefreshResult dataclasses plus a RefreshError exception. Synchronous get_provider_model_source(provider) reads the on-disk cache and (when a loop is running) schedules a background refresh on absent / stale / schema-mismatched / malformed reads. Async refresh_provider_models(provider) performs the actual fetch and atomically writes via tempfile + os.replace. Module-level dedupe dict keys background tasks by provider so chat-spam of /models collapses to a single in-flight refresh.
  • src/kai/config.py: New model_source_for_backend(...) dispatches per (backend, provider) to discovery for goose+openrouter and to the existing curated dicts for everyone else. Import topology between config.py and discovery.py uses a TYPE_CHECKING block, a quoted return-type annotation, and a local runtime import inside the function body to avoid a cycle (discovery.py imports DATA_DIR from config.py at module load).
  • src/kai/bot.py: _render_models_keyboard(current, source, page=0) replaces the old _models_keyboard(...); curated keyboards keep direct model:<id> callbacks, discovered keyboards use model_pick:<cache_gen>:<index> so OpenRouter IDs that exceed Telegram's 64-byte callback_data cap stay selectable (one already exists in the live catalog). Pagination at 20 entries per page with models_page:<cache_gen>:<page>. New handle_model_pick_callback and handle_models_page_callback re-read the cache on click and answer Catalogue changed; please re-open /models. on any stale-gen / missing / malformed / out-of-range condition. The /model, /settings model, and /workspace config model no-args usage replies collapse to Usage: <cmd> <model_id>. Use /models to browse available models. whenever the source is discovered/discovered_stale or larger than the page size. Validation is unchanged: /model <id> against an open-ended provider still passes without the cache being warm.
  • src/kai/refresh_models.py: New openrouter branch calls discovery.refresh_provider_models("openrouter") directly (admin CLI bypasses the runtime scheduler with schedule_refresh=False), prints a cache-age summary plus added/removed diff, maps RefreshError to exit code 2 with the prior cache preserved. Module docstring and argparse description reworded to "admin audit and one-shot cache refresh." Existing curated-provider fetchers (anthropic, openai, google) unchanged.
  • Tests: 18 in tests/test_discovery.py (new), 4 in tests/test_refresh_models.py (new), 18 in tests/test_bot.py (new) plus 3 migrated existing tests. Plus an import-cycle smoke that pins the topology so a future contributor cannot regress to a top-level from kai.discovery import ProviderModelSource in config.py.
  • Docs: README's "Model switching" section explains the three layers (registered / discovered / curated). Wiki page draft for Model-Discovery.md at /var/lib/kai/home/2114582497/docs/wiki-model-discovery.md to push after merge.

Closes #705

Test plan

  • make check (ruff + format) clean
  • make test clean (4598 passed, 1 skipped; +40 over main)
  • Live OpenRouter catalog has an ID at 61 bytes; the test fixture uses it and asserts every rendered button's callback_data is ≤64 bytes
  • Import-cycle smoke (test_import_cycle_smoke) imports kai.config and kai.discovery and round-trips through model_source_for_backend("goose", "openrouter")
  • Acceptance grep from the v3 spec: discovery module, model_source_for_backend, no top-level from kai.discovery import in config.py, TYPE_CHECKING block present, installer free of runtime-surface imports, discovered_stale / catalogue updating / model_pick: / models_page: / Catalogue changed all wired in bot.py
  • Manual: post-merge admin smoke — sudo -u kai env KAI_DATA_DIR=/var/lib/kai KAI_INSTALL_DIR=/opt/kai /opt/kai/venv/bin/python -m kai.refresh_models to warm the OpenRouter cache against the live endpoint, then a Telegram /models from an openrouter-effective user

Phase 1 of the model discovery work. A new src/kai/discovery.py owns a per-provider cache with TTL, background refresh, atomic writes, and indirect-callback paginated keyboards in /models. config.py gains model_source_for_backend() as the bot/runtime sibling to models_for_backend(), with a TYPE_CHECKING + local-runtime-import topology that keeps discovery.py free to import DATA_DIR from config.py without a cycle. The /model, /settings model, and /workspace config model no-args usage paths collapse to a short generic sentence when the source is discovered or larger than the keyboard page size; on-click resolution re-reads the cache and answers 'Catalogue changed' on stale/missing/malformed/out-of-range conditions. refresh_models is repositioned as admin audit with a new openrouter branch that maps RefreshError to exit code 2 with the prior cache preserved. Curated providers (claude, codex, opencode, anthropic/openai/google/deepseek under goose) keep their existing surfaces; later phases bring them into discovery one at a time. Adds 40 tests (18 discovery + 4 refresh + 18 bot).
refresh_provider_models() now raises RefreshError when the fetcher returns an empty mapping while a non-empty prior cache exists. An upstream incident returning shape-valid but semantically empty data could otherwise overwrite a working catalog with {}, taking /models from 'stale but usable' to 'empty keyboard with no buttons' on a transient outage. handle_models_page_callback() now bounds the click against the catalog size with the same defensive shape handle_model_pick_callback uses for indices; a stale or tampered models_page:<gen>:999 whose cache_gen still matches now answers 'Catalogue changed' instead of blanking the keyboard. Three new tests pin both fixes.
The prior guard only fired when a non-empty prior cache existed. A first-run install whose initial OpenRouter fetch landed during an upstream incident would still write {} as the cache, locking /models into kind=discovered with an empty keyboard for one TTL instead of falling back to the open-ended text prompt while the provider recovers. The guard is now unconditional: an empty fetch is treated as an upstream fault and raises RefreshError, leaving the cache absent or unchanged so the next scheduled refresh has a chance to recover. New test asserts no cache file is written and the next read remains open_ended.
@dcellison dcellison merged commit 2b33c5a into main Jun 22, 2026
1 check passed
@dcellison dcellison deleted the feature/model-discovery branch June 22, 2026 11:38
dcellison added a commit that referenced this pull request Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Epic: Discover provider and backend model lists instead of hard-coding selectable models

2 participants