Add model discovery layer with OpenRouter as the seed provider#717
Merged
Conversation
Phase 1 of the model discovery work. A new src/kai/discovery.py owns a per-provider cache with TTL, background refresh, atomic writes, and indirect-callback paginated keyboards in /models. config.py gains model_source_for_backend() as the bot/runtime sibling to models_for_backend(), with a TYPE_CHECKING + local-runtime-import topology that keeps discovery.py free to import DATA_DIR from config.py without a cycle. The /model, /settings model, and /workspace config model no-args usage paths collapse to a short generic sentence when the source is discovered or larger than the keyboard page size; on-click resolution re-reads the cache and answers 'Catalogue changed' on stale/missing/malformed/out-of-range conditions. refresh_models is repositioned as admin audit with a new openrouter branch that maps RefreshError to exit code 2 with the prior cache preserved. Curated providers (claude, codex, opencode, anthropic/openai/google/deepseek under goose) keep their existing surfaces; later phases bring them into discovery one at a time. Adds 40 tests (18 discovery + 4 refresh + 18 bot).
refresh_provider_models() now raises RefreshError when the fetcher returns an empty mapping while a non-empty prior cache exists. An upstream incident returning shape-valid but semantically empty data could otherwise overwrite a working catalog with {}, taking /models from 'stale but usable' to 'empty keyboard with no buttons' on a transient outage. handle_models_page_callback() now bounds the click against the catalog size with the same defensive shape handle_model_pick_callback uses for indices; a stale or tampered models_page:<gen>:999 whose cache_gen still matches now answers 'Catalogue changed' instead of blanking the keyboard. Three new tests pin both fixes.
The prior guard only fired when a non-empty prior cache existed. A first-run install whose initial OpenRouter fetch landed during an upstream incident would still write {} as the cache, locking /models into kind=discovered with an empty keyboard for one TTL instead of falling back to the open-ended text prompt while the provider recovers. The guard is now unconditional: an empty fetch is treated as an upstream fault and raises RefreshError, leaving the cache absent or unchanged so the next scheduled refresh has a chance to recover. New test asserts no cache file is written and the next read remains open_ended.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 1 of the model-discovery work. Adds a per-provider discovery layer in
src/kai/discovery.pythat wraps the existing curated lists insrc/kai/config.py. Provider registration stays explicit, defaults stay curated, validation stays canonical. The Telegram/modelskeyboard now consults the layer via a newmodel_source_for_backend(...)and renders an indirect-callback paginated keyboard for discovered sources;/model <id>,/settings model, and/workspace config modelall migrate to the same surface and collapse to a short usage sentence for discovered or large catalogs.OpenRouter is the seed discoverable provider; cache TTL 24h with on-demand background refresh, atomic writes, dedupe, and graceful fallback. Later phases (filed separately after this lands) bring Ollama, the direct API providers, and OpenCode into the same layer.
What changed
src/kai/discovery.py(new):ProviderModelSourceandRefreshResultdataclasses plus aRefreshErrorexception. Synchronousget_provider_model_source(provider)reads the on-disk cache and (when a loop is running) schedules a background refresh on absent / stale / schema-mismatched / malformed reads. Asyncrefresh_provider_models(provider)performs the actual fetch and atomically writes viatempfile + os.replace. Module-level dedupe dict keys background tasks by provider so chat-spam of/modelscollapses to a single in-flight refresh.src/kai/config.py: Newmodel_source_for_backend(...)dispatches per (backend, provider) to discovery forgoose+openrouterand to the existing curated dicts for everyone else. Import topology betweenconfig.pyanddiscovery.pyuses aTYPE_CHECKINGblock, a quoted return-type annotation, and a local runtime import inside the function body to avoid a cycle (discovery.pyimportsDATA_DIRfromconfig.pyat module load).src/kai/bot.py:_render_models_keyboard(current, source, page=0)replaces the old_models_keyboard(...); curated keyboards keep directmodel:<id>callbacks, discovered keyboards usemodel_pick:<cache_gen>:<index>so OpenRouter IDs that exceed Telegram's 64-byte callback_data cap stay selectable (one already exists in the live catalog). Pagination at 20 entries per page withmodels_page:<cache_gen>:<page>. Newhandle_model_pick_callbackandhandle_models_page_callbackre-read the cache on click and answerCatalogue changed; please re-open /models.on any stale-gen / missing / malformed / out-of-range condition. The/model,/settings model, and/workspace config modelno-args usage replies collapse toUsage: <cmd> <model_id>. Use /models to browse available models.whenever the source is discovered/discovered_stale or larger than the page size. Validation is unchanged:/model <id>against an open-ended provider still passes without the cache being warm.src/kai/refresh_models.py: New openrouter branch callsdiscovery.refresh_provider_models("openrouter")directly (admin CLI bypasses the runtime scheduler withschedule_refresh=False), prints a cache-age summary plus added/removed diff, mapsRefreshErrorto exit code 2 with the prior cache preserved. Module docstring and argparse description reworded to "admin audit and one-shot cache refresh." Existing curated-provider fetchers (anthropic, openai, google) unchanged.tests/test_discovery.py(new), 4 intests/test_refresh_models.py(new), 18 intests/test_bot.py(new) plus 3 migrated existing tests. Plus an import-cycle smoke that pins the topology so a future contributor cannot regress to a top-levelfrom kai.discovery import ProviderModelSourceinconfig.py.Model-Discovery.mdat/var/lib/kai/home/2114582497/docs/wiki-model-discovery.mdto push after merge.Closes #705
Test plan
make check(ruff + format) cleanmake testclean (4598 passed, 1 skipped; +40 overmain)callback_datais ≤64 bytestest_import_cycle_smoke) importskai.configandkai.discoveryand round-trips throughmodel_source_for_backend("goose", "openrouter")model_source_for_backend, no top-levelfrom kai.discovery importinconfig.py,TYPE_CHECKINGblock present, installer free of runtime-surface imports,discovered_stale/catalogue updating/model_pick:/models_page:/Catalogue changedall wired inbot.pysudo -u kai env KAI_DATA_DIR=/var/lib/kai KAI_INSTALL_DIR=/opt/kai /opt/kai/venv/bin/python -m kai.refresh_modelsto warm the OpenRouter cache against the live endpoint, then a Telegram/modelsfrom an openrouter-effective user