Repository navigation
0.1a11
Pre-release
Pre-release
OpenAI-compatible endpoints can now be configured directly, with no extra LLM plugin. #12
plugins:
datasette-llm:
default_model: lmstudio-qwen
endpoints:
- model: lmstudio-qwen
model_name: qwen/qwen3.8-27b
api_base: http://localhost:1234/v1
responses: true
- model: hosted-assistant
model_name: organization/assistant
api_base: https://models.example.com/v1
key: openrouter
options:
temperature: 0.2- A model reference can now be a dictionary with an
api_base, pointing at any OpenAI-compatible Chat Completions or Responses endpoint. Endpoint models support tools and streaming by default. - New
endpointssetting, global or per purpose, registers endpoint models without turning on themodelsallowlist. Endpoint dictionaries also work indefault_model, a purpose'smodel, andmodelslists. llm.model()andllm.group()accept a model-reference dictionary as well as a model ID.keynames the credential for a model. It is looked up indatasette-secretsfirst, then in LLM's stored keys under the same name, sokey: openrouteruses the key saved withllm keys set openrouter. An endpoint without akeyis treated as unauthenticated and never receives a provider's default key.- An endpoint's
modelis its local ID and must not match an installed model ID or alias. Usemodel_namefor the ID the remote server expects. - Aliases now work in
modelsandblocked_models. - Model lookup is faster: each call takes one snapshot of the installed models, so the work no longer grows with installed models multiplied by configuration entries.
- A
models,endpoints,blocked_modelsorpurposeskey left empty in YAML is now treated as absent. An empty purpose-levelmodelskey previously raised aTypeError. - Put credentials in
key, notheaders. Extraheadersare visible in Datasette's/-/config.json.
Compatibility changes
- Allowlists, blocklists and
llm_filter_modelshooks now apply tomodel()andgroup(), including explicit model IDs and configured defaults. Selecting an excluded model raisesModelNotFoundError. Previously they only affectedmodels(). model()andgroup()now raisellm.NeedsKeyExceptionif the selected model's key is missing andrequire_keysis on. Previously the error surfaced when the first prompt ran.- An explicit
keyon a model no longer falls back to the provider's default credential. If that name is missing from both datasette-secrets and LLM's stored keys, selecting the model raisesNeedsKeyException. Removekeyto restore normal provider key resolution. - Model-reference dictionaries now reject unknown fields, and
key: nulloroptions: null, withdatasette_llm.ModelConfigurationError. Omit optional fields instead. An invalid reference under a purpose only affects lookups for that purpose. - A purpose's model allowlist now also includes the global
default_model. - Unknown model IDs raise
datasette_llm.ModelNotFoundErrorinstead ofllm.UnknownModelError. - Requires LLM 0.37.1 or newer.