Skip to content

llama-runtime 0.5.0: read-only model discovery

Choose a tag to compare

@Damel91 Damel91 released this 16 Oct 18:46
· 57 commits to main since this release

Adds a public, typed, read-only model discovery API while preserving the existing generation and embedding APIs.

from runtime_llama import discover_models

catalog = discover_models("https://inference.example.net", timeout=5.0)
for model in catalog.loaded_models:
    print(model.id, model.capabilities.chat, model.capabilities.embeddings)
  • Separates observed catalog entries from proven loaded models, including native direct-server sleep observations and router states.
  • Preserves unknown capabilities, independent identity assertions and the provenance of loaded versus training context.
  • Uses at most two bounded GET requests, one deadline and cooperative cancellation; no model selection, loading, inference or administration.
  • Adds optional decompressed JSON limits to the existing HTTP client and explicit owned-session ambient authentication policy. Discovery ignores ambient netrc/proxy settings; existing raw-client defaults are preserved.

Python distribution: runtime-llama; import: runtime_llama; Python >=3.11. Authentication remains optional and explicit. Endpoint validation is stricter for malformed or credential-bearing URLs.

Validation: 727 deterministic tests passed, including 144 new discovery/transport cases; Ruff passed; sdist-to-wheel build and clean installed-wheel smoke passed. Source, licenses, normalized archives and privacy checks passed. No new live model qualification, consumer migration or Linux/Windows execution is claimed.

API contract and limits · Verification

Install the attached wheel or the tagged source. Verify the wheel and source archive against the attached SHA256SUMS. Apache-2.0; copyright Davide Mele; upstream MIT attribution retained.