Repository navigation
llama-runtime 0.5.0: read-only model discovery
Adds a public, typed, read-only model discovery API while preserving the existing generation and embedding APIs.
from runtime_llama import discover_models
catalog = discover_models("https://inference.example.net", timeout=5.0)
for model in catalog.loaded_models:
print(model.id, model.capabilities.chat, model.capabilities.embeddings)- Separates observed catalog entries from proven loaded models, including native direct-server sleep observations and router states.
- Preserves unknown capabilities, independent identity assertions and the provenance of loaded versus training context.
- Uses at most two bounded GET requests, one deadline and cooperative cancellation; no model selection, loading, inference or administration.
- Adds optional decompressed JSON limits to the existing HTTP client and explicit owned-session ambient authentication policy. Discovery ignores ambient netrc/proxy settings; existing raw-client defaults are preserved.
Python distribution: runtime-llama; import: runtime_llama; Python >=3.11. Authentication remains optional and explicit. Endpoint validation is stricter for malformed or credential-bearing URLs.
Validation: 727 deterministic tests passed, including 144 new discovery/transport cases; Ruff passed; sdist-to-wheel build and clean installed-wheel smoke passed. Source, licenses, normalized archives and privacy checks passed. No new live model qualification, consumer migration or Linux/Windows execution is claimed.
API contract and limits · Verification
Install the attached wheel or the tagged source. Verify the wheel and source archive against the attached SHA256SUMS. Apache-2.0; copyright Davide Mele; upstream MIT attribution retained.