Repository navigation
runtime-llama 0.11.1
Hardens the automatic embedding boundary without changing the existing scheduler, category 80% / mixed 90% recommendations, consumer configurations or model profiles.
AutomaticEmbeddingConfigexposes existing finiteembedding_max_request_bytes,embedding_max_response_bytes,embedding_max_pending_bytesandembedding_max_pending_tokenscontrols. Defaults are unchanged. Decoded HTTP response size remains separate from retained Python result capacity. An explicit bounded pending budget can accommodate high-context direct batches without invented token counts.http_connection_policy="reuse"selects the existing keep-alive policy for an owned client. Default remains"close"; injected clients retain policy and ownership. No retry or transport fallback is added.- Calling draining close from the same service's cancellation callback now raises a typed diagnostic before shutdown mutation, preventing self-deadlock across synchronous, calibration and streaming paths. Callbacks must remain side-effect free; external cancellation and drained shutdown are preserved.
- Explicit cold character-budget calibration checks actual private persistence feasibility before remote tokenization. Valid readable caches do not require directory writes, final publication remains atomic, and late timeout failures retain their type.
- A verified single-model statusless catalog can be attested by coherent read-only root
/props, sharing authority with the declared adapter. Router, sleeping, ambiguous, unverified and inconsistent cases remain rejected. - The mixed recommendation's cross-category limitation is documented and tested. No statistical input admission, implicit tokenizer, silent truncation or exact-preparation fallback is introduced.
Verification: 1,811 tests, Ruff check/format, normalized source/wheel builds and fresh isolated installation. New regressions exercise a real loopback HTTP response with 128 vectors × 4,096 dimensions, default 8 MiB rejection and explicit 16 MiB success, decoded gzip bounds, incomplete bodies, no replay, ordered results and drain; owned/borrowed connection policy; high-context resource capacity/fairness; callback self-close in supervised subprocesses; cache feasibility/cancellation/read-only reuse; and statusless discovery.
Installed-wheel live qualification used both already loaded Nomic replicas and unchanged numerical thresholds. 511 direct texts before calibration: 1.5078 seconds, 4 embedding POSTs + 16 metadata GETs, zero tokenizer calls. Exact preparation plus embedding: 3.7297 seconds, 511 tokenizer POSTs + 4 embedding POSTs + 15 metadata GETs. All 511 vectors matched exactly. Only the explicit cold budget request performed 10,000 tokenizations, in 53.8941 seconds; category selection and warm reuse needed no additional counts. These are client wall times including applicable metadata, preparation, HTTP and server work, not pure GPU timings or a controlled version-speed comparison.
Lifecycle and host-reported evidence are managed with Flower MCP. See the configuration/API guide, hardening qualification and public release record.
The portable Python wheel's 36 runtime payload files match the live-qualified candidate and public source byte-for-byte. Optional tsnet archives retain v0.6.0 executables and license inventory; no new native build, OS execution, VPN enrollment or VPN inference is claimed. Apache-2.0, Davide Mele copyright, NOTICE and upstream attribution remain unchanged. Consumer installations, code/config/pins, services, models and databases were not changed. No PyPI upload is included.
RELEASE-MANIFEST.json binds source and artifact provenance; verify the seven assets using SHA256SUMS.