runtime-llama 0.7.0

Python client for native llama.cpp HTTP servers, with corrected embedding batching and bounded preparation, independent HTTP connections by default, and optional typed native generation timings.

• batch_size is the total text window, distributed over verified usable replicas. A 256-text window used two concurrent POSTs of 128 texts; native computation batches no longer impose an artificial aggregate HTTP token cap.
• Exact task-aware preparation, once-only prefixes/special tokens, oversized-input measurement, identity/topology checks, finite deadlines and drained cancellation remain available.
• Generation bookkeeping is reduced while preserving reasoning, tool calls and runaway guards. Keep-alive is explicit; native unpatched llama-server works with the default connection-close policy.

Qualification: 1164 deterministic Python tests and Ruff checks passed. On server build b11716-797de63b2, two EmbeddingGemma 2 BF16 HIP replicas with Flash Attention off passed the unchanged numerical thresholds: all 77 discriminating comparisons had zero raw coordinate difference, including 33 cross-replica comparisons. The clean-installed candidate wheel passed the historical 551-output SDK replay; the final wheel contains the identical qualified runtime payload. An independent offline audit passed 394 checks.

These are bounded deployment results, not a guarantee for every input/backend or a pure GPU throughput measurement. Build and Flash Attention were changed together, so the numerical improvement is not attributed to one isolated cause. Earlier failed qualifications remain historical evidence.

Install the wheel below or use the tagged source:

python -m pip install 'git+https://github.com/Damel91/llama-runtime.git@v0.7.0'

Distribution: runtime-llama; Python import: runtime_llama; Python 3.11+; Apache-2.0. Optional tsnet helper archives explicitly reuse the byte-identical 0.6.0 executables and include licenses and original build provenance. This release adds no new OS/VPN execution or CI coverage claim. No consumer installation, service or model configuration was changed.

See verification and limits. Use RELEASE-MANIFEST.json and SHA256SUMS to verify the downloadable artifacts.