Skip to content

v0.6.2 — audit-fix patch release

Choose a tag to compare

View all tags
@offbyonebit offbyonebit released this 11 Aug 20:34
· 97 commits to main since this release
v0.6.2
2e5ea08

Bugfix release clearing the remaining concurrency and leak findings from the full-source audit. No new features, no config or API changes. (0.6.1, with identical code, shipped a broken sdist and no wheel — use 0.6.2.)

Fixes

  • A dropped streamed request permanently disabled auto-tune. If a stream's BackgroundTask never ran (upstream dies mid-generation, shutdown with an open stream), the global in-flight counter leaked, and the auto-tuner gates every sweep on it. The decrement now runs from the body generator's finally, idempotently, with a zero floor. (#27)
  • Model swaps and recipe edits no longer kill generations mid-stream. Eviction and rebuild_model both drain the incumbent's per-model in-flight count (bounded) before stopping it. (#28, #31)
  • The lock-free fast path can no longer hand out a dying server. Acquisition is atomic with the readiness check, and a _stopping mark closes the teardown window. (#29)
  • Auto-tune no longer evicts a just-arrived request. measure() re-checks the abort hook after the edit POST — the only window where that request can register. (#30)
  • A failed end-of-sweep restore no longer persists the last candidate. It retries, then falls back to a direct config write. (#37)
  • Upstream proxying no longer leaks the httpx client pool on every call, nor the response on mid-stream failure. (#38)
  • Autotuner start/stop are serialized; a racing pair can't kill the loop or orphan a task. (#34)
  • Plus: dry-run no longer marks models tuned (#22), no auto draft-MTP on hybrid SSM (#23), loaded-reporting waits for health (#40), deferred restore keeps abort signals (#32), atomic config writes (#36), audit finding #41 verified already fixed.

Verification

599 tests passing (19 new), ruff clean, and an end-to-end run on an Arc Pro B60: cold start, SSE streaming, model swap with eviction drain, a mid-stream swap race (stream completed cleanly while the swap waited), upstream proxy in both modes, admin edit → rebuild → reload, and clean shutdown with zero orphaned processes.

pip install --upgrade arc-llama
Assets 2
Loading