Skip to content

mini-dynamo v0.1.0

Choose a tag to compare

View all tags
@nessie993 nessie993 released this 13 Aug 13:34
· 442 commits to main since this release
v0.1.0
b0e0700
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

mini-dynamo's first public Rust release: an OpenAI-compatible, KV-cache-locality-aware inference router for multi-replica serving.

Stable serving surface:

  • approximate prefix-locality plus weighted-load routing;
  • health-gated failover and replica-aware /health readiness;
  • immediate upstream cancellation when the client disconnects;
  • streaming compatibility shims, usage/TTFT/cache metrics, and privacy-bounded route journals;
  • bounded local/remote tokenizer observation with unconditional approximate fallback.

Exact KV events, compact snapshot companions, and exact placement remain experimental, non-mutating shadow or disabled by default.

Qualified on node06 (8× RTX PRO 6000, two DSpark/vLLM TP4 replicas):

  • 2/2 healthy; zero LB or engine restarts;
  • 206/206 successful real-workflow chat requests, no upstream error series or warning/error logs;
  • cancellation released the LB reservation 46ms after client close and vLLM work 269ms after close;
  • 82.5% cached prompt tokens in the fresh 3-app × 4-session × 2-turn locality gate;
  • c12 shared-app: exact 6/6 split, 12/12 success, 452 tok/s;
  • isolated TP4 c12/max256: 794.5 tok/s while production remained active on the peer.

Immutable images:

  • LB: ghcr.io/helixml/mini-dynamo:v0.1.0@sha256:62d949e0e6b3880796fab6c12f148f24d3f76449cb8397da6e81fe6e57dd70a1
  • Companion: ghcr.io/helixml/mini-dynamo:companion-v0.1.0@sha256:4af08be5c011ac56d1bde2463e525c1d57d9ddd21391a3565ec55183566d9f95
  • node06 rollback LB: ghcr.io/helixml/mini-dynamo:rust-r33-replay-profile-afdd3ed@sha256:26f7a30fb5523be5b8fdecc251545a33580eb9b4fb8c66eba4b512de7a32052f

Both release images preserve OCI version 0.1.0 and revision b0e070073d4266018d2f907ff35a7ee88adfdcd4. The semver manifests were copied from the exact qualified SHA-tagged images without rebuilding.

Assets 2
Loading