Repository navigation
Releases
Releases · inference-sim/vllm
Fix crash in kv cache CPU offloading logic
What's Changed
Full Changelog: v0.6.5...v0.6.6
Add more logging for KV Offloading
What's Changed
Full Changelog: v0.6.4...v0.6.5
Bugfix for decode block offloading
What's Changed
Full Changelog: v0.6.3...v0.6.4
Fix KV cache transfer publication and behavioral tests
What's Changed
Full Changelog: v0.6.2...v0.6.3
Fix event emission
What's Changed
- [Feature] Enable CPU KV offloading in Kubernetes test manifest by @sriumcp in #42
- [Bugfix] Publish worker-side KV cache transfer events by @sriumcp in #43
Full Changelog: v0.6.1...v0.6.2
Bugfix in Docker image creation
What's Changed
Full Changelog: v0.6.0...v0.6.1
Updated Kubernetes test manifest and docs
What's Changed
Full Changelog: v0.5.0...v0.6.0
Enhanced KV event stream for modeling
What's Changed
- [Feature] Add KV cache offloading tracing event types (Phase 1) by @sriumcp in #36
- [Feature] Add KV cache offloading local transfer events (Phase 2) by @sriumcp in #37
- [Feature] Add KV cache remote transfer events (Phase 3) by @sriumcp in #38
- [Feature] Production-ready KV events subscriber (Phase 4) by @sriumcp in #39
Full Changelog: v0.4.6...v0.5.0
Fix to step event span creation and docs
What's Changed
Full Changelog: v0.4.5...v0.4.6