Skip to content

Releases

Fix crash in kv cache CPU offloading logic

Choose a tag to compare

@sriumcp sriumcp released this 06 Feb 00:50
fc9d89f

What's Changed

  • [Bugfix] Fix IndexError during chunked prefill with CPU KV offloading by @sriumcp in #47

Full Changelog: v0.6.5...v0.6.6

Add more logging for KV Offloading

Choose a tag to compare

@sriumcp sriumcp released this 05 Feb 13:44
5b79ecd

What's Changed

  • [Feature] Add comprehensive KV event debugging logs by @sriumcp in #46

Full Changelog: v0.6.4...v0.6.5

Bugfix for decode block offloading

Choose a tag to compare

@sriumcp sriumcp released this 05 Feb 04:29
618bc4c

What's Changed

  • [Bugfix] Fix CacheStoreCommitted not emitted for decode-formed blocks by @sriumcp in #45

Full Changelog: v0.6.3...v0.6.4

Fix KV cache transfer publication and behavioral tests

Choose a tag to compare

@sriumcp sriumcp released this 05 Feb 02:29
e90045f

What's Changed

  • [Feature] Publish worker-side KV cache transfer events (Phase 5) by @sriumcp in #44

Full Changelog: v0.6.2...v0.6.3

Fix event emission

Choose a tag to compare

@sriumcp sriumcp released this 05 Feb 00:58
7caa9ba

What's Changed

  • [Feature] Enable CPU KV offloading in Kubernetes test manifest by @sriumcp in #42
  • [Bugfix] Publish worker-side KV cache transfer events by @sriumcp in #43

Full Changelog: v0.6.1...v0.6.2

Bugfix in Docker image creation

Choose a tag to compare

@sriumcp sriumcp released this 04 Feb 21:53
e0d1829

What's Changed

  • [Bugfix] Include examples directory in fork Docker image by @sriumcp in #41

Full Changelog: v0.6.0...v0.6.1

Updated Kubernetes test manifest and docs

Choose a tag to compare

@sriumcp sriumcp released this 04 Feb 15:03
3fa8f2d

What's Changed

  • [Feature] Add KV events to Kubernetes test manifest (Phase 5) by @sriumcp in #40

Full Changelog: v0.5.0...v0.6.0

Enhanced KV event stream for modeling

Choose a tag to compare

@sriumcp sriumcp released this 04 Feb 12:56
76fd127

What's Changed

  • [Feature] Add KV cache offloading tracing event types (Phase 1) by @sriumcp in #36
  • [Feature] Add KV cache offloading local transfer events (Phase 2) by @sriumcp in #37
  • [Feature] Add KV cache remote transfer events (Phase 3) by @sriumcp in #38
  • [Feature] Production-ready KV events subscriber (Phase 4) by @sriumcp in #39

Full Changelog: v0.4.6...v0.5.0

Fix to step event span creation and docs

Choose a tag to compare

@sriumcp sriumcp released this 31 Jan 13:06
a0d1f6f

What's Changed

  • [Feature] Add periodic span closure for step tracing event export by @sriumcp in #35

Full Changelog: v0.4.5...v0.4.6

Yet another attempt to fix CI

Choose a tag to compare

@sriumcp sriumcp released this 31 Jan 02:08
01a92a6

What's Changed

Full Changelog: v0.4.4...v0.4.5