Skip to content
This repository was archived by the owner on Oct 10, 2026. It is now read-only.

Releases v0.3.0 实现 offloading prefix caching

Choose a tag to compare

View all tags
@noooop noooop released this 18 Jan 10:44
· 86 commits to main since this release
v0.3.0
74c73df
  1. 实现 offloading prefix caching，使用cpu 内存作为巨大kv cache池子
  2. 优化 prefix caching，这个是使用 gpu 版本的 prefix caching

祝好
wde v0.3.0

Assets 2
Loading