Repository navigation
UltraQuant on AMD Instinct: More Efficient Agentic Serving for Qwen3.8-MXFP4 — ROCm Blogs #575
giscus[bot]
Bot
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
UltraQuant on AMD Instinct: More Efficient Agentic Serving for Qwen3.8-MXFP4 — ROCm Blogs
A native 4-bit MXFP4 KV cache that speeds up Qwen3.8 decode over 8-bit KV on AMD Instinct MI355X, with no accuracy loss.
https://rocm.blogs.amd.com/software-tools-optimization/ultraquant-kv4/README.html
All reactions