Skip to content

llm-scaler-vllm beta release 0.26.0-b1

Pre-release
Pre-release

Choose a tag to compare

View all tags
@liu-shaojun liu-shaojun released this 02 Sep 04:45
· 1 commit to upgrade/v0.26.0 since this release
vllm-0.26.0-b1
25b645a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Highlights

Resources

What’s new

  • vLLM:
    • Upgrade vLLM to v0.26.0
    • Improve Time to First Token (TTFT) for Qwen models
    • Fix accuracy issues for Qwen models
    • Fix LoRA serving issues and improve performance
    • Improve FP8 KV cache performance
    • Bug fixes
Assets 2
Loading