Skip to content

v0.2.0

Latest
Latest

Choose a tag to compare

View all tags
@258SusanLiu 258SusanLiu released this 02 Nov 22:46
· 1 commit to main since this release
v0.2.0
This tag was signed with the committer’s verified signature.
258SusanLiu
SSH Key Fingerprint: iR9zTea5R0lt3k3pFZwv1RtAMKivETNxiihZhwy8bmA
Verified
Learn about vigilant mode.
d0f47fc
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Highlights

  • Two new skills — the pack now has 22:
    • vllm-omni-xpu-run — serve omni-modality and generative models (Qwen-Omni chat, text-to-image, video, text-to-speech) with vLLM-Omni on Intel Arc / Arc Pro B-series GPUs (#36)
    • enable-xpu-monitoring — deploy a single-node monitoring stack (XPU Manager, Prometheus, Grafana, vLLM metrics) with Docker Compose (#27)
  • [Performance] The vLLM skills now default to XPU graphs on vLLM v0.31.0+ instead of forcing --enforce-eager, which made decode about 1.8× slower per token (64 ms vs 36 ms, 4× Arc Pro B70). Eager mode stays available as an opt-in for older images. (#34)
  • [Serving] The SGLang skills now use the official prebuilt lmsysorg/sglang:v0.5.22-xpu image, with no local image build or user-space driver patching, and document the FI_TCP_IFACE=lo fix for multi-GPU (--tp > 1) hangs on a single host. (#24, #35)
  • [Accuracy] model-can-it-fit and model-config-recommend now size sliding-window, hybrid SSM/Mamba, and mixed-precision (e.g. NVFP4) models more accurately instead of treating every layer as full attention. (#29)

See the README for the full skill list and setup.

All changes (9 pull requests)

Skills

  • vllm-omni-xpu-run: new skill for serving omni-modality and generative models with vLLM-Omni on Arc / Arc Pro B-series by @YuningQiu in #36
  • enable-xpu-monitoring: new skill that deploys XPU, host, and vLLM monitoring with Docker Compose by @yogit2020 in #27
  • model-can-it-fit, model-config-recommend: size sliding-window, hybrid SSM, and mixed-precision models per layer by @skavulya in #29
  • vLLM skills: default to XPU graphs on vLLM v0.31.0+ instead of forcing eager mode by @GopeshKh in #34
  • SGLang skills: move to the official lmsysorg/sglang XPU release image and fix multi-GPU (--tp > 1) hangs on a single host by @YuningQiu in #24, #35
  • torch-xpu-run: stop recommending installing Transformers from git main; use pip install -U transformers or the version the model card pins (fixes issue #13) by @YuningQiu in #31
  • model-can-it-fit: remove unnecessary f-string prefixes in fit.py (code cleanup, no change to output) by @YuningQiu in #33

Docs

  • Add compatibility and platform badges to the README by @258SusanLiu in #32

Thanks to the 5 contributors to this release: @YuningQiu, @GopeshKh, @258SusanLiu, @yogit2020, @skavulya — and welcome to first-time contributors @yogit2020 and @skavulya!

Full Changelog: v0.1.0...v0.2.0

Contributors

skavulya, yogit2020, and 3 other contributors
Assets 2
Loading