Skip to content

v0.9.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 03 Nov 01:03
· 29 commits to develop since this release
v0.9.0
ae11ae4
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

UCM 0.9.0 Release Notes

1. Compatibility

1.1 vLLM Ascend 0.30.0

UCM 0.9.0 has been adapted to the official release of vLLM Ascend 0.30.0 and supports the following models:

  • GLM5.2 (A5)
  • GLM5.3
  • Qwen3.8-27B
  • DSV4.1
  • DeepSeek V4 (A5)

Note: The (A5) label indicates that the model has been validated and is supported on Atlas A5 950-series accelerators.

1.2 vLLM Ascend 0.27.1

The following models are supported:

  • DeepSeek V4
  • Non-hybrid models, including MiniMax-M3, GLM5.2, and GLM5.3

2. New Features

  • SGLang Linker integration: Added support for integrating UCM L1 with SGLang Linker, extending UCM caching capabilities to SGLang inference scenarios.

3. Bug Fixes

  • Fixed inference accuracy issues caused by concurrent UCM writes: Resolved an issue in Layerwise mode where premature shard commits during concurrent PosixStore writes could result in incomplete cache data and incorrect inference results when cached data was loaded from SSD. Related issue: #1544.

  • Fixed accuracy issues in the multimodal Encoder Cache Connector: Resolved inference accuracy issues associated with cache hits in UCM's multimodal Encoder Cache Connector.

4. Known Issues

  1. Performance regression with DSV4-Flash 0731 on A5

    On Atlas A5, enabling UCM may reduce performance for DSV4-Flash 0731 when the HBM cache hit rate is 100%.

  2. Inference accuracy degradation with DSV4 on A5

    Inference accuracy degradation has been observed with DSV4 on Atlas A5.

  3. Inference accuracy issues with GLM5.3 and vLLM Ascend Balance Scheduler

    On vLLM Ascend 0.30.0, GLM5.3 may produce incorrect inference results when Balance Scheduler is enabled.

    The corresponding fix has been merged into the vLLM Ascend develop branch but is not included in the currently supported official release.

    Temporary workaround: Remove the following setting from the vLLM Ascend 0.30.0 configuration:

    'enable_balance_scheduling': True

  4. Inference service hang with GLM5.3-Flash on vLLM Ascend 0.30.0

    On vLLM Ascend 0.30.0, the inference service may hang when GLM5.3-Flash accesses cached data from SSD through UCM.

    Temporary workaround: Set the following option in ucm_config.yaml:

    share_buffer_enable: false

    Disabling the shared buffer can mitigate the issue, but may reduce the performance gains provided by UCM caching.

    Recommendation: The combination of UCM 0.9.0, vLLM Ascend 0.30.0, and GLM5.3-Flash is not recommended. Use a subsequent release that includes a fix for this issue.

5. Version Selection Recommendations

UCM Version Inference Engine Version Supported Models / Scope
0.9.0 vLLM Ascend 0.30.0 GLM5.2 (A5), GLM5.3, Qwen3.8-27B, DSV4.1, DeepSeek V4 (A5)
0.9.0 vLLM Ascend 0.27.1 DeepSeek V4; non-hybrid models including MiniMax-M3, GLM5.2, and GLM5.3
0.8.1 vLLM / vLLM Ascend ≤ 0.27.1 Suitable for vLLM / vLLM Ascend 0.27.1 and earlier versions

Recommendations:

  • Newer engine versions: Use UCM 0.9.0 with vLLM Ascend 0.30.0, subject to the supported-model list above.
  • Existing deployments: UCM 0.8.1 remains an option for vLLM / vLLM Ascend 0.27.1 and earlier versions.
  • GLM5.3-Flash: UCM 0.9.0 with vLLM Ascend 0.30.0 is not a recommended configuration for this model. Use a subsequent release with the issue resolved.

Status: complete

Backend Wheels: 20
Chart: unified-cache-chart-0.9.0.tgz

vLLM OpenAI

Runtime: docker.io/vllm/vllm-openai

Images: GHCR ghcr.io/modelengine-group/vllm-openai · Docker Hub docker.io/unifiedcachemanager/vllm-openai — 8 image families / 16 architecture members

Runtime capability Upstream Runtime tags
docker.io/vllm/vllm-openai：		 Runtime tags
GHCR: ghcr.io/modelengine-group/vllm-openai
Docker Hub: docker.io/unifiedcachemanager/vllm-openai		 Python ABI Wheel
CUDA 12.9 v0.27.1-cu129
v0.27.1-cu129-ubuntu2404
v0.30.0-cu129
v0.30.0-cu129-ubuntu2404		 v0.27.1-cu129-ucm-0.9.0
v0.27.1-cu129-ubuntu2404-ucm-0.9.0
v0.30.0-cu129-ucm-0.9.0
v0.30.0-cu129-ubuntu2404-ucm-0.9.0		 cp312 PyPI
pip install "uc-manager[cu129]==0.9.0"
CUDA 13.0 v0.27.1
v0.27.1-ubuntu2404
v0.30.0
v0.30.0-ubuntu2404		 v0.27.1-ucm-0.9.0
v0.27.1-ubuntu2404-ucm-0.9.0
v0.30.0-ucm-0.9.0
v0.30.0-ubuntu2404-ucm-0.9.0		 cp312 PyPI
pip install "uc-manager[cu130]==0.9.0"

vLLM-Ascend

Runtime: quay.io/ascend/vllm-ascend

Images: GHCR ghcr.io/modelengine-group/vllm-ascend · Docker Hub docker.io/unifiedcachemanager/vllm-ascend — 12 image families / 24 architecture members

Runtime capability Upstream Runtime tags
quay.io/ascend/vllm-ascend：		 Runtime tags
GHCR: ghcr.io/modelengine-group/vllm-ascend
Docker Hub: docker.io/unifiedcachemanager/vllm-ascend		 Python ABI Wheel
CANN 9.1.0 / A2 v0.27.1rc1
v0.27.1rc1-openeuler		 v0.27.1rc1-ucm-0.9.0
v0.27.1rc1-openeuler-ucm-0.9.0		 cp312 PyPI
pip install "uc-manager[cann910-a2]==0.9.0"
CANN 9.1.0 / A3 v0.27.1rc1-a3
v0.27.1rc1-a3-openeuler		 v0.27.1rc1-a3-ucm-0.9.0
v0.27.1rc1-a3-openeuler-ucm-0.9.0		 cp312 PyPI
pip install "uc-manager[cann910-a3]==0.9.0"
CANN 9.1.0 / A5 v0.27.1rc1-a5
v0.27.1rc1-a5-openeuler		 v0.27.1rc1-a5-ucm-0.9.0
v0.27.1rc1-a5-openeuler-ucm-0.9.0		 cp312 PyPI
pip install "uc-manager[cann910-a5]==0.9.0"
CANN 9.2.1 / A2 v0.30.0
v0.30.0-openeuler		 v0.30.0-ucm-0.9.0
v0.30.0-openeuler-ucm-0.9.0		 cp312 PyPI
pip install "uc-manager[cann921-a2]==0.9.0"
CANN 9.2.1 / A3 v0.30.0-a3
v0.30.0-a3-openeuler		 v0.30.0-a3-ucm-0.9.0
v0.30.0-a3-openeuler-ucm-0.9.0		 cp312 PyPI
pip install "uc-manager[cann921-a3]==0.9.0"
CANN 9.2.1 / A5 v0.30.0-a5
v0.30.0-a5-openeuler		 v0.30.0-a5-ucm-0.9.0
v0.30.0-a5-openeuler-ucm-0.9.0		 cp312 PyPI
pip install "uc-manager[cann921-a5]==0.9.0"

SGLang

Runtime: docker.io/lmsysorg/sglang

Images: GHCR ghcr.io/modelengine-group/sglang · Docker Hub docker.io/unifiedcachemanager/sglang — 6 image families / 12 architecture members

Runtime capability Upstream Runtime tags
docker.io/lmsysorg/sglang：		 Runtime tags
GHCR: ghcr.io/modelengine-group/sglang
Docker Hub: docker.io/unifiedcachemanager/sglang		 Python ABI Wheel
CANN 9.0.0 / A2 v0.5.20-cann9.0.0-910b v0.5.20-cann9.0.0-910b-ucm-0.9.0 cp311 PyPI
pip install "uc-manager[cann900-a2]==0.9.0"
CANN 9.0.0 / A3 v0.5.20-cann9.0.0-a3 v0.5.20-cann9.0.0-a3-ucm-0.9.0 cp311 PyPI
pip install "uc-manager[cann900-a3]==0.9.0"
CANN 9.1.0 / A3 v0.5.22-cann9.1.0-a3 v0.5.22-cann9.1.0-a3-ucm-0.9.0 cp312 PyPI
pip install "uc-manager[cann910-a3]==0.9.0"
CUDA 13.0 v0.5.20-cu130
v0.5.21-cu130
v0.5.22-cu130		 v0.5.20-cu130-ucm-0.9.0
v0.5.21-cu130-ucm-0.9.0
v0.5.22-cu130-ucm-0.9.0		 cp312 PyPI
pip install "uc-manager[cu130]==0.9.0"

Toolkit

ucm_toolkit-0.9.0-py3-none-any.whl
Install the optional toolkit extra with the same-version meta package, or install this Wheel independently.

Assets 28
Loading