Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/100-documentation.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 📚 Documentation
description: Report an issue related to unifiedcache official website
description: Report an issue related to ucm official website
title: "[Doc]: "
labels: ["documentation"]

Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/110-user-story.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 📚 User Story
description: Apply for an user story to be displayed on unifiedcache official website
description: Apply for an user story to be displayed on ucm official website
title: "[User Story]: "
labels: ["user-story"]

Expand All @@ -23,7 +23,7 @@ body:
Tell us how what kind of challenge you faced in this user story.
- type: textarea
attributes:
label: Solving challenges with unifiedcache and benefits
label: Solving challenges with ucm and benefits
description: >
Tell us how unifiedcache helped you overcome the challenges, including details like how you use it, what version you used, hardware info, etc. And what kind of benefit do you get from using unifiedcache
- type: textarea
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/200-installation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ body:
required: true
- type: textarea
attributes:
label: How you are installing unifiedcache, also vllm and vllm-ascend.
label: How you are installing ucm, also vllm and vllm-ascend.
description: |
Paste the full command you are trying to execute.
value: |
Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/300-usage.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 💻 Usage
description: Raise an issue here if you don't know how to use unifiedcache.
description: Raise an issue here if you don't know how to use ucm.
title: "[Usage]: "
labels: ["usage"]

Expand Down Expand Up @@ -30,7 +30,7 @@ body:
required: true
- type: textarea
attributes:
label: How would you like to use unifiedcache.
label: How would you like to use ucm.
description: |
A detailed description of how you want to use unifiedcache.
value: |
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/500-feature-request.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 🚀 Feature request
description: Submit a proposal/request for a new unifiedcache feature
description: Submit a proposal/request for a new ucm feature
title: "[Feature]: "
labels: ["feature request"]

Expand Down
6 changes: 3 additions & 3 deletions .github/ISSUE_TEMPLATE/600-new-model.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: 🤗 Support request for new model supported from huggingface/modelscope/modelers on unifiedcache
description: Submit a proposal/request for a new model from huggingface/modelscope/modelers on unifiedcache
name: 🤗 Support request for new model supported from huggingface/modelscope/modelers on ucm
description: Submit a proposal/request for a new model from huggingface/modelscope/modelers on ucm
title: "[New Model]: "
labels: ["new model"]

Expand All @@ -19,7 +19,7 @@ body:
required: true
- type: textarea
attributes:
label: The closest model unifiedcache already supports.
label: The closest model ucm already supports.
description: >
Here is the list of models already supported by unifiedcache in our official website . Which model is the most similar to the model you want to add support for?
- type: textarea
Expand Down
4 changes: 2 additions & 2 deletions .github/ISSUE_TEMPLATE/700-performance-discussion.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: ⚡ Discussion on the performance of unifiedcache
description: Submit a proposal/discussion about the performance of unifiedcache
name: ⚡ Discussion on the performance of ucm
description: Submit a proposal/discussion about the performance of ucm
title: "[Performance]: "
labels: ["performance"]

Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/800-others.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ body:
#### Before submitting an issue, please make sure the issue hasn't been already addressed by searching through [the existing and past issues](https://github.com/ModelEngine-Group/unified-cache-management/issues?q=is%3Aissue%20sort%3Acreated-desc).
- type: textarea
attributes:
label: Anything you want to discuss about unifiedcache.
label: Anything you want to discuss about ucm.
description: >
Anything you want to discuss about unifiedcache.
validations:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ucmnfsstore-ut.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ jobs:
- name: Configure CMake
# Configure CMake in a 'build' subdirectory. `CMAKE_BUILD_TYPE` is only required if you are using a single-configuration generator such as make.
# See https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html?highlight=cmake_build_type
working-directory: ${{github.workspace}}/unifiedcache/csrc/ucmnfsstore
working-directory: ${{github.workspace}}/ucm/csrc/ucmnfsstore
run: cmake -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DBUILD_TESTS=ON -DCOVERAGE_ENABLED=ON -DDOWNLOAD_DEPENDENCE=ON -DRUNTIME_ENVIRONMENT=simu

- name: Build
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ repos:
hooks:
- id: codespell
args: [
'--skip', 'unifiedcache/csrc/**,./unifiedcache.egg-info/**,.github/**',
'--skip', 'ucm/csrc/**,./ucm.egg-info/**,.github/**',
'-L', 'CANN,cann,NNAL,nnal,ASCEND,ascend,EnQue,CopyIn'
]
- repo: https://github.com/psf/black
Expand Down
File renamed without changes.
4 changes: 2 additions & 2 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ RUN export PLATFORM="cuda" && \

# Apply patch for vLLM
RUN cd $(pip show vllm | grep Location | awk '{print $2}') \
&& git apply /vllm-workspace/unified-cache-management/unifiedcache/patch/0.9.2/vllm-adapt.patch \
&& git apply /vllm-workspace/unified-cache-management/unifiedcache/patch/0.9.2/vllm-adapt-sparse.patch
&& git apply /vllm-workspace/unified-cache-management/ucm/patch/0.9.2/vllm-adapt.patch \
&& git apply /vllm-workspace/unified-cache-management/ucm/patch/0.9.2/vllm-adapt-sparse.patch

ENTRYPOINT ["/bin/bash"]
8 changes: 4 additions & 4 deletions docker/Dockerfile-NPU
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,12 @@ RUN export PLATFORM="ascend" && \

# Apply patch for vLLM
RUN cd /vllm-workspace/vllm \
&& git apply /vllm-workspace/unified-cache-management/unifiedcache/patch/0.9.2/vllm-adapt.patch \
&& git apply /vllm-workspace/unified-cache-management/unifiedcache/patch/0.9.2/vllm-adapt-sparse.patch
&& git apply /vllm-workspace/unified-cache-management/ucm/patch/0.9.2/vllm-adapt.patch \
&& git apply /vllm-workspace/unified-cache-management/ucm/patch/0.9.2/vllm-adapt-sparse.patch

# Apply patch for vLLM-Ascend
RUN cd /vllm-workspace/vllm-ascend \
&& git apply /vllm-workspace/unified-cache-management/unifiedcache/patch/0.9.2/vllm-ascend-adapt.patch \
&& git apply /vllm-workspace/unified-cache-management/unifiedcache/patch/0.9.2/vllm-ascend-adapt-sparse.patch
&& git apply /vllm-workspace/unified-cache-management/ucm/patch/0.9.2/vllm-ascend-adapt.patch \
&& git apply /vllm-workspace/unified-cache-management/ucm/patch/0.9.2/vllm-ascend-adapt-sparse.patch

CMD ["/bin/bash"]
2 changes: 1 addition & 1 deletion docs/source/getting-started/installation/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ docker run \
Refer to [Set up using docker](https://docs.vllm.ai/en/latest/getting_started/installation/gpu.html#set-up-using-docker) for more information to run your own vLLM container. After installation, please apply patch to ensure uc_connector can be used:
```bash
cd /vllm-workspace/vllm
git apply /vllm-workspace/unified-cache-management/unifiedcache/patch/0.9.2/vllm-adapt.patch
git apply /vllm-workspace/unified-cache-management/ucm/patch/0.9.2/vllm-adapt.patch
```
Refer to this [issue](https://github.com/vllm-project/vllm/issues/21702) to see details of this patch's changes.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,9 @@ docker run --rm \
Codes of vLLM and vLLM Ascend are placed in /vllm-workspace, you can refer to [vLLM-Ascend Installation](https://vllm-ascend.readthedocs.io/en/latest/installation.html) for more information. After installation, please apply patches to ensure uc_connector can be used:
```bash
cd /vllm-workspace/vllm
git apply /vllm-workspace/unified-cache-management/unifiedcache/patch/0.9.2/vllm-adapt.patch
git apply /vllm-workspace/unified-cache-management/ucm/patch/0.9.2/vllm-adapt.patch
cd /vllm-workspace/vllm-ascend
git apply /vllm-workspace/unified-cache-management/unifiedcache/patch/0.9.2/vllm-ascend-adapt.patch
git apply /vllm-workspace/unified-cache-management/ucm/patch/0.9.2/vllm-ascend-adapt.patch
```
Refer to these issues [vllm-issue](https://github.com/vllm-project/vllm/issues/21702) and [vllm-ascend-issue](https://github.com/vllm-project/vllm-ascend/issues/2057) to see details of patches' changes.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ CUDA_VISIBLE_DEVICES=0 vllm serve /home/models/Qwen2.5-7B-Instruct \
--kv-transfer-config \
'{
"kv_connector": "UnifiedCacheConnectorV1",
"kv_connector_module_path": "unifiedcache.integration.vllm.uc_connector",
"kv_connector_module_path": "ucm.integration.vllm.uc_connector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"ucm_connector_name": "UcmNfsStore",
Expand Down Expand Up @@ -54,7 +54,7 @@ CUDA_VISIBLE_DEVICES=1 vllm serve /home/models/Qwen2.5-7B-Instruct \
--kv-transfer-config \
'{
"kv_connector": "UnifiedCacheConnectorV1",
"kv_connector_module_path": "unifiedcache.integration.vllm.uc_connector",
"kv_connector_module_path": "ucm.integration.vllm.uc_connector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"ucm_connector_name": "UcmNfsStore",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ CUDA_VISIBLE_DEVICES=0 vllm serve /home/models/Qwen2.5-7B-Instruct \
--kv-transfer-config \
'{
"kv_connector": "UnifiedCacheConnectorV1",
"kv_connector_module_path": "unifiedcache.integration.vllm.uc_connector",
"kv_connector_module_path": "ucm.integration.vllm.uc_connector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"ucm_connector_name": "UcmNfsStore",
Expand All @@ -52,7 +52,7 @@ CUDA_VISIBLE_DEVICES=1 vllm serve /home/models/Qwen2.5-7B-Instruct \
--kv-transfer-config \
'{
"kv_connector": "UnifiedCacheConnectorV1",
"kv_connector_module_path": "unifiedcache.integration.vllm.uc_connector",
"kv_connector_module_path": "ucm.integration.vllm.uc_connector",
"kv_role": "kv_producer",
"kv_connector_extra_config": {
"ucm_connector_name": "UcmNfsStore",
Expand Down Expand Up @@ -80,7 +80,7 @@ CUDA_VISIBLE_DEVICES=2 vllm serve /home/models/Qwen2.5-7B-Instruct \
--kv-transfer-config \
'{
"kv_connector": "UnifiedCacheConnectorV1",
"kv_connector_module_path": "unifiedcache.integration.vllm.uc_connector",
"kv_connector_module_path": "ucm.integration.vllm.uc_connector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"ucm_connector_name": "UcmNfsStore",
Expand All @@ -106,7 +106,7 @@ CUDA_VISIBLE_DEVICES=3 vllm serve /home/models/Qwen2.5-7B-Instruct \
--kv-transfer-config \
'{
"kv_connector": "UnifiedCacheConnectorV1",
"kv_connector_module_path": "unifiedcache.integration.vllm.uc_connector",
"kv_connector_module_path": "ucm.integration.vllm.uc_connector",
"kv_role": "kv_consumer",
"kv_connector_extra_config": {
"ucm_connector_name": "UcmNfsStore",
Expand Down
2 changes: 1 addition & 1 deletion docs/source/user_guide/examples/dram_conn.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ vllm serve /home/models/Qwen2.5-14B-Instruct \
--kv-transfer-config \
'{
"kv_connector": "UnifiedCacheConnectorV1",
"kv_connector_module_path": "unifiedcache.integration.vllm.uc_connector",
"kv_connector_module_path": "ucm.integration.vllm.uc_connector",
"kv_role": "kv_both",
"kv_connector_extra_config": {
"ucm_connector_name": "UcmDram",
Expand Down
2 changes: 1 addition & 1 deletion docs/source/user_guide/examples/mooncake_conn.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ vllm serve /home/models/Qwen2.5-14B-Instruct \
--kv-transfer-config \
'{
"kv_connector": "UnifiedCacheConnectorV1",
"kv_connector_module_path": "unifiedcache.integration.vllm.uc_connector",
"kv_connector_module_path": "ucm.integration.vllm.uc_connector",
"kv_role": "kv_both",
"kv_connector_extra_config": {
"ucm_connector_name": "UcmMooncakeStore",
Expand Down
2 changes: 1 addition & 1 deletion docs/source/user_guide/examples/nfs_conn.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ vllm serve /home/models/Qwen2.5-14B-Instruct \
--kv-transfer-config \
'{
"kv_connector": "UnifiedCacheConnectorV1",
"kv_connector_module_path": "unifiedcache.integration.vllm.uc_connector",
"kv_connector_module_path": "ucm.integration.vllm.uc_connector",
"kv_role": "kv_both",
"kv_connector_extra_config": {
"ucm_connector_name": "UcmNfsStore",
Expand Down
2 changes: 1 addition & 1 deletion docs/source/user_guide/examples/sparse_attn.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ vllm serve /home/models/Qwen2.5-14B-Instruct \
--kv-transfer-config \
'{
"kv_connector": "UnifiedCacheConnectorV1",
"kv_connector_module_path": "unifiedcache.integration.vllm.uc_connector",
"kv_connector_module_path": "ucm.integration.vllm.uc_connector",
"kv_role": "kv_both",
"kv_connector_extra_config": {
"ucm_connector_name": "UcmNfsStore",
Expand Down
4 changes: 2 additions & 2 deletions examples/offline_inference.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from vllm.config import KVTransferConfig
from vllm.engine.arg_utils import EngineArgs

from unifiedcache.logger import init_logger
from ucm.logger import init_logger

logger = init_logger(__name__)

Expand Down Expand Up @@ -65,7 +65,7 @@ def print_output(


def main():
module_path = "unifiedcache.integration.vllm.uc_connector"
module_path = "ucm.integration.vllm.uc_connector"
name = "UnifiedCacheConnectorV1"
model = os.getenv("MODEL_PATH", "/home/models/Qwen2.5-14B-Instruct")

Expand Down
10 changes: 4 additions & 6 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@
from setuptools.command.develop import develop

ROOT_DIR = os.path.abspath(os.path.dirname(__file__))
SRC_DIR = os.path.join(ROOT_DIR, "unifiedcache", "csrc", "ucmnfsstore")
INSTALL_DIR = os.path.join(ROOT_DIR, "unifiedcache", "ucm_connector")
SRC_DIR = os.path.join(ROOT_DIR, "ucm", "csrc", "ucmnfsstore")
INSTALL_DIR = os.path.join(ROOT_DIR, "ucm", "store")
PLATFORM = os.getenv("PLATFORM")


Expand Down Expand Up @@ -104,9 +104,7 @@ def build_cmake(self, ext: CMakeExtension):

src_path = os.path.join(so_search_dir, so_file)
dev_path = os.path.join(INSTALL_DIR, so_file)
dst_path = os.path.join(
self.build_lib, "unifiedcache", "ucm_connector", so_file
)
dst_path = os.path.join(self.build_lib, "ucm", "store", so_file)
os.makedirs(os.path.dirname(dst_path), exist_ok=True)
shutil.copy(src_path, dst_path)
print(f"[INFO] Copied {src_path} → {dst_path}")
Expand All @@ -116,7 +114,7 @@ def build_cmake(self, ext: CMakeExtension):


setup(
name="unifiedcache",
name="ucm",
version="0.0.2",
description="Unified Cache Management",
author="Unified Cache Team",
Expand Down
6 changes: 3 additions & 3 deletions test/test_mooncake.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@

import torch

from unifiedcache.logger import init_logger
from unifiedcache.ucm_connector.base import Task
from unifiedcache.ucm_connector.ucm_mooncake import UcmMooncakeStore
from ucm.logger import init_logger
from ucm.store.base import Task
from ucm.store.ucm_mooncake import UcmMooncakeStore

logger = init_logger(__name__)

Expand Down
4 changes: 2 additions & 2 deletions test/test_uc_connector.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,14 @@
from vllm.sampling_params import SamplingParams
from vllm.v1.request import Request

from unifiedcache.integration.vllm.uc_connector import (
from ucm.integration.vllm.uc_connector import (
LoadPara,
ReqMeta,
SavePara,
UCConnectorV1Metadata,
UnifiedCacheConnectorV1,
)
from unifiedcache.ucm_connector.base import Task, UcmKVStoreBase
from ucm.store.base import Task, UcmKVStoreBase


def make_request(
Expand Down
2 changes: 1 addition & 1 deletion test/test_ucm_dram.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ def patch_cuda_for_cpu():


patch_cuda_for_cpu()
from unifiedcache.ucm_connector.ucm_dram import DramTask, UcmDram # isort: skip
from ucm.store.ucm_dram import DramTask, UcmDram # isort: skip


def make_request(
Expand Down
File renamed without changes.
Loading