Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions docs/source/getting-started/installation_gpu.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,13 @@ docker run \
```
Refer to [Set up using docker](https://docs.vllm.ai/en/latest/getting_started/installation/gpu.html#set-up-using-docker) for more information to run your own vLLM container.

### Install by pip
Install by pip or find the pre-build wheels on [Pypi](https://pypi.org/project/uc-manager/).
```
pip install uc-manager
```


### Build from source code
Follow commands below to install unified-cache-management:

Expand All @@ -46,14 +53,25 @@ pip install -v -e . --no-build-isolation

**Note:** Patches are now applied automatically via dynamic patching when you import the unified-cache-management package. You no longer need to manually apply patches using `git apply`. The patches are automatically applied when you use the `UnifiedCacheConnectorV1` connector.


## Setup from docker

### Build image from source
Download the pre-built `vllm/vllm-openai:v0.9.2` docker image and build unified-cache-management docker image by commands below:
```bash
# Build docker image using source code, replace <branch_or_tag_name> with the branch or tag name needed
git clone --depth 1 --branch <branch_or_tag_name> https://github.com/ModelEngine-Group/unified-cache-management.git
cd unified-cache-management
docker build -t ucm-vllm:latest -f ./docker/Dockerfile ./
```


### Pre-built images

```bash
docker pull unifiedcachemanager/ucm:latest
```

Then run your container using following command. You can add or remove Docker parameters as needed.
```bash
# Use `--ipc=host` to make sure the shared memory is large enough.
Expand Down