vLLM RBLN Plugin

This repository provides the hardware plugin that enables vLLM on RBLN NPUs, including ATOM and REBEL.

Built on top of vLLM’s Plugin System, it allows seamless integration with the vLLM ecosystem and provides high-throughput, low-latency LLM serving on RBLN hardware. Our plugin supports a wide range of popular LLMs and continues to expand to support all features enabled in vLLM, including advanced attention mechanisms.

• rebel-compiler
• optimum-rbln

You can install this project using pip or from source.

uv pip install vllm-rbln --extra-index-url https://wheels.vllm.ai/0.30.0/cpu --torch-backend cpu

pip install vllm-rbln --extra-index-url https://wheels.vllm.ai/0.30.0/cpu --extra-index-url https://download.pytorch.org/whl/cpu

⚠️ Installing on Ubuntu 22.04 or Enterprise Linux 9

If installation fails with an error like the following, use the installation command below:

vllm==0.30.0+cpu has no wheels with a matching platform tag
(e.g., `manylinux_2_35_x86_64`)

hint: Wheels are available for `vllm` (v0.30.0+cpu) on the following platforms:
`manylinux_2_39_aarch64`, `manylinux_2_39_x86_64`

This error means that the upstream vLLM CPU wheel requires a newer glibc version than your system provides. The wheel requires glibc 2.39 or later and libstdc++ from GCC 14 or later (CXXABI_1.3.15). Ubuntu 22.04 (glibc 2.35, GCC 12) and Enterprise Linux 9 (RHEL, AlmaLinux, and Rocky Linux 9.x; glibc 2.34, GCC 11) do not meet these requirements.

The following command uses --find-links to install a compatible vLLM wheel from the vllm-rbln v0.12.0 release assets:

pip3 install \
  --extra-index-url https://wheels.vllm.ai/0.30.0/cpu \
  --extra-index-url https://pypi.rbln.ai/simple \
  --extra-index-url https://download.pytorch.org/whl/cpu \
  --find-links https://github.com/RBLN-SW/vllm-rbln/releases/expanded_assets/v0.12.0 \
  "vllm-rbln[kv-connectors]==0.12.0"

git clone https://github.com/rbln-sw/vllm-rbln.git
cd vllm-rbln
uv pip install -e .

git clone https://github.com/rbln-sw/vllm-rbln.git
cd vllm-rbln
pip install -e . --extra-index-url https://wheels.vllm.ai/0.30.0/cpu --extra-index-url https://download.pytorch.org/whl/cpu

Building from source? See DEVELOPMENT.md for the uv dev workflow.

• Overview & Supported Models
• API Tutorial

We welcome all contributions! Whether it's reporting issues, proposing enhancements, or improving docs—your input helps make the project better.

See our CONTRIBUTING.md for more information.

This project is licensed under the Apache License 2.0.

See the LICENSE file for more information.

• Join discussions and get answers in our Developer Community
• Contact maintainers at support@rebellions.ai