Skip to content

v2.0 - One-click Native vLLM + ROCm for AMD RDNA2 on Windows

Choose a tag to compare

View all tags
@sebastianmechno-sys sebastianmechno-sys released this 18 Aug 09:36
· 27 commits to main since this release
V2.0
8b8115e

v2.0 - One-click Native vLLM + ROCm for AMD RDNA2 on Windows

v2.0 — One-click native vLLM + ROCm 7.15 for AMD RX 6000 (RDNA2) on Windows

Native vLLM + ROCm 7.15 (TheRock) for the whole AMD Radeon RDNA2 family on
Windows — no WSL2, no NVIDIA, no compiler. One-click installer, everything
prebuilt, plus an OpenAI-compatible chat server that works like the NVIDIA stack.

Verified on AMD Radeon RX 6750 XT 12 GB (gfx1031) — Windows 11 — August 2026

Verification

Test Result
ROCm GPU detection (native process) HIP 7.15.26290, 12.0 GB VRAM, 20 CUs
rocBLAS FP16 GEMM 4096×4096×4096 25 674 Gflops (≈26 TFLOPS)
vLLM decode, Qwen3.5-4B AWQ 4-bit, 512 tok 59.4 tok/s (peak 62.5)
Optimization progression 8.3 → 62.5 tok/s (7.5×)

Installation

You do not need to extract or download these archives manually.
Just clone/download the repository and run INSTALL.bat — the installer
auto-elevates, checks GPU and disk, downloads the four archives below from
this release, installs Python 3.11.9, fetches the Qwen3.5-4B 4-bit model from
Hugging Face and runs a verification benchmark.

Then:

  • CHAT.bat — open the web chat (server starts automatically)
  • SERVE.bat — OpenAI-compatible API at http://127.0.0.1:8000/v1
  • VERIFY.bat — full 3-step verification

Installed layout:
C:\Python311 Python 3.11.9 C:\TheRock.venv torch 2.12+rocm7.15 venv (vLLM 0.19.1) C:\TheRock\build\dist\rocm ROCm runtime C:\TheRock\ROCM_VLLM_RUNTIME vLLM + plugin + rocBLAS + rocblas-bench C:\vw_cext_build, C:\vw_hipgemv_build native HIP kernels

Assets (the installer downloads these automatically)

File Size Content
the-rock-venv.tar.zst 1.34 GB torch ROCm venv
therock-rocm-dist.tar.zst 0.84 GB ROCm runtime (HIP, rocBLAS, Tensile)
vllm-stack.tar.zst 0.14 GB vLLM + plugin + rocBLAS + rocblas-bench.exe
native-kernels.tar.zst ~1 MB HIP GEMV kernels (fat binary gfx1030/1031/1032)

Requirements

Item Requirement
OS Windows 10/11 (Windows 11 recommended)
GPU AMD RDNA2 — RX 6400–6950, 8+ GB VRAM
Driver AMD Adrenalin (standard gaming driver)
Disk ~25 GB free on C:
Internet during installation only (~6 GB)

Highlights

  • Native HIP W4 GEMV fat-binary kernel for the entire RDNA2 family
  • Tuned decode: CUDA graphs + M=1 GEMV + weight-cast caching (7.5× throughput)
  • OpenAI-compatible API + browser chat (Thinking… spinner, final answer only)
  • Apache 2.0 — built on ROCm/TheRock, PyTorch ROCm and the vLLM project. Not affiliated with AMD
Assets 9
Loading