Paiton

More from your AMD GPU.
Faster local chat, coding, image and video models for AMD Radeon —
as ready-to-run containers or a plugin for your own vLLM.

Quick start · Models · Launch commands · Your own vLLM · Docs · Releases · Paiton Studio

Paiton makes open models run faster on AMD Radeon GPUs. It combines Paiton's compiler-generated GPU kernels with established runtimes — vLLM for language models, Diffusers and ComfyUI for images and video — and ships every model with a setup guide, a defined support scope and a reproducible benchmark.

Tested on one Radeon AI PRO R9700 (32 GB, RDNA4, gfx1201). Other GPUs have not been qualified.

Measured on one Radeon AI PRO R9700. Three examples from the model library:

Different workloads — not a cross-model ranking or a universal speedup claim. How to read the numbers →

Two ways to run a model; both serve the same OpenAI-compatible API. The example uses MiniCPM5-2B, the smallest download in the library (2.11 GB).

A. Run a container — needs Linux, Docker and access to the AMD GPU. The container brings its own inference environment.

git clone --depth 1 https://github.com/Eliovp-BV/paiton-vllm-plugin.git
cd paiton-vllm-plugin
./models/MiniCPM5-2B/serve-docker.sh

B. Use your own vLLM environment — first activate the model's supported vLLM build. Paiton does not change your vLLM, PyTorch or ROCm installation.

python -m pip install https://github.com/Eliovp-BV/paiton-vllm-plugin/releases/download/v0.3.4/paiton_vllm_plugin-0.3.4-py3-none-any.whl
paiton serve minicpm5

The first launch downloads the weights (reusing your Hugging Face cache when it already has them) and may build runtime components. When the server reports that it is ready, send a request from another terminal:

curl http://127.0.0.1:8036/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "minicpm5-2b",
    "messages": [{"role": "user", "content": "Hello!"}],
    "max_tokens": 64
  }'

For a terminal chat, run python3 models/MiniCPM5-2B/chat.py from the repository.

Next: pick another model · already have the weights? · requirements

Each model name links to its guide: weights, launch options, API examples, limits and the full benchmark report.

_{Context = input + generated tokens. GPU memory = sampled driver VRAM at the tested settings, including runtime overhead; it is not a minimum requirement. — = not reported. C1, C2, C8 = one, two, eight concurrent requests.}

All language models expose an OpenAI-compatible API through vLLM.

• Reasoning tokens share the output budget with the answer; choose the mode explicitly when comparing responses or benchmarking. MiniCPM5's W4 thinking mode is experimental; direct answers are its default.
• Qronos and Ornith are text-only in these packages, although their upstream architectures include vision components.
• NEO runs the author's mixed GGUF weights directly through Paiton's vLLM integration: see its image API and native GGUF notes. Its MTP path is disabled.

The Qwen-Image container can also serve Paiton's MXFP4 conversion of an uncensored fine-tune (--model uncensored; its R9700 performance has not been measured). A portable MXFP4 package covers other hardware and runtimes. FLUX image editing is not qualified.

Starting from an image? Use Wan2.2 or MiniMax H3; FastWan is qualified for text input only. Video timings include file encoding. Gains from distillation or fewer sampling steps are not counted as Paiton acceleration.

Meeting (review candidate) turns a recording into a transcript with anonymous speaker labels and timestamped notes, using Parakeet speech recognition, speaker diarization and a compact Granite summary model, from the CLI or a container. 2.2% lower processing time vs stock: 291.0 → 284.5 s for a 39-minute meeting.

Run launchers from the repository root. Run one model at a time on the tested single-GPU setup (several language models use port 8000).

Language models

¹ Prepare the target and draft weights first (how); set PAITON_W3ROT_DIR to add the optional 3-bit weights; use run-rocm10-200k.sh for the 200K profile. ² Text-only, 65K, without speculative decoding; see native presets.

--chat opens a terminal chat once the server is ready. Each model guide shows its chat command and API examples.

Images, video and meeting notes

The ComfyUI launchers also need Docker Compose. The included workflows expose the supported prompts, inputs and generation settings.

Cloning this repository gets you the launchers and guides, not the model weights.

• First launch: most launchers and native presets download their pinned weights automatically. Qwen3.8 MXFP4 + DFlash2 and Meeting need a preparation step first.
• Already downloaded? Every model guide has a Model weights and existing downloads section for weights in a local folder or in your Hugging Face cache. Use the exact model and quantization it lists. Containers only see host caches that are mounted as described there.
• Native CLI: weights already in your Hugging Face cache are reused and missing files are downloaded. For a checkpoint folder elsewhere, run paiton --model-dir /absolute/path serve NAME.

The weights and cache guide explains cache locations (HF_HOME, HF_HUB_CACHE), exact revisions and Docker mounts.

The paiton CLI serves a supported model by name inside an existing vLLM environment. It selects the pinned checkpoint, downloads and verifies the matching native bundle, and starts vLLM with the qualified profile.

• It needs neither the private compiler nor a repository checkout.
• It leaves your vLLM, PyTorch and ROCm installation untouched, and explains the mismatch instead of starting when the runtime or payload does not match.
• Installing it does not change plain vllm serve commands.

Install the wheel as in the Quick start, activate the preset's environment, then:

paiton doctor           # check the environment
paiton models           # list the presets and their preparation status
paiton serve minicpm5   # launch a preset by name

These are separate existing environments; Paiton does not switch between them. Image, video and meeting models use their launchers instead.

# Verify and remember an existing checkpoint without starting the server
paiton --model-dir /models/existing-minicpm5 --prepare-only serve minicpm5

# No downloads and no outbound runtime networking
paiton --offline serve minicpm5

# vLLM arguments go after the model
paiton serve minicpm5 --port 9000 --served-model-name local-mini

# An explicit profile with a local path (repository IDs also work)
paiton --profile minicpm5-awq-text-8k vllm serve /models/existing-minicpm5 --port 9000

• GPU: tested on one Radeon AI PRO R9700 (32 GB, RDNA4). Smaller GPUs have not been qualified.
• Containers: Linux, Docker and access to the AMD GPU devices (/dev/kfd, /dev/dri). The ComfyUI launchers also need Docker Compose.
• Your own vLLM: the preset's supported environment, listed above.
• First launch: downloads the weights and may build or compile runtime components; later launches reuse persistent caches. Host RAM, disk space and preparation time vary substantially, so check the model guide before downloading.
• GPU memory: reported figures apply to the tested settings. Longer context, more concurrent requests or higher resolutions can need more.

Each result compares Paiton with a baseline on the same model and workload; the library is not a ranking across models. Every report documents the baseline, sampling settings, repetitions, quality checks and known limitations.

Public integrations. Compiled runtime artifacts. A proprietary compiler.

This repository distributes Paiton's public integrations and compiled runtime artifacts. The Paiton compiler remains proprietary. Model weights are fetched from their pinned publishers and retain their own licenses.

The vLLM plugin is Apache-2.0 licensed. Bundled components retain their applicable licenses, including ComfyUI and separate image tools. See the root notices and each model's notices.

Paiton's broader work also includes AMD Instinct accelerators and multi-GPU inference. Explore Paiton or discuss your workload →

Build locally. Get more from your GPU.

Release downloads · Containers · Hugging Face · Paiton Studio · Report an issue