The first public release of the vLLM serving recipe: a local OpenAI-compatible endpoint for Qwen3.
- vLLM image pinned by digest.
- FP8 weights and KV cache, with explicit context and sequence limits.
- Persistent weight cache and configurable GPU memory budget.
- Reusable Docker Compose service and a curl chat example.
The recipe has passed Compose validation and real GPU chat generation. Capacity, latency and throughput depend on the GPU and workload; no performance benchmark is claimed.
Start with the tagged quickstart. This repository contributes the deployment recipe; the engine and model are credited upstream projects. No model weights are included.