Skip to content

Releases: bitdeep/vllm-serving-stack

vLLM serving stack v0.1.0

Choose a tag to compare

@bitdeep bitdeep released this 13 Sep 17:08

The first public release of the vLLM serving recipe: a local OpenAI-compatible endpoint for Qwen3.

  • vLLM image pinned by digest.
  • FP8 weights and KV cache, with explicit context and sequence limits.
  • Persistent weight cache and configurable GPU memory budget.
  • Reusable Docker Compose service and a curl chat example.

The recipe has passed Compose validation and real GPU chat generation. Capacity, latency and throughput depend on the GPU and workload; no performance benchmark is claimed.

Start with the tagged quickstart. This repository contributes the deployment recipe; the engine and model are credited upstream projects. No model weights are included.