veloGB10 v0.5.4

veloGB10 v0.5.4 — Built-in OpenTelemetry, FP8 support, DFlash2 tree mode

Built-in OpenTelemetry. --otel-endpoint streams OTLP/HTTP-JSON generation telemetry (the
actual SSE chunk bytes, with model.id / topology / request.id / token.index / event attributes), off
by default, near-zero decode interference. request.id is now the conversation key so a reply's turns
join one continuous session; generation.id stays per-POST. Companion flags: --otel-batch-size,
--otel-batch-interval-ms, --otel-include-tokens, --otel-model-id, --otel-topology.

FP8 support expanded. Direct load of Qwen fine-grained block-128 FP8 (weight_scale_inv); DFlash2
FP8 drafter bake (--df2-bake-fp8) plus weight-only NVFP4 (--df2-bake-nvfp4); --df2-quant-fidelity
gate tool. Fixed FP8 kernel bugs across gpu_batch.cu.

DFlash2 tree verification. New opt-in --spec-source dflash2-tree mode (additive; MTP and
DFlash2 unchanged). Minor bug fixes and optimizations.

Source: dev repo @ e94430c. Built from the public tree (self-contained), binary sha256
269607fc2801210a44fbfe35c16b121cf924d70d9a23e5b97cbd884abfe495aa.

Downloads

• velogb10-v0.5.4-gb10-sm121.tar.gz - engine binary + all 15 PTX kernel artifacts (incl. vision) + TP launch scripts
• SHA256SUMS.txt - checksums
• PROVENANCE.txt - build provenance

veloGB10 v0.5.2

veloGB10 v0.5.2 — vLLM-compatible tokenize / detokenize endpoints

POST /v1/tokenize — vLLM-compatible tokenization: {tokens, count, max_model_len}, a pure
tokenizer call (no forward / KV / GPU). Accepts a prompt string or a chat messages array; the
messages mode renders exactly as the chat path so its count equals usage.prompt_tokens. Empty
prompt returns {tokens: [], count: 0} (vLLM behavior); over-length returns 400
code=context_length_exceeded (truncation keeps the last n tokens).

POST /v1/detokenize — vLLM-compatible decode half of the pair ({model, prompt}) for exact-N
prompt building.

Added so our engine can be benchmarked more correctly. max_model_len mirrors the configured context
size. Minor bug fixes and optimizations.

Source: dev repo @ 5f6b77c. Built from the public tree (self-contained), binary sha256
dbaf23dc88c61363778d25871af961a2a96eca87ff89a3e0da085601e6e68e1e.

Downloads

• velogb10-v0.5.2-gb10-sm121.tar.gz - engine binary + all 15 PTX kernel artifacts (incl. vision) + TP launch scripts
• SHA256SUMS.txt - checksums
• PROVENANCE.txt - build provenance

veloGB10 v0.5.1

veloGB10 v0.5.1 — Vision robustness, reasoning-effort, graceful-load fixes

Vision generalization + boot fix. The GPU vision tower now bootstraps opportunistically: a
non-vision or geometry-incompatible model serves text-only instead of crashing at startup (fixes a
v0.5.0 boot crash on non-27B packs, issue #2). Vision is generalized across the Qwen3.5/3.8 VL
family, so all vision-tower models serve images.

OpenAI reasoning_effort. Full level table (none/low/medium/high/xhigh/max) with per-family
normalization plus --reasoning-effort; the high mapping no longer silently drops thinking
(regression fix).

Tool-call + reasoning-mode fix. Tool-call markup is held back in reasoning mode too, fixing a
first-call double-emit leak.

Graceful model-load exit. Corrupted / stale / wrong-format checkpoints exit with a clear
actionable message instead of a panic / OOM / core-dump.

--output-prompts [n] human-readable chat-request logging; --vision-cpu now listed in --help.
Minor bug fixes and optimizations.

Source: dev repo @ 704688f. Built from the public tree (self-contained), binary sha256
fb5f173b745de85c96177d1ee955d095e4f610ca82c33ff39402b7bf447a2bbc.

Downloads

• velogb10-v0.5.1-gb10-sm121.tar.gz - engine binary + all 15 PTX kernel artifacts (incl. vision) + TP launch scripts
• SHA256SUMS.txt - checksums
• PROVENANCE.txt - build provenance

veloGB10 v0.5.0

veloGB10 v0.5.0 — Vision support

Vision. Image input is supported end-to-end on a GPU vision tower (gpu_vision kernels), with
PNG/JPEG/WebP/GIF decoding and a --vision-cpu escape hatch to the CPU reference path. The release
now ships the gpu_vision.ptx kernel artifact alongside the existing PTX set.

Better tool-call support. A single canonical serializer now handles streaming and non-streaming
tool-call output identically, repairs malformed tool-call tags, and no longer drops or leaks text
around tool-call boundaries. New tool-call compliance and serializer test suites.

Prefill / TTFT optimizations. New opt-in prefill levers (tensor-core flash-attention prefill,
v2 W4A4 prefill GEMM, GDN tensor-core chunked scan), all env-gated default off, so the default
serving path is unchanged. Minor bug fixes and optimizations.

Model-id fix. /v1/models and responses now report the model card's base_model instead of a local
directory fragment. --model-name still overrides.

See CHANGELOG.md for the full release history.

Source: dev repo @ 9b60021. Built from the public tree (self-contained), binary sha256
6d6aa435704d2843aa6340036da868c7abe4481c825dde16b43c344a66f2a8e4.

Downloads

• velogb10-v0.5.0-gb10-sm121.tar.gz - engine binary + all 15 PTX kernel artifacts (incl. vision) + TP launch scripts
• SHA256SUMS.txt - checksums
• PROVENANCE.txt - build provenance

veloGB10 v0.4.2

veloGB10 v0.4.2 — HOTFIX: accept OpenAI multipart content

Fix: ChatMessage.content deserialized as Option, but the OpenAI spec allows content to
be a STRING or an ARRAY of content parts (text / image_url). Agent clients (OpenAI agent SDK, Pi
harness) send the array form, which serde rejected with 422
(messages[N].content: invalid type: sequence, expected a string). Purely request-schema; the
engine/serve path does not change.

• string -> used verbatim
• null/absent -> None (agents send content: null on the assistant tool_calls turn)
• array -> extract {type:text,text:...} parts; single part verbatim, multiple joined with newline

Adds tests/hotfix_content/ fixtures covering the accepted shapes.

Source: dev repo @ 72e20be. Built from the public tree (self-contained), binary sha256
d7f0ae3a7eb49ad528a5c574946e9a07f1974cedb6312e5f1210cd4a3e850566.

Downloads

• velogb10-v0.4.2-gb10-sm121.tar.gz - engine binary + all PTX kernel artifacts + TP launch scripts
• SHA256SUMS.txt - checksums
• PROVENANCE.txt - build provenance

veloGB10 v0.4.1

veloGB10 v0.4.1 — fix --draft-dir gate

The --draft-dir check was previously unconditional, which broke plain-MTP launches
(because --spec-source defaults to dflash2-auto when absent). Now --draft-dir is
mandatory (exit 2) only when --spec-source explicitly names dflash2 / dflash2-rq /
dflash2-auto; it is optional otherwise (MTP fallback, no round). A provided but
nonexistent directory is always a hard stop, and the rule is validated before any
model load or GPU work.

Source: dev repo @ 65a9c01. Built from the public tree (self-contained), binary
sha256 353f9a38f0dc3b1df1343ddb96266a5d971822de64eab8b49b0ef435dd5c5b31.

Downloads

• velogb10-v0.4.1-gb10-sm121.tar.gz - engine binary + all PTX kernel artifacts + TP launch scripts
• SHA256SUMS.txt - checksums
• PROVENANCE.txt - build provenance

veloGB10 v0.4.0

veloGB10 v0.4.0 — Qwen3.8 27B NVFP4 + DFlash 2, TP=4

Qwen3.8 27B NVFP4 support with native DFlash 2 speculative decoding, full 256K context, and TP=4 serving.

• New Qwen3.8 27B NVFP4-FULL model support, launched with --spec-source dflash2-auto --draft-dir <dflash2 dir>.
• TP=4 serving (three nodes + head), plus TP=2 and single-node.
• New DSV4 / DFlash2 / DSpark / MXFP4 kernel set (all kernels/*.cu + all src/ptx/*.ptx).
• README: Update section with Qwen3.8 27B performance table and live throughput traces.
• New docs: QWEN_27B_SETUP.md and MANAGING_CACHE.md.

Performance (Qwen3.8 27B NVFP4 + DFlash 2, greedy, full 256K ctx)

Mode	Average	Bottoms	Peaks
Single node	> 30 tok/s	~18 tok/s	45-50 tok/s
TP=2	56 tok/s	~30 tok/s	75 tok/s
TP=4	75 tok/s	~45 tok/s	125 tok/s

Peak rates are typically on code content generation. See the README for the full performance traces.

Downloads

• velogb10-v0.4.0-gb10-sm121.tar.gz - the engine binary + all PTX kernel artifacts + TP launch scripts
• SHA256SUMS.txt - checksums
• PROVENANCE.txt - build provenance

Built from the public repo tree (self-contained), binary sha256 ef0007a533ee5f476aed3ae3930873b991e8190d323ae3fe5603f102cef6f90e.

veloGB10 v0.3.1

veloGB10 v0.3.1 — KAT-Coder support, supported-models table

This release adds KAT-Coder-V2.5-Dev to the supported-model family and presents all supported models in a table with links to their HuggingFace artifacts.

Binary sha256: e159a701dd3ef723aba90cb43ec386159f4aaa66bd2cb777862bb4bc73ffb32e

Model cards and weights: https://huggingface.co/doth4580

veloGB10 v0.3.0

veloGB10 v0.3.0 — README generalization + load pipeline features

This release broadens the README to describe veloGB10 as an engine for a hand-selected set of large language models (currently Qwen3.5/3.6 and Tencent Hy3), rather than a Qwen3.5/3.6-only engine. It also ships the recent load-speed work: parallel assembly pipeline and bounded async uploads.

Binary sha256: a079b5ba0221b4e4c2ffada7affeb146802b5fd11a36c2c2f7b2094fab5783b7

Model cards and weights: https://huggingface.co/doth4580

veloGB10 v0.2.0 — Hy3 family support

veloGB10 v0.2.0 — Hy3 (hy_v3) family support

Headline: Hy3 (295B-A21B MoE) serving incl. TP=2 shard-at-load, MTP speculative decoding (96.9% acceptance), FR-Spec draft head, 4-bit KV cache (GB10_KV_QUANT), head_dim-generic attention (128..512), MoE shared-expert fusion, CLI session flags, curated run presets, tests/ now in the curated set.

Built from the public tree at tag v0.2.0 — the repo builds standalone.
Binary sha256: 9e9300bb9fa01c4111f07cb6bbf1519254b60bd52d99c779340601c8d9550c96

Artifact: velogb10-v0.2.0-gb10-sm121.tar.gz — engine binary + required PTX kernels + TP=2 launch scripts. Verify with sha256sum -c SHA256SUMS.txt; details in PROVENANCE.txt.

Runtime: NVIDIA DGX Spark (GB10, sm_121), current CUDA driver; one machine or two (TP=2 over ConnectX-7). Model weights are NOT included — see https://huggingface.co/doth4580 for the NVFP4 model artifacts.