HaloFPX v1.7.0 — Standalone Vulkan Engine (Mesa RADV Wave64) & Lean Containers

HaloFPX v1.7.0 — Standalone Vulkan Engine Release

This release establishes Vulkan (Mesa RADV Wave64) as the primary, default execution backend for HaloFPX on AMD Strix Halo (Ryzen AI Max APUs) and AMD Radeon GPUs.

Highlights:

• Standalone Static Vulkan Engine: Built with GGML_VULKAN=ON, GGML_HIP=OFF, and BUILD_SHARED_LIBS=OFF. Zero link-time dependencies on libhipblas.so.3 or libamdhip64.so.7. Runs everywhere on standard Linux distributions with Mesa RADV and Vulkan.
• Lean Default Container Image: The default Docker image drops the ~1.2 GB ROCm runtime layer, reducing container size from ~2.5 GB to ~350 MB, and requires only mounting /dev/dri (no /dev/kfd needed).
• Optional ROCm Profile: Users requiring ROCm HIP kernels for high-concurrency prefill can use docker compose --profile rocm up -d or compile with ./scripts/build_engine.sh --rocm.
• Automatic Backend Fallback: Engine lifecycle manager automatically selects Vulkan0 and cleanly handles fallback if ROCm0 is requested on non-ROCm systems.

Asset SHA256:

strix-halo-vulkan-engine-v1.7.0-linux-x86_64.tar.gz: 839f3398181d4df0fee61eb266a0ae923b1f6c86b61f89933895a2855399b3df

HaloFPX v1.6.1 — sharded pull, size-aware health poll, Ling MTP profile

what's new since v1.6.0:

• sharded pull fix: pull/delete now handle all *-of-* shards
  (previously only shard-00001 arrived); per-file checksums, total size,
  subdir paths. covered by tests.
• size-aware health poll: engine load wait scales with model size
  (6 s/GiB, 30 s floor) instead of flat 30 s — large-model loads no
  longer false-fail.
• ling3-flash MTP profile: 11-config sweep winner n2/p0.6 ships
  (55.1 tok/s, 94.3% acceptance, +22% over n4).
• ling3-flash docs: Q8_0 full-load limit recorded (126G file hangs
  124G-UMA boxes — requant source only); Q4_K_M + Q8_0 sha256 backfilled
  so pull verification is live.

docker: ghcr.io/julianmb/halofpx:1.6.1 (and :latest) — builds automatically from this release.

HaloFPX v1.6.0 — Ling 3.0 Flash zoo entry

what's new since v1.5.0:

• ling3-flash: Ling 3.0 Flash 124B MoE joins the zoo (bailingmoe3 arch).
  default Q4_K_M (71.72 GiB): coherence QA pass, wikitext-2 PPL 4.6064,
  Vulkan0 pp512 372.68 / tg128 45.23, MTP n4/p0.6 at 79–91% acceptance
  (+13% decode). ships mtp_enabled with bounded ctx 8192 + -fa off.
  Q8_0 (126.3 GiB) registered as reference/requant source.
  ROCmFP4 family explicitly parked (FAST: PPL 7.04 + incoherent output;
  STRIX_LEAN: repetitive garbage) — not published, Q4_K_M serves.
• BENCHMARKS.md gains the Ling §7 section with full methodology.

docker: ghcr.io/julianmb/halofpx:1.6.0 (and :latest) — builds automatically from this release.

HaloFPX v1.5.0 — Ornith MTP refresh, flash-next quants, baked ROCm image

what's new since v1.4.0 (25 commits):

• ornith-1.5-35b now ships mtp enabled: official aug-24 refresh requantized and re-benched on strix halo — 105.6 tok/s effective, 87.98% draft acceptance (n4/p0.6)
• qwen38-flash-next: two verified quants added — IQ4_XS (115.5G, paris-verified, insultated from hf) and ROCmFP4_FAST (113G) — plus per-variant hf repo support in the model manager
• docker: the rocm 7.2.3 runtime subset is baked into the default image — the server starts out of the box with no host mounts, both backends available (fixes #4). image is ~8G.
• engine behavior: spec-stateful mtp + prompt-cache concurrency, scaled context checkpoints on 128gb boxes
• registry hardening: mandatory quant provenance on every variant, ornith mtp config pinned to measured evidence
• glm-5.3-flash remains standalone (glm5nextrocm) until upstream pr ggml-org/llama.cpp#27754 lands and rocmfpx rebases

docker: ghcr.io/julianmb/halofpx:1.5.0 (and :latest) — builds automatically from this release.

HaloFPX v1.4.0 — Ollama-Compatible API

Highlights

🦙 Drop-In Ollama Compatibility

halofpx now exposes Ollama's core API surface — existing Ollama clients and tools work without changes:

export OLLAMA_HOST=http://localhost:8010
ollama list
ollama run ornith-1.5-35b:latest "Say hi"

Endpoint	Behavior
GET /api/version	Server version
GET /api/tags	Active model first + full zoo, with size/digest/quantization metadata
POST /api/chat	Streaming NDJSON and non-streaming
POST /api/generate	Same, with Ollama response semantics

• Ollama options mapped: num_predict, temperature, top_p, seed, stop
• Guards: 503 when no model is loaded, 409 when the requested model differs from the active one (no silent wrong-weights answers)
• Token stats counted client-side; engine usage honored when provided

Validated end-to-end against live Ornith-1.5-35B ROCmFP4 on gfx1151 (streaming NDJSON shape, non-streaming completions, model-mismatch rejection).

🧪 Tests

Suite grew to 33 tests, including SSE→NDJSON translation coverage with a mocked engine.

Full changelog: v1.3.0...v1.4.0

HaloFPX v1.3.0 — Vision, Security & Registry Hardening

Highlights

👁️ Automatic Vision (Multimodal) Support

• Model projectors (mmproj) are now first-class registry assets with repo/size/SHA256 metadata
• halofpx pull downloads and checksum-verifies the vision projector alongside the weights
• halofpx list reports vision readiness as a separate column; pull results include projector path
• Validated end-to-end: Ornith-1.5-35B answers image prompts via the standard OpenAI API

🔒 Security & Portability Sanitize

• CORS: allow_credentials disabled (invalid/unsafe with wildcard origins)
• Server now defaults to loopback (127.0.0.1) instead of 0.0.0.0; Docker still binds explicitly
• Blocking model load/unload/pull moved off the event loop (asyncio.to_thread)
• Version string centralized; machine-specific paths replaced by Path.home() + HALOFPX_HF_CACHE_DIRS / HALOFPX_ENGINE_SEARCH_PATHS overrides

🐛 Registry Fixes

• Fixed false "downloaded" reports from Hugging Face .no_exist negative-cache stubs
• Adopted Vulkan backend default for laguna-s21

📚 Documentation

• README refocused on optimized model serving with measured per-model decode numbers
• New benchmark section: quantization fidelity (perplexity ROCmFP4 5.95 vs Q4_K_M 5.64) and validated 131K/262K long-context results for Ornith-1.5-35B

Full changelog: v1.2.0...v1.3.0

HaloFPX v1.2.0 — Unified Model Zoo & Hybrid NPU Acceleration

🚀 HaloFPX v1.2.0 Release Notes

HaloFPX is the high-throughput unified model serving framework and model zoo designed specifically for AMD Strix Halo APUs (gfx1151 / Ryzen AI Max+ 395) and AMD Radeon discrete GPUs.

🌟 Key Highlights & Enhancements

• HaloFPX Branding & CLI (halofpx / halo): Full renaming to HaloFPX with updated Python package, CLI entrypoints (halofpx serve, halo list, halofpx load, halo status), and Docker ecosystem.
• Max Supported Context Windows: Default context sizes expanded across the entire model zoo:
  • Qwen 3.8 27B: 262,144 tokens (262K)
  • Nemotron 3.5 30B MoE: 262,144 tokens (262K)
  • Ornith 1.0 35B: 262,144 tokens (262K)
  • DeepSeek V4 Flash 284B MoE: 163,840 tokens (160K)
  • Laguna S 2.1: 131,072 tokens (128K)
• Multi-Model Hybrid NPU Pipeline (scripts/run_pipeline.py):
  • Connects the 50 TOPS AMD XDNA 2 NPU (/dev/accel/accel0) with any large target model on the iGPU.
  • Streams first ~24 tokens instantly from the NPU drafter (qwen3.5-0.8b-FLM / gemma3-1b-FLM) for sub-350ms TTFT (1.8× faster first-token latency) before seamlessly handing off to the iGPU.
• Standalone NPU Capabilities: Documented zero-VRAM offloads for audio transcription (whisper-v3-turbo-FLM), embeddings (embed-gemma-300m-FLM), and 2W always-on background request routing.
• Optimized Speculation Presets: Aligned default draft speculation parameters to empirical single-user optimum (K=4 / p0.0).
• Dynamic Hardware & Vulkan Detection: Auto-detects memory capacity, VRAM headroom, APU architecture, and multi-distro Vulkan ICDs.

📦 Quick Install & Usage

pip install -e .
halofpx list
halofpx serve --port 8010

ROCmFPX Server v1.1.0

ROCmFPX Server Release v1.1.0 — Multi-GPU Model Serving Framework for AMD Platforms

What's Changed in v1.1.0:

• Multi-GPU & VRAM Tier Sizing: Dynamic hardware detection across AMD APUs (Strix Halo 128G/64G) and discrete Radeon GPUs (16G, 24G, 32G) with rocmfpx list VRAM-fit guidance.
• Docker Container with Windows WSL2 & Linux Support: Modular Compose stack with default headless high-performance server and optional Open WebUI (--profile webui).
• Optional API Key Authentication: Secure endpoints with ROCMFPX_API_KEY (disabled by default for local development).
• Dynamic Vulkan ICD Discovery: Automatic resolution for Fedora, Arch, Debian, and RHEL distro paths.
• Model Zoo Updates: Supported variants for Qwen 3.8 27B (FP4, FP8, Q3, FP2), Nemotron 3.5 30B, Ornith 35B, DeepSeek V4 Flash 284B, and Laguna S 2.1.
• MTP Multi-Slot Concurrency: Configurable --slots, --draft-n, --draft-p, and --strict flags in API and CLI.

Assets:

• strix-halo-rocmfpx-engine-v1.0.0-linux-x86_64.tar.gz (SHA256: bbc7845db0c012b97f1c9b8a2733a7083c6f9a749a453866fbe1994151d3364f)

ROCmFPX Server v1.0.0 for AMD Strix Halo

ROCmFPX Server v1.0.0 — Unified Model Serving Framework for AMD Strix Halo

Features:

• Unified Model Zoo: Download, verify, and serve quantized models (Qwen 3.8 27B, Nemotron 3.5 30B, Ornith 35B, DeepSeek V4 Flash, Laguna S 2.1) directly from Hugging Face.
• Single-Endpoint Architecture: Serves standard OpenAI /v1/chat/completions (with streaming SSE) + Lemonade-style management API /api/v1/{load, unload, pull, status, system-info} on port 8010.
• Dual-Backend Acceleration: Automatic Vulkan0 (Mesa RADV Wave64 KHR_coopmat) + ROCm0 HIP execution.
• Docker Compose Stack: 1-command deployment with Open WebUI integration.
• MTP Speculation & TurboQuant KV Cache: Sub-linear memory scaling and up to 36 tok/s generation throughput.

Included Asset:

• strix-halo-rocmfpx-engine-v1.0.0-linux-x86_64.tar.gz (SHA256: bbc7845db0c012b97f1c9b8a2733a7083c6f9a749a453866fbe1994151d3364f)