Skip to content

HaloFPX v1.2.0 — Unified Model Zoo & Hybrid NPU Acceleration

Choose a tag to compare

View all tags
@julianmb julianmb released this 18 Aug 23:36
· 76 commits to main since this release
v1.2.0
bcb7b39

🚀 HaloFPX v1.2.0 Release Notes

HaloFPX is the high-throughput unified model serving framework and model zoo designed specifically for AMD Strix Halo APUs (gfx1151 / Ryzen AI Max+ 395) and AMD Radeon discrete GPUs.

🌟 Key Highlights & Enhancements

  • HaloFPX Branding & CLI (halofpx / halo): Full renaming to HaloFPX with updated Python package, CLI entrypoints (halofpx serve, halo list, halofpx load, halo status), and Docker ecosystem.
  • Max Supported Context Windows: Default context sizes expanded across the entire model zoo:
    • Qwen 3.8 27B: 262,144 tokens (262K)
    • Nemotron 3.5 30B MoE: 262,144 tokens (262K)
    • Ornith 1.0 35B: 262,144 tokens (262K)
    • DeepSeek V4 Flash 284B MoE: 163,840 tokens (160K)
    • Laguna S 2.1: 131,072 tokens (128K)
  • Multi-Model Hybrid NPU Pipeline (scripts/run_pipeline.py):
    • Connects the 50 TOPS AMD XDNA 2 NPU (/dev/accel/accel0) with any large target model on the iGPU.
    • Streams first ~24 tokens instantly from the NPU drafter (qwen3.5-0.8b-FLM / gemma3-1b-FLM) for sub-350ms TTFT (1.8× faster first-token latency) before seamlessly handing off to the iGPU.
  • Standalone NPU Capabilities: Documented zero-VRAM offloads for audio transcription (whisper-v3-turbo-FLM), embeddings (embed-gemma-300m-FLM), and 2W always-on background request routing.
  • Optimized Speculation Presets: Aligned default draft speculation parameters to empirical single-user optimum (K=4 / p0.0).
  • Dynamic Hardware & Vulkan Detection: Auto-detects memory capacity, VRAM headroom, APU architecture, and multi-distro Vulkan ICDs.

📦 Quick Install & Usage

pip install -e .
halofpx list
halofpx serve --port 8010
Assets 2
Loading