Skip to content

v0.8.4 — Windows CUDA backend node (#365)

Choose a tag to compare

View all tags
@dahai80 dahai80 released this 06 Aug 23:51
v0.8.4
7447219
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Patch release.

Windows CUDA backend node (#365)

Optional vLLM-powered OpenAI-compatible server for heavy LLM inference (DeepSeek 70B / Qwen 72B FP8) on Windows CUDA hosts. New fusion-mlx cuda-node subcommand builds a FastAPI app embedding vLLM's AsyncLLMEngine, serving /health, /v1/models, /v1/chat/completions, /v1/completions. The node self-registers with the cluster via mDNS under platform=windows-cuda so a fusion-gateway can route heavy-model intents to it (gateway-side platform routing landed in v0.8.0).

Platform detection (FUSION_PLATFORM env → sys.platform + CUDA probe → mac) surfaces a platform TXT record on every node's mDNS advertisement. vLLM is imported lazily so the package stays importable on Mac; starting the node without vLLM raises a clear RuntimeError. LLM-only scope (diffusion-on-CUDA tracked separately).

See docs/cuda-node.md.

Install (Windows CUDA host): pip install vllm then pip install fusion-mlx.

PyPI upload is blocked by #384 (trusted-publisher re-registration). Homebrew tap updated below.

Assets 2
Loading