v0.8.6

In-place LoRA swap (#389)

Keep a single base engine resident and swap the LoRA adapter onto it in place via mlx_lm's LoRALinear machinery (load_adapters / remove_lora_layers) instead of reloading the full base model per adapter.

• Correct for 4-bit/8-bit quantized bases (low-rank arrays added beside the quantized linear, never fused into packed weights); allocates no second base copy.
• Per-base asyncio.Lock serializes the apply → infer → restore window; bare-base requests wait for any in-flight swap to restore.
• Opt-in via FUSION_LORA_INPLACE_SWAP=1 (default OFF preserves existing per-adapter derived-engine behavior). Requires FUSION_LORA_ALLOWED_DIRS allow-list.
• Switch latency ~7 ms apply / ~1 ms restore on Qwen3-0.6B-4bit.

See docs/lora-inplace-swap.md.

Full Changelog: v0.8.5...v0.8.6

v0.8.5

Patch release.

• #372: Accept HuggingFace repo ids in model resolution — /v1/chat/completions (and all OpenAI/Anthropic routes) now accept org/repo forms like mlx-community/Qwen3-0.6B-4bit in addition to registry short names. EnginePool._resolve_hf_repo_id matches source_repo_id before lookup; unmatched ids return 404 with available short names (no more opaque 502).
• #390: Fix stale --lora-path CLI help — now reflects that per-request adapter hot-swap is supported via the request adapters field.

Tests: 7 new (TestHfRepoIdResolution); full engine_pool 105 pass. Lint clean. Local 1534 pass.

v0.8.4 — Windows CUDA backend node (#365)

Patch release.

Windows CUDA backend node (#365)

Optional vLLM-powered OpenAI-compatible server for heavy LLM inference (DeepSeek 70B / Qwen 72B FP8) on Windows CUDA hosts. New fusion-mlx cuda-node subcommand builds a FastAPI app embedding vLLM's AsyncLLMEngine, serving /health, /v1/models, /v1/chat/completions, /v1/completions. The node self-registers with the cluster via mDNS under platform=windows-cuda so a fusion-gateway can route heavy-model intents to it (gateway-side platform routing landed in v0.8.0).

Platform detection (FUSION_PLATFORM env → sys.platform + CUDA probe → mac) surfaces a platform TXT record on every node's mDNS advertisement. vLLM is imported lazily so the package stays importable on Mac; starting the node without vLLM raises a clear RuntimeError. LLM-only scope (diffusion-on-CUDA tracked separately).

See docs/cuda-node.md.

Install (Windows CUDA host): pip install vllm then pip install fusion-mlx.

PyPI upload is blocked by #384 (trusted-publisher re-registration). Homebrew tap updated below.

v0.8.3 — Stable Cascade (Würstchen)

Stable Cascade (Würstchen) native MLX image pipeline (#370)

Native MLX port of Stable Cascade — a 3-stage Würstchen pipeline
(prior → decoder → VQGAN), from scratch under fusion_mlx/image/cascade/.
Does not wrap mflux.

Highlights

• Unified StableCascadeUNet serves both the prior
  (switch_level=(False,) → UpDownBlock2d 1×1 mapping, no spatial
  change) and the decoder (switch_level=None → Conv2d(k=2,s=2) down
  / ConvTranspose2d(k=2,s=2) up).
• PaellaVQModel decode-only VQGAN (scale_factor=0.3764).
• CLIP-ViT-bigG text encoder (dims=1280, 32 layers, 20 heads).
• DDPMWuerstchenScheduler (cosine _alpha_cumprod).
• NHWC throughout (MLX conv2d-native).

Weight-layout fix (key gotcha)

• Conv2d/DepthwiseConv2d (out,in,k,k) → OHWI (0,2,3,1)
• ConvTranspose2d (in,out,k,k) → OHWI (1,2,3,0) — different transpose
• nn.Linear / nn.Embedding left as-is (MLX stores (out,in) like
  PyTorch; transposing 2D weights on load is a bug)

Validation

• Key alignment vs real stabilityai weights: prior 1550/1550,
  decoder 1726/1726, VQGAN 121/122, CLIP 517/517.
• 24 unit tests green; real-weights E2E (512×512 / 768×768 PNG, CFG);
  server-path E2E via ImageGenEngine.
• ruff + black clean across the repo.

Wiring

• image_gen engine stable_cascade variant (auto-detected from
  cascade/wuerstchen model names) → /v1/images/generate.
• model_discovery maps StableCascadePriorPipeline /
  StableCascadeDecoderPipeline → text-to-image.
• docs: docs/cascade-image.md

Closes #370.

PyPI publish is blocked by trusted-publisher re-registration
(issue #384); Homebrew tap updated.

v0.8.2 - SDXL image generation

Patch release

• Add SDXL image generation: native MLX port of StableDiffusionXLPipeline with CosXL (cosxl_edit) and SDXS variant support (#371, PR #383). Dual text encoders (CLIP-L + OpenCLIP-G, cross-attn dim 2048), EulerDiscreteScheduler, AutoencoderKL. Wired into the image_gen engine (sdxl/cosxl/sdxs variants) and the /v1/images/generate API. Validated end-to-end with real stabilityai/stable-diffusion-xl-base-1.0 weights.

Full changelog: v0.8.1...v0.8.2

v0.8.1

Patch release

• #381 — Fix background fine-tune jobs crashing with BrokenPipeError when tqdm flushes the closed stderr pipe of the background service. The train() call is now wrapped in redirect_stderr(io.StringIO()) so tqdm writes to an in-memory buffer instead of the dead pipe.
• #380 — Fix 7 pre-existing full-suite test failures that blocked CI green on main: url_safety shadowing in the security test fixtures and a vlm video load_video path mismatch. No runtime behavior change. Unit suite 7937 pass / 0 fail locally; CI test (3.11) drops from a 2h+ hang to ~15m.

See CHANGELOG.md for full details.

v0.8.0

Stable Diffusion 3-Medium full MLX txt2img (#369)

From-scratch MLX port of SD3-Medium: MMDiT (24 joint transformer blocks) + AutoencoderKL VAE + FlowMatchEuler rectified-flow scheduler, in fusion_mlx/image/sd3/. Reuses mflux T5-XXL + CLIP-L, custom parametrized CLIP-G (20 heads). Wired into /v1/images/generate with model="sd3-medium" — supports negative_prompt and shift (unlike Flux). fp8 T5 (t5xxl_fp8_e4m3fn) preferred over sharded fp16 with auto-fallback. SD3_LOCAL_DIR for offline weight resolution. Real-model E2E validated. 28 unit tests.

See docs/sd3-image.md for usage, weights, and gotchas.

v0.7.11

Video DiT diffusion throughput (#367)

HunyuanVideo and Cosmos diffusion loops ran two full-latent DiT forwards per step (uncond + cond, standard CFG). At production frame counts (57-121 frames) every video DiT workflow (2B/7B/13B) exceeded the 1800s budget before reaching VAEDecode.

Performance

• CFG batched guidance: fuse uncond+cond into a single batched B=2 DiT forward (~2x throughput, no quality change). Both HunyuanVideo and Cosmos DiTs are batch-safe along dim 0.
• Single-forward shortcut: cfg_scale <= 1.0 skips the uncond branch (cond-only) for guidance-distilled / low-cfg workflows.
• Step-level it/s INFO logging: replaces debug-only line; reports avg dt + it/s, makes hangs vs. slow steps diagnosable, lets ComfyUI report progress.
• Wan2/VACE already used batched CFG — unchanged.

Verified on real models

• HunyuanVideo 13B DiT + Cosmos 7B DiT, 3-step diffusion: batched vs. two-pass converge, no NaN, relative latent diff <1% (Hunyuan 0.86%, Cosmos 0.25%).
• 122 unit/integration tests pass; ruff + black clean.

Fixed

• test_gen_acceleration_knobs.py::TestImageGenKnobFlow: 3 pre-existing failures resolved — mocks updated for the FLUX.1 path (mflux.models.flux.cli.flux_generate.Flux1, ModelConfig.schnell()/dev()) from #368/#375.

Full changelog: v0.7.10...v0.7.11

v0.7.10

Fixed

• #373 - Dead guided-decoding branch on /v1/responses. Removed the use_guided / engine.generate_with_schema / engine.supports_guided_generation references from fusion_mlx/routes_internal/responses.py (_resolve_strict_context and the _non_stream dispatch). No engine class defines those symbols — the branch was dead code and a latent AttributeError. Strict json_schema requests on /v1/responses now go straight to the R12-4 post-generate validation path (the only live constrained path on this buffered-only surface). Live constrained decoding for the chat surface runs through the grammar-compiler (xgrammar/llguidance) path in openai_routes.py, unchanged.

Removed

• tests/unit/test_responses_chat_template_kwargs.py: deleted the TestBatchedEngineGuidedHonorsEnableThinking class (3 dead-contract tests) and test_strict_via_guided_path_also_auto_disables; simplified the _Engine mock. 15 tests remain green.

Full changelog: https://github.com/dahai80/fusion-mlx/blob/main/CHANGELOG.md

v0.7.9 — fix(#364) Qwen3 thinking models empty content in non-stream mode

Fixed

#364 — Non-stream content: null on Qwen3 thinking models

Non-stream /v1/chat/completions and /v1/responses returned content: null (with completion_tokens > 0) for Qwen3 thinking models. Streaming worked; the bug was non-stream path only.

1. scheduler/spec_decode.py — the EAGLE3 draft-model speculative-decode path (spec_decode_step) finished the request without setting out.output_text on the final RequestOutput. Non-streaming reads output_text off the merged final output (engines/batched.py → clean_special_tokens), so content came back empty while tokens were still billed. Streaming (per-token new_text) was unaffected. Mirrored the output_text decode already present in the sibling ngram_spec_step and dflash_spec_step finish paths.

2. routes_internal/responses.py — enable_thinking was set as a top-level chat kwarg, but engine.chat only forwards chat_template_kwargs to the chat-template render, so the Qwen3 template ran thinking-on by default and a max_tokens-truncated response lost the visible answer. Routed enable_thinking through chat_template_kwargs and applied the shared disable-by-default (resolve_enable_thinking_default) on both stream and non-stream paths, matching /v1/chat/completions.

Test debt

• test_responses_chat_template_kwargs.py: corrected 3 stale mock targets (vllm_mlx.service.helpers.get_config → fusion_mlx.config.get_config) after the vllm_mlx→fusion_mlx rename; skipped 3 TestBatchedEngineGuidedHonorsEnableThinking tests pinning a removed guided-decoding contract (tracked in #373).
• test_stub_modules.py: removed broken TestParentWatchdogStub — _parent_watchdog is now a real implementation; the old tests installed it with a bogus ppid + no on_orphan mock, so the default orphan callback SIGTERM-killed the test runner. Covered by test_parent_watchdog.py.

Related

• #373 filed for the guided-decoding architecture drift discovered during this fix (independent latent bug).

Full diff: #374