Skip to content

v0.7.3 — DiffusionGemma tool calls + revert in-house loop

Choose a tag to compare

View all tags
@raullenchai raullenchai released this 12 Jun 05:06
v0.7.3
41a0d89
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What changed

Revert PR #555 (in-house DiffusionGemma generation loop, shipped in v0.7.2). The fixed_steps=8 default cap caused quality regressions on long Chinese / multi-paragraph output. Performance was never the bottleneck — direct mlx-vlm passthrough already matches what the in-house loop achieved (49.5 tok/s rapid-mlx HTTP vs 53.0 tok/s mlx-vlm direct on M3 Ultra, 256-token sweep, well within HTTP wrapper tax).

Fix tool calling on the mlx-vlm passthrough path so DiffusionGemma's call:NAME{...} wire form surfaces as structured tool_calls, not leaked delta.content:

  • Gemma4ToolParser regex now accepts the post-HF-decode stripped form (no outer <|tool_call>/<tool_call|> wrappers). HF's tokenizer.decode(skip_special_tokens=True) strips those marker ids regardless of the skip_special_token_ids carve-out, so the body call:NAME{...} is what reaches the parser in production.
  • StreamingPostProcessor._detect_tool_calls no longer short-circuits the parser when the delta has no < or [ char — it now also queries the active parser's has_pending_tool_call so the gemma4 stripped opener call:\w+\{ is recognised.
  • Real-tokenizer end-to-end test added under TestStreamingPostProcessorGemma4StrippedForm so this class of bug can't ship silently again.

Alias naming — diffusion-gemma-26b split into diffusion-gemma-26b-4bit and diffusion-gemma-26b-8bit per the project <family>-<size>-<quant> SOP. The bare alias is removed.

v0.7.2 status

v0.7.2 will be yanked from PyPI and its release deleted. Users on 0.7.2 should upgrade to 0.7.3 — quality regressions from fixed_steps=8 will not be patched in-place.

Verification

  • rapid-mlx share diffusion-gemma-26b-4bit end-to-end: SSE response ships only structured tool_calls, no leaked content
  • Big-AGI UI confirms Weather + Web Search tools render as proper tool boxes (not raw call:NAME{...} text)
  • 26 parser unit tests + 3 real-tokenizer postprocessor tests + 47 escape-hatch tests all pass

Release SOP gates (all green)

G1 release-smoke · G3 cli/config audit · G4 make smoke · G8 parser microbench · G10 mlx-version-bump scan (skip — no mlx-* deps changed) · G11 escape-hatch tests · G6 live server repro

🤖 Generated with Claude Code

Assets 2
Loading