Skip to content

v0.1.7 - Qwen3.6 Benchmark & Hermes Agent

Choose a tag to compare

View all tags
@jaigouk jaigouk released this 24 Apr 09:39
· 100 commits to main since this release
v0.1.7
4a8a62f

New Features

  • Qwen3.6 vs Gemma4 architecture benchmark — 4 models compared across 15 iterations each:
    • Qwen3.6-35B-A3B (UD-Q4_K_S): 90.0 mean, 0.0 std dev, 173.7 TPS
    • Qwen3.6-35B-A3B (UD-IQ4_XS): 87.3 mean, 174.5 TPS
    • Gemma 4 E4B (BF16): 88.3 mean, 82.9 TPS
    • Qwen3.6-27B (Q4_K_M): 80.3 mean, 46.9 TPS
  • Hermes Agent mode switched to Qwen3.6-35B-A3B IQ4_XS — 2x faster than Gemma4, fits with embedding model on 24GB VRAM
  • Qwen3.6 presets: qwen36-27b-q4, qwen36-35b-a3b-q4, qwen36-35b-a3b-iq4xs

Fixes

  • ConfigFetcher hardening: follow HuggingFace 307 redirects, map httpx errors to ConfigFetchError, SSRF redirect hostname allow-list
  • Benchmark TPS measurement: read timing from response body instead of deprecated X-Llama-Timings header
  • CI test fixes for hermes-agent mode count

Key Finding

MoE (3B active) is 3.7x faster than dense 27B with higher quality. IQ4_XS is a viable swap for Q4_K_S — statistically tied on quality, identical TPS, 1 GB VRAM savings.

Full Changelog

v0.1.6...v0.1.7

Assets 2
Loading