Skip to content

v0.2.0 — SGLang engine support + tuning pipeline localization

Pre-release
Pre-release

Choose a tag to compare

@htobty htobty released this 02 Oct 06:39
· 6 commits to main since this release

New

  • SGLang engine support — deploy and monitor SGLang alongside llama.cpp and vLLM, with Prometheus metrics collection and in-tool installation.

Improvements

  • Tuning pipeline fully localized — tuning logs and user-facing messages now follow the interface language (backend message registry, 85 call sites). English UI users no longer get Chinese analysis text: AI tuning now instructs the model to write its reasoning in English, while the Chinese prompt body stays untouched.
  • AI tuning is the first and default tab on the Tune page, and every backend request carries the current UI language.

Fixes

  • Windows GPU detection — cards that expose no standard name (e.g. AMD) are now recognized via WMI, registry qwMemorySize and GPU performance counters (#1).
  • Speculative decoding tuning — --spec-draft-p-min joins the continuous grid (0.0 / 0.5 / 0.8), with float parameter parsing fixed (#2).
  • Benchmark accuracy — decode throughput was measured over a window that hit EOS after ~24 tokens, reading 30–45% below the real rate; the benchmark now uses ignore_eos, a 512-token window and 2 warmup runs.

Full Changelog: v0.1.1...v0.2.0