Skip to content

BaseRT engine 0.1.7

Choose a tag to compare

View all tags
@prabod prabod released this 21 Jul 03:10
· 42 commits to main since this release
v0.1.7
7a63702
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Maintenance release focused on the three most-reported serve/CLI issues, plus converter and chat quality-of-life fixes.

Fixed

  • Tool calls for Qwen 3.5 / 3.6 (#21) — basert serve now emits structured tool_calls for the Qwen XML tool-call dialect, in both streaming and non-streaming modes. Truncated generations never execute partial arguments, and logprobs stay aligned across tool-call segments.
  • Deterministic HTTP 500 on some short CJK prompts (#22) — fixed a byte-alphabet hole (soft hyphen 0xAD) in GPT-2-style byte tokenization; tokenization is now HF-exact for the affected models. Server errors are structured and logged instead of opaque 500s.
  • --no-think and chat-template handling (#23) — --no-think now works across all thinking-capable architectures; the CLI renders model-embedded chat templates; fixed a Gemma 4 channel-marker leak and KV-reuse correctness on divergent history.
  • MLX 3/5/6-bit conversion — MLX quantized checkpoints at 3/5/6-bit widths now unpack correctly (little-endian bitstream); previously these converted to wrong weights.
  • basert pull HF-cache duplication — pulled models are no longer duplicated in the global Hugging Face cache (staged download with resume preserved; set BASERT_KEEP_HF_SOURCES=1 to keep sources).

Added

  • Chat line editing — cursor movement, kill ops, and word delete (Ctrl+W / Alt+Backspace / Ctrl+Backspace) in basert chat, with UTF-8/CJK support.
  • q6 MoE kernels — Q6 expert GEMM (plus q8/q6 scale-dtype siblings) and a bundled default-q6 conversion profile (--target base-q6).
  • baseRT_decode_token_raw() — length-preserving raw-byte token decode in the C API (byte-level BPE tokens can legally contain NUL; the C-string variants truncate there). No struct-layout changes.

Changed

  • Chat / complete / bench banners show the model id (org/repo · variant) instead of model.base.
  • basert help output regrouped runtime-first.

Performance is unchanged vs 0.1.6 (validated dense + MoE on M4 Pro and M5 Pro, within noise).

Assets 4
Loading