BaseRT engine 0.2.2

BaseRT 0.2.2

Muse Glimmer, and native GGUF k-quants. BaseRT now runs Muse Glimmer 30B —
text and vision — on Apple Silicon, and reads GGUF k-quantized weights
(Q4_K / Q5_K / Q6_K) directly, without a dequantize-and-repack step. Short
prompts prefill 1.4–1.55× faster on M5.

Added

Muse Glimmer

• Muse Glimmer 30B, text and vision, including the perception tower — so
  the model answers about images, not just text.

• Pre-converted bundles at
  basecompute/Muse-Glimmer-30B:

  basert pull basecompute/Muse-Glimmer-30B              # 21 GB, recommended
  basert pull basecompute/Muse-Glimmer-30B:q4k-17gb     # 18 GB, smaller

  Both are the model's own published k-quants, repackaged — the super-block
  bytes are copied through unchanged, so the weights are bit-identical to the
  upstream GGUFs rather than a re-quantization of them. Both carry the vision
  tower; there is no separate projector file to manage.

• Images use the model's own <|patch|> placeholder:

  basert complete basecompute/Muse-Glimmer-30B --chat \
    --image photo.png --prompt "<|patch|>Describe this image."

Native GGUF k-quants

• Q4_K / Q5_K / Q6_K run natively — prefill, decode and embedding
  lookup all read the quantized weights directly. Q2_K and Q3_K are
  supported as well.
• --kquant-passthrough copies GGUF super-blocks into a .base bundle
  verbatim, so the weights stay bit-identical to the source. Re-quantizing an
  already-quantized model compounds its error; copying the blocks through
  introduces none, so this is not a quality trade and needs no override flag.
• --mmproj folds a companion mmproj-*.gguf perception tower into the
  same bundle. Without it a bundle converted from GGUF is text-only, because
  the tower ships as a separate file. A bundle built this way is
  interchangeable with one converted from the original safetensors.

Changed

• Faster short-prompt prefill on M5 — 1.4–1.55× measured on M5 Max. Short
  prompts previously fell back to a slower path than the hardware supports;
  they now use the same accelerated one longer prompts already did. Long
  prefill is unchanged, because it was already there.
• Faster k-quant prefill — 3.4×, from running k-quantized weights on the
  accelerated path rather than the general one.
• Wider accelerated coverage for the 5-, 2- and 3-bit quantizations,
  alongside the 4-, 6- and 8-bit ones.
• Decode speed is unchanged. At these model shapes it is limited by memory
  bandwidth rather than compute, so the changes above do not move it, and the
  decode-side ideas tried for this release measured slower and were dropped.

Fixed

• M3 Ultra was identified as a base M3, so it was tuned for a far smaller
  chip than it is. Reachable only when the system does not report a GPU core
  count, but it would have affected every future per-chip decision.
• Language bindings could corrupt memory. The model config is returned by
  value, and the Python, Rust and Node definitions of that struct had fallen
  behind the C header — a mirror even four bytes short lets the engine write
  past the caller's allocation. All four bindings are back in step, and a test
  now holds them there.
• Models converted from GGUF ignored the end-of-turn token and kept
  generating past the end of a reply.
• baseRT_format_chat is available without a chat-capable build —
  it no longer depends on the chat layer being present.

Notes

• k-quant support is Apple Silicon (Metal) only in this release.
• Folding a perception tower from a companion mmproj GGUF is exact for still
  images, which is what these models take today. Such a file does not carry
  everything a video-frame path would need.