BaseRT engine 0.2.4

BaseRT 0.2.4

Three new model families, and a context window that sizes itself. gpt-oss,
Nemotron 3 Nano and GLM 5.2 all run now, with continuous batching rather than
one request at a time. And basert serve no longer starts every model at 4096
tokens — it measures the machine and gives the model the window the hardware
can hold, up to whatever it was trained for.

Added

The context window sizes itself

--max-context defaulted to 4096, which threw away most of a 32k model on a
workstation and overran the budget on a laptop when raised by hand. It is now
derived from the GPU memory available, the weights, and what a token of KV
cache costs — capped only by the model's trained window.

Context window: 131072 tokens (auto-sized for 96 GB of GPU memory at 8 lanes).
Pass --max-context N to pin a different one.

--max-tokens follows: unset, a request may fill the rest of the window
instead of stopping at 2048. basert chat and basert complete size
themselves the same way.

On a Mac sharing its GPU with another program, pin --max-context — the
measurement can't see another process's allocation.

gpt-oss

basecompute/gpt-oss-20b in bf16, Q8 and Q4. The MXFP4 experts are carried
across verbatim, never requantized. Harmony tool calls are parsed and emitted
as tool calls.

Nemotron 3 Nano

basecompute/NVIDIA-Nemotron-3-Nano-30B-A3B in bf16, Q8 and Q4 — a Mamba-2
hybrid, where most layers carry recurrent state instead of a KV cache.

GLM 5.2

Multi-head latent attention, with the sparse-indexer path for long contexts.
Continuous batching for it is Metal-only in this release; on CUDA it runs one
sequence at a time.

Fixed

• gpt-oss returned NaN on roughly one prompt position in twenty.
• Large models could emit a stray end-of-turn token before answering, on Metal
  under continuous batching.
• A loaded LoRA adapter ignored repetition, presence and frequency penalties,
  logit_bias, and seed.
• gpt-oss could keep generating past a finished tool call.
• A failed conversion left a model-sized temporary file behind.
• Image requests overreported prompt token usage when prompt caching was on.
• --validate could not pass on a current MLX checkpoint.

Changed

• BaseRTModelConfig changed layout — recompile anything built against
  0.2.3 that reads it.
• The shared library reports its real version again, and the macOS archive
  now ships the symlink its install name needs. Linking against the library
  from 0.2.1–0.2.3 failed to load; the bundled tools were unaffected.