BaseRT engine 0.2.1

BaseRT 0.2.1

Speech-to-text. BaseRT now runs the full OpenAI Whisper family — all twelve
variants from tiny to large-v3-turbo, including the English-only builds — as
.base bundles on both Apple Silicon (Metal) and NVIDIA GB10 (CUDA), served
through an OpenAI-compatible transcription API with live streaming.

Added

Whisper speech-to-text

• All twelve Whisper variants convert to .base and run end-to-end:
  tiny, base, small, medium, large, large-v2, large-v3,
  large-v3-turbo, and the .en English-only builds. Special-token IDs are
  read from each model's own tokenizer rather than inferred from vocabulary
  size, so large-v3 and large-v3-turbo (100 languages, 128 mel bins) map
  correctly.
• Quantized bundles — whisper-q8 and whisper-q4 profiles quantize the
  linear projections while the convolutional front end, positional and token
  embeddings, norms, and biases stay f16. Q8 meets the same word-error-rate
  parity gates as f16 against the reference implementation.
• CUDA support — Whisper runs on NVIDIA GB10 with transcripts byte-identical
  to the Metal path.
• POST /v1/audio/transcriptions and POST /v1/audio/translations,
  OpenAI-compatible, including verbose_json with per-segment
  avg_logprob, no_speech_prob, compression_ratio and temperature, plus
  the detected language.
• Streaming transcription — stream: true emits segments from the decode
  loop as each window completes, in the documented transcript.text.delta /
  transcript.text.done wire format, so official OpenAI SDKs parse it directly.
  Time-to-first-event is one window's decode rather than the whole file.
• Decode parity with the reference implementation — task=translate,
  language auto-detection (language: "auto"), initial_prompt with
  condition-on-previous-text, and the temperature-fallback ladder.
  Word-level timestamps remain future work.

C API

• baseRT_load_model_ex with BaseRTLoadOptions — per-load KV width, paged-KV,
  batch size, prefix cache, prefill chunk and paged-weight mode, replacing the
  process-wide setters. The options struct is ABI-guarded by struct_size.
• Transcription accessors: baseRT_set_task, baseRT_set_initial_prompt,
  baseRT_set_condition_on_previous_text, per-segment results, detected
  language, and source-audio duration.
• Batched logits: baseRT_batch_logits_stride, baseRT_argmax_logits_row,
  baseRT_sample_logits_row, baseRT_logits_row_logprobs,
  baseRT_mask_logits_row.
• baseRT_capabilities reports what the running backend supports.

All of the above are bound in the Python, Node, Swift and Rust bindings.

Changed

• Faster shared-prefix serving — concurrent requests that share a prompt
  prefix now detect and reuse it through a cascade dispatch path.
• Faster MoE decode on CUDA — fused SiLU gate/up for 4-bit experts.
• Metal GEMM accuracy — f32 accumulation on the paths where f16
  accumulation was measurably lossy.

Fixed

• context_length_exceeded reported a capped token count. The number came
  from a fixed tokenizer buffer, so any over-long prompt reported the same
  figure no matter how far over it was — a client could not trim to a number
  that was itself the ceiling. It now reports the real count.
• Image requests under-reported usage and never returned
  finish_reason: "length": the first token produced during image prefill was
  streamed to the client but not counted, so a truncated answer looked complete.
• Gemma 4 audio produced unusable embeddings. The channel-last convolution
  indexed its weights as [out, kh, kw, in] where the bundle stores
  [out, in, kh, kw]. For a single input channel the two are identical, so the
  first subsampling convolution was correct and masked the fault while the
  second convolved with transposed weights.
• Repetition penalty silently did nothing on some Apple GPUs — a sampling
  kernel read a scratch buffer that was never bound, which is undefined
  behaviour and therefore hardware-dependent.
• A KV-cache shift kernel had a data race — an in-place shift whose source
  and destination overlap, spread across threads, so results depended on
  scheduling order.
• Rate limiting could grow without bound. Idle buckets expired on time
  alone, so a source rotating addresses added an entry per request and nothing
  became evictable within the window. Bucket count is now capped.
• Absolute --media-dir paths are honoured as written and must resolve
  inside the configured root, instead of being re-anchored under it — which
  made a legitimate absolute path fail as an image-decoding error.
• Rejections now appear in the access log, so an operator can tell a
  refused request from a hung one.
• Second images and audio parts in a chat request are rejected with a clear
  400 instead of a 500 or, for audio, a confident answer about audio the model
  never received.
• Image-decoding failures no longer echo the server's resolved filesystem path
  back to the caller.
• basert inspect reported 0 tensors for working vision and audio towers: it
  counted name prefixes that converted bundles do not use.
• Chat-template cache eviction and a Qwen3 tool round-trip fault.

Notes

• BASERT_VERSION now advertises the patch component, and the release pipeline
  verifies the tag, the compiled-in version, and every package manifest agree
  before publishing.