Skip to content

basal 2.8.0

Choose a tag to compare

@rkinas rkinas released this 07 Nov 23:14
· 6 commits to main since this release

basal 2.8.0 (8 November 2026). Bug fixes and the groundwork for 3.0, with contributions from Dariusz Koryto, Rafal Wiktorowicz
(NovaraX), Miłosz Kucharski and Marcin Brzeżański. Answers to valid requests are byte-identical to 2.6.0, except where a change
below says otherwise. This was checked on an H100 and on an RTX PRO 6000 Blackwell:

  • basal-omni-1.0-mini in --mode fast, --mode eager and with --media on;
  • basal-omni-1.0-mini-FP8, basal-omni-pico and basal-2.0-mini-FP8 in --mode fast;
  • basal-2.0-4.5B and basal-omni-1.0-mini with --state-cache-bits 8.

Inside, the server is rebuilt the way 3.0 needs it: one request path, admission and bounds, and one calibration core.

Defaults that change:

  • The long lane is on by default (--long-lane auto). While other requests are in flight, a long request (over 2,048
    tokens) takes a lane of its own, so short requests no longer wait behind long documents. On an RTX PRO 6000 Blackwell
    with 60k-token states every 3 s, short p50 went from 3.9 s to 0.17 s. A request alone on the server is still answered by
    the main lane, byte for byte. Under a steady stream of long states, long requests take longer; --long-lane off keeps
    the old behaviour. The measurements and the case came from Dariusz Koryto
    (#130). GET /health and /metrics now show the long lane apart.
  • basal-omni-pico on an RTX PRO 6000 Blackwell (SM120) runs --mode fast when no --mode is given
    (#114, Marcin Brzeżański).

Calibration:

  • Your own calibration survives a model-card update. A --calibration file is now matched by the model's files, not by
    the commit, so a card-only commit no longer drops its thresholds. basal-calibrate --url records the revision and
    files the server actually serves, and GET /v1/models reports them.
  • One calibration core in basal-calibrate, with three fixes. A compared file of another model keeps only its
    temperatures. The compared file is read as basal-serve reads it, and a file of another prompt format is refused.
  • basal-calibrate --keep-shipped TYPE keeps a type's shipped temperature and thresholds when your labels cannot
    certify it. A kept threshold your held-out data refutes is withdrawn
    (#113, Dariusz Koryto).
  • basal-serve checks a calibration file's thresholds at start-up, as it checked its temperatures.

basal-run:

  • Resumes an interrupted run from <output>.part.
  • Stops when the server stops answering, keeping .part, so rerunning the same command goes on from there.

Hardening, from a second bug hunt (about 25,000 fuzz cases):

  • Request size: requests are bounded before any work (--max-body-mb, --max-branches, --max-prompt-mb). A
    rank list with "chunk": 0 whose one shared context would exceed --max-prompt-mb is read in chunks of 50, as a
    list longer than the context is, so basal-navigate's defaults work on basal-serve's defaults.
  • Linear-time checks: extract schemas and named options are checked in linear time, where some large requests used to
    freeze the server. act's short-form costs no longer build a cost matrix.
  • basal-router:
    • --replica-timeout no longer takes a healthy replica out;
    • a failed re-ask of a gated question fails the request;
    • GET /health/ answers.
  • Other server fixes:
    • a rank request whose part fails stops the others;
    • --ui full keeps its memory budget;
    • a large body is checked off the event loop;
    • --max-requests keeps an admitted request's slot;
    • HEAD /health stays open under the host check.
  • The decision log tells requests with other media or another language apart, and stays strict JSON
    (#119, Rafal Wiktorowicz).

Speed (Miłosz Kucharski):

  • Less host work per request (#100).
  • --state-cache-bits 8 quantises the layers of one shape together: a short state's store and read are 4–13× faster,
    with answers unchanged (#110).

Prepared kernels: a start that loads a prepared/ directory now reads and writes only its own compile cache, so a
builder's path on a shared machine can no longer fail it. Every shipped prepared/ directory is rebuilt for 2.8.0.

Docs: the 2.5.0 notes announced four items for 2.5.1 that were not delivered. They are corrected.

Known issue (added 8 Nov): with several clients at once, --long-lane auto (the default for basal-omni in --mode fast) sends every request over about 2,000 tokens to one serial lane, so 8 clients get about half the throughput of one (basal-omni-1.0-mini on an H100: 20 instead of 37 decisions/s). If your clients send many such requests at once and throughput matters, start the server with --long-lane off: 37 decisions/s at 8 clients. Short requests then wait behind long ones (median 0.02 s → 1.7 s at 8 clients), so keep the default where short requests next to long documents must stay fast. basal 3.0 fixes this.

Thank you, Dariusz, Rafal, Miłosz and Marcin! All changes:
WHATS_NEW.

Install: uv pip install "basal[fp8] @ https://github.com/rkinas/basal/archive/refs/tags/v2.8.0.tar.gz"