Repository navigation
basal 2.8.0
basal 2.8.0 (8 November 2026). Bug fixes and the groundwork for 3.0, with contributions from Dariusz Koryto, Rafal Wiktorowicz
(NovaraX), Miłosz Kucharski and Marcin Brzeżański. Answers to valid requests are byte-identical to 2.6.0, except where a change
below says otherwise. This was checked on an H100 and on an RTX PRO 6000 Blackwell:
- basal-omni-1.0-mini in
--mode fast,--mode eagerand with--media on; - basal-omni-1.0-mini-FP8, basal-omni-pico and basal-2.0-mini-FP8 in
--mode fast; - basal-2.0-4.5B and basal-omni-1.0-mini with
--state-cache-bits 8.
Inside, the server is rebuilt the way 3.0 needs it: one request path, admission and bounds, and one calibration core.
Defaults that change:
- The long lane is on by default (
--long-lane auto). While other requests are in flight, a long request (over 2,048
tokens) takes a lane of its own, so short requests no longer wait behind long documents. On an RTX PRO 6000 Blackwell
with 60k-token states every 3 s, short p50 went from 3.9 s to 0.17 s. A request alone on the server is still answered by
the main lane, byte for byte. Under a steady stream of long states, long requests take longer;--long-lane offkeeps
the old behaviour. The measurements and the case came from Dariusz Koryto
(#130).GET /healthand/metricsnow show the long lane apart. - basal-omni-pico on an RTX PRO 6000 Blackwell (SM120) runs
--mode fastwhen no--modeis given
(#114, Marcin Brzeżański).
Calibration:
- Your own calibration survives a model-card update. A
--calibrationfile is now matched by the model's files, not by
the commit, so a card-only commit no longer drops its thresholds.basal-calibrate --urlrecords the revision and
files the server actually serves, andGET /v1/modelsreports them. - One calibration core in
basal-calibrate, with three fixes. A compared file of another model keeps only its
temperatures. The compared file is read as basal-serve reads it, and a file of another prompt format is refused. basal-calibrate --keep-shipped TYPEkeeps a type's shipped temperature and thresholds when your labels cannot
certify it. A kept threshold your held-out data refutes is withdrawn
(#113, Dariusz Koryto).- basal-serve checks a calibration file's thresholds at start-up, as it checked its temperatures.
basal-run:
- Resumes an interrupted run from
<output>.part. - Stops when the server stops answering, keeping
.part, so rerunning the same command goes on from there.
Hardening, from a second bug hunt (about 25,000 fuzz cases):
- Request size: requests are bounded before any work (
--max-body-mb,--max-branches,--max-prompt-mb). A
ranklist with"chunk": 0whose one shared context would exceed--max-prompt-mbis read in chunks of 50, as a
list longer than the context is, sobasal-navigate's defaults work onbasal-serve's defaults. - Linear-time checks:
extractschemas and named options are checked in linear time, where some large requests used to
freeze the server.act's short-form costs no longer build a cost matrix. - basal-router:
--replica-timeoutno longer takes a healthy replica out;- a failed re-ask of a gated question fails the request;
GET /health/answers.
- Other server fixes:
- a rank request whose part fails stops the others;
--ui fullkeeps its memory budget;- a large body is checked off the event loop;
--max-requestskeeps an admitted request's slot;- HEAD
/healthstays open under the host check.
- The decision log tells requests with other media or another language apart, and stays strict JSON
(#119, Rafal Wiktorowicz).
Speed (Miłosz Kucharski):
- Less host work per request (#100).
--state-cache-bits 8quantises the layers of one shape together: a short state's store and read are 4–13× faster,
with answers unchanged (#110).
Prepared kernels: a start that loads a prepared/ directory now reads and writes only its own compile cache, so a
builder's path on a shared machine can no longer fail it. Every shipped prepared/ directory is rebuilt for 2.8.0.
Docs: the 2.5.0 notes announced four items for 2.5.1 that were not delivered. They are corrected.
Known issue (added 8 Nov): with several clients at once, --long-lane auto (the default for basal-omni in --mode fast) sends every request over about 2,000 tokens to one serial lane, so 8 clients get about half the throughput of one (basal-omni-1.0-mini on an H100: 20 instead of 37 decisions/s). If your clients send many such requests at once and throughput matters, start the server with --long-lane off: 37 decisions/s at 8 clients. Short requests then wait behind long ones (median 0.02 s → 1.7 s at 8 clients), so keep the default where short requests next to long documents must stay fast. basal 3.0 fixes this.
Thank you, Dariusz, Rafal, Miłosz and Marcin! All changes:
WHATS_NEW.
Install: uv pip install "basal[fp8] @ https://github.com/rkinas/basal/archive/refs/tags/v2.8.0.tar.gz"