basal 2.2

basal 2.2

• Calibrate any basal or basal-omni model on your own labelled data. Label a few hundred of your own requests,
  run basal-calibrate, and serve the file it writes: the probabilities, and the max_error thresholds that decide
  what may be automated, are then certified on your data instead of ours.
• basal-omni on llama.cpp, Ollama and Apple Silicon (text only), and new model repositories for these paths.
• Community contributions: GGUF export, a TPU engine, an AMD Strix Halo engine and
  basal-viz, from Piotr Durlej, Kuba Sołtys, Michał Kardyś and @lukaspoloki.

Install:

uv pip install "basal[media] @ https://github.com/rkinas/basal/archive/refs/tags/v2.2.0.tar.gz"

Calibrate on your own data

1. Write one /v1/systemone request per line, exactly as you send it in production, with the correct answer of
   each question in "gold":

   {"id": "t-17", "group": "customer-17", "state": "...",
    "questions": {
      "dept": {"type": "choice", "instructions": "...", "criteria": {"returns": "...", "it": "..."}, "gold": "returns"},
      "late": {"type": "noul", "instructions": "...", "gold": "yes"}}}

2. Fit it with the model, engine and option orders you serve, then serve the file:

   basal-calibrate my_labelled.jsonl --model Remek/basal-omni-1.0-max --mode eager    # writes CALIBRATION.custom.json
   basal-serve --model Remek/basal-omni-1.0-max --mode eager --calibration CALIBRATION.custom.json

A few dozen labels already move the temperatures. A few hundred usually certify the 5% target, and the 1% target
needs about 1,000 items or more. --report-only prints the report without writing a file, so you can evaluate a
model, or a calibration file, on your own data.

New in 2.2:

• Task thresholds from your labels. Labelled classify, verify and extract questions certify the automated
  flag of their answers (task_thresholds), by the same bound as the decision types. extract fields now report
  automated too.
• Small data. Below --min-per-type decisions, a type's temperature is blended with the shipped one in proportion
  to the labels you have (--min-blend, from 10), instead of staying as shipped.
• Measured as served. The report shows how the file you would otherwise serve (the shipped one, or
  --calibration FILE) does on your held-out decisions: the share it automates at each target and its error, each
  type at the threshold basal-serve applies.
• A file knows its model. fitted_on records the model and its Hugging Face revision. basal-serve --calibration
  warns when a file was fitted on another model or revision, and --strict-calibration refuses it.
• Fix: a looser max_error target no longer keeps a stricter threshold than a tighter one (also for the task
  thresholds).

Measured on a domain the model never saw (basal-omni-1.0-mini on
typed-decisions, test split): the shipped file's 1%
setting would automate 12.6% of decisions at 8.0% error. Calibrated on the train split, the calibration error falls
from 0.268 / 0.106 / 0.104 to 0.083 / 0.073 / 0.052 (choice / yes-no / score) and nothing is automated above its
target; on verify, the 1% threshold fitted on the train split automates 54.8% of the test claims with none wrong
(one-sided 95% bound on the error: 1.1%).
Details: docs/CALIBRATION.md.

basal-omni on llama.cpp, Ollama and Apple Silicon (text only)

• --mode gguf --gguf <file> runs a GGUF conversion of basal-omni-1.0 through llama.cpp in process (Metal, CUDA or
  CPU; basal[gguf], llama-cpp-python 0.3.36 or newer). --mode mlx / mlx-q8 runs the bf16 weights through mlx-lm
  on Apple Silicon.
• Against the eager bf16 engine on all 10,606 development decisions, Q8_0 and Q6_K pass the port gate; Q5_K_M and
  Q4_K_M do not.
• --mode gguf loads CALIBRATION.gguf.json and --mode mlx CALIBRATION.mlx.json when the model ships one.
• New on Hugging Face: basal-omni-1.0-mini-GGUF and
  -max-GGUF (Q8_0 and Q6_K, Modelfiles for Ollama, a calibration
  fitted on Q8_0), and basal-omni-1.0-mini-FP8 and
  -max-FP8 (12 GB and 14 GB, --mode vllm, their own
  CALIBRATION.vllm.json).
• MLX 8-bit ports for Apple Silicon (--mode mlx, text only):
  • basal-omni-1.0-mini-MLX (8.6 GB): passes the port gate
    against bf16 (agreement 0.986 on 2-10 options, 0.999 where bf16 is sure, accuracy +0.3 pt, ECE unchanged); 0.73
    decisions/s on an M4 Pro 24 GB (8 in flight), 173 ms for one question; own calibration CALIBRATION.mlx.json,
    34% / 74% automated at 1% / 5% target error.
  • basal-omni-1.0-max-MLX (13.6 GB): passes the port gate
    against bf16 (agreement 0.989 on 2-10 options, 1.000 where bf16 is sure, accuracy -0.1 pt, ECE +0.002); 0.25
    decisions/s on an M4 Pro 24 GB (8 in flight; long prompts can run out of memory at 24 GB), 453 ms for one question;
    own calibration CALIBRATION.mlx.json, 36% / 85% automated at 1% / 5% target error.
  • basal-omni-1.1-max-MLX (13.6 GB): passes the port gate
    against bf16 (agreement 0.984 on 2-10 options, 0.999 where bf16 is sure, accuracy +0.6 pt, ECE -0.003); 0.27
    decisions/s on an M4 Pro 24 GB (8 in flight; long prompts can run out of memory at 24 GB), 525 ms for one question;
    own calibration CALIBRATION.mlx.json, 35% / 80% automated at 1% / 5% target error.
• VLLM_BATCH_INVARIANT=1 makes --mode vllm answers repeatable: on basal-omni-1.0-max-FP8, two runs gave the same top
  answer on all 2,000 decisions (94.7% without it), at about 23% fewer decisions per second.

Gates and speeds: docs/OMNI.md.

Community contributions

These pull requests from the community ship in 2.2. Thank you!

• basal-export-gguf by Piotr Durlej (@pdurlej, #8): converts a basal
  model to GGUF, or repairs an existing GGUF file, so that llama.cpp and Ollama tokenize exactly like the Hugging Face
  tokenizer, as the published -GGUF files do (extra basal[gguf-export]).
• --mode tpu by Kuba Sołtys (@qooba, #15), experimental,
  community-maintained: basal-1.5 / basal-2.0 in JAX / XLA on a Google TPU (extra basal[tpu]), with a Colab notebook.
• --mode strix by Michał Kardyś (@kardysm, #14), experimental,
  community-maintained: a basal-1.5 / basal-2.0 forward with Triton kernels for AMD Strix Halo (gfx1151).
• basal-viz by @lukaspoloki (#2): a local page that shows every step of
  a decision, from the prompt to the per-layer readout, the calibration and the answer basal-serve returns, for the
  Llama-architecture models (basal-1.0, basal-1.5, basal-2.0).

The speed and agreement numbers in the TPU and Strix Halo docs are the contributors' own measurements.

Contributors to this release: Piotr Durlej (@pdurlej), Kuba Sołtys (@qooba), Michał Kardyś (@kardysm),
@lukaspoloki. Thank you!

Thanks also to Paweł Kiszczak (@pawelkiszczak) for #4 (Apple Silicon and
llama.cpp / Ollama engines, an earlier release).

Compatibility with 2.1

• Same API and the same models. Every other request takes the same code path as in v2.1.0.
• Calibration files load as before. A file written by basal-calibrate 2.1 records its model but not the revision,
  so basal-serve --calibration checks the model name only and warns if it differs. The files that ship with a model
  are not checked.
• A new basal-calibrate run can give different files than 2.1 on the same data: small types are blended
  (--min-blend 0 keeps the 2.1 behaviour) and the threshold fix above applies.

Upgrade: run the install line again with the extras you use (fp8, media, vllm, gguf, mlx, mcp), e.g.

uv pip install "basal[fp8] @ https://github.com/rkinas/basal/archive/refs/tags/v2.2.0.tar.gz"

For --mode gguf on NVIDIA, build llama-cpp-python with CUDA (docs/GGUF.md).

Full notes: docs/WHATS_NEW.md.