Skip to content

basal 2.1

Choose a tag to compare

@rkinas rkinas released this 16 Oct 08:43
· 738 commits to main since this release

basal 2.1

basal 2.1 adds four task types (experimental) to the engine, on the same API and for the models you already
serve: classify, verify, compress and extract. The models still do not write text: each task is
expanded into typed decisions over one shared state, so every answer comes with calibrated probabilities, and an
extract value is always exact text of the document, with character offsets.

Install:

uv pip install "basal[media] @ https://github.com/rkinas/basal/archive/refs/tags/v2.1.0.tar.gz"

Task types

task what you send what you get
classify labels with a rubric each the label, a calibrated probability per label, whether it may be automated
verify statements about the state per statement: supported / contradicted / not stated, with probabilities
compress a question (optional) the sentences, lines or paragraphs needed for it, as shorter text
extract a JSON schema the values, copied exactly from the state with character offsets, or chosen from an enum

Measured with basal-omni-1.0-max (eager, bf16, one H100) on sets it was not trained on: CLINC150 85.4%, BANKING77
74.6%, VitaminC 80.3%, RAGTruth 80.3% (accuracy), QASPER 90.1% of the needed paragraphs kept at 30% of the text,
synthetic invoices 90.1% of fields correct. Details, request and answer formats: docs/TASKS.md;
every question type on one page: docs/QUESTION_TYPES.md.

  • automated is reported only with thresholds certified for the task (task_thresholds in the calibration file):
    the decision-type thresholds did not hold on new label sets.
  • Experimental: the task API may change in a minor release.

Everything else is unchanged

Every request without a task takes the same code path as in v2.0.0. On basal-omni-1.0-max, v2.1.0 and v2.0.0 gave the
same answer on 23,276 of 23,278 served-check rows (the two others were near-ties).

New models (Hugging Face, Sat 17 Oct, 09:00 CEST)

  • basal-omni-1.0-ultra (31B, Gemma 4 31B; text and images): Werdykt 0.879, the highest of the basal models; with
    basal-omni-1.0-ultra-FP8 for --mode vllm (text only).
  • basal-omni-1.1-max (experimental): basal-omni-1.0-max with extra training on the task formats. Tasks mean 85.1
    vs 83.5; slightly behind 1.0-max on general decisions (Werdykt 0.821 vs 0.823), so 1.0-max stays the default.

Full notes: docs/WHATS_NEW.md.