basal 2.0

basal 2.0

basal-2.0 is a family of three typed-decision models for Polish and English, with engine v2.0.0: up to 255 options per
question, one forward pass with one option order, a faster single-GPU server, a tool question type, calibration you
can refit on your own data, and an experimental thinking mode on the 4.5B model. Models on Hugging Face:
Remek/basal-2.0-mini,
Remek/basal-2.0-4.5B,
Remek/basal-2.0-max
(basal-2.0 collection).

The same engine also serves basal-omni-1.0 (experimental): basal-omni-1.0-mini (4B) and basal-omni-1.0-max (12B),
typed-decision models built on Google Gemma 4 that can take images and audio in a request:
Remek/basal-omni-1.0-mini,
Remek/basal-omni-1.0-max.

Highlights

• Three models: basal-2.0-mini (1.5B), basal-2.0 (4.5B) and basal-2.0-max (11B), in bf16, FP8 (NVIDIA ModelOpt,
  served by the basal engine), GGUF (Ollama, llama.cpp) and MLX 8-bit (Apple Silicon).
• Werdykt (hidden Polish–English decision benchmark, 5,000 decisions): basal-2.0-mini 0.618, basal-2.0 0.729,
  basal-2.0-max 0.784, against 0.607 / 0.721 / 0.773 for basal-1.5 (+1.2 / +0.7 / +1.1 points). Calibration error
  0.035 / 0.060 / 0.044 against 0.086 / 0.068 / 0.074. Hosted API models and larger open decision systems score higher;
  the public board lists every tested system.
• Faster on one H100 (bf16, 16 clients, Werdykt speed pass): mini 106.4 decisions/s (basal-1.5-mini 20.5, ×5.2);
  4.5B 32.2 (basal-1.5 on SGLang 23.0, ×1.4); max 16.2 (basal-1.5-max on SGLang 10.2, ×1.6).
• Up to 255 options per question (basal-1.5: 10), read in one forward pass with one option order by default.
  basal-1.5 left 17.2% of the Decision Index (JDI 0.2.1) requests unanswered; basal-2.0 answers all but 26 of 150,317
  (mini) or all of them (4.5B, max). JDI: 26.69 / 39.86 / 44.07 (basal-1.5: 16.41 / 27.65 / –).
• Where basal-2.0 is not better: on a panel of 10 public English decision tasks all three are below basal-1.5
  (−2.1 / −3.6 / −1.8 points); basal-2.0-mini is below basal-1.5-mini on Polish decisions (−1.4 points); at the 1%
  target error on Werdykt, 4.5B and max automate fewer decisions than basal-1.5.
• Calibration per question type and option-count bucket, fitted on question families the models were not trained
  on; the shipped files certify the 5% error target, not the 1% target (except CALIBRATION.ollama.json of
  basal-2.0-max-GGUF, which certifies 1% for choice, confidence 0.989). basal-calibrate refits temperatures and
  certified thresholds on your own labelled requests.
• Thinking mode (basal-2.0 4.5B, experimental, off by default): --think auto writes a short reasoning note for
  uncertain questions on a vLLM sidecar. On Werdykt it lowered the macro score (0.719 vs 0.729), raised the calibration
  error (0.099 vs 0.060) and cut throughput to 11.5 from 31.2 decisions/s; we recommend leaving it off.

basal-omni-1.0 (experimental): images and audio

• Two models on Google Gemma 4: basal-omni-1.0-mini (4B, Gemma 4 E4B) and basal-omni-1.0-max (12B). They
  answer the same typed questions over the same API; with --media on a request may also carry images and audio
  ("media").
• Trained on text decisions only: Gemma 4's own encoders read the images and audio zero-shot, and answers with media
  use the text calibration ("calibration": "text"). Validate them on your own media before automating with them.
• --mode eager only, bf16 on an NVIDIA GPU: no FP8, NVFP4, GGUF or MLX ports and no thinking mode yet.
• Install the media extra and serve with --media on:

uv pip install torch==2.11.0 torchvision==0.26.0 --index-url https://download.pytorch.org/whl/cu128
uv pip install "basal[media] @ https://github.com/rkinas/basal/archive/refs/tags/v2.0.0.tar.gz"
basal-serve --model Remek/basal-omni-1.0-mini --mode eager --media on --port 8000   # or Remek/basal-omni-1.0-max

Details: images and audio and
docs/OMNI.md.

Install

uv venv --python 3.12 --allow-existing ~/basal-env && source ~/basal-env/bin/activate
uv pip install torch==2.11.0 --index-url https://download.pytorch.org/whl/cu128
uv pip install "basal[fp8] @ https://github.com/rkinas/basal/archive/refs/tags/v2.0.0.tar.gz"
basal-serve --model Remek/basal-2.0-4.5B --mode fast --template-prefix off --port 8000

Every engine (FP8, vLLM, SGLang, MLX, Ollama, llama.cpp) and the thinking mode:
README. basal-serve is the decision layer in every mode: vLLM, SGLang,
MLX, Ollama and llama.cpp run the weights, and the typed, calibrated answers come from basal-serve --mode <engine> in
front of them.

Engine v2.0.0

Models and formats

• Prompt format 2: up to 255 single-token option codes, the state before the question, one option order by default
  (--orders 1 | 2 | auto); runtime contract with test vectors in docs/FORMAT2.md.
• Calibration file 2: a temperature per question type and option-count bucket, thresholds certified per question type,
  gates and the multi fallback threshold; per-engine files (CALIBRATION.<engine>.json) and per-order files
  (CALIBRATION.o2.json).
• Option-code logits in fp32 in every backend (PyTorch, MLX).
• --mode fp8 loads NVIDIA ModelOpt FP8 checkpoints natively in the CUDA graph engine; --fp8-accum fast | full.
• vLLM reads more than 128 option codes.
• the basal-omni models (Gemma 4 architecture) in --mode eager, and with --media on and the basal[media] extra (pillow,
  soundfile, scipy, torchvision) requests with images and audio: "media", "detail" (image tokens) and "lang".
• --mode eager, every model: option codes read in fp32, left-padded prompts read at their own positions, prompts
  over 2,048 tokens prefilled in 2,048-token chunks, batches capped at 16,384 padded tokens.

Speed

• Compact batching (--batches compact, FlashAttention 3 varlen): +19% decisions/s at 16 clients, +17% at 32, +8% at
  one client (4.5B, bf16, one H100).
• The prompt template computed once at start-up (--template-prefix): +20% at 8 clients (mini, max).
• A scheduler for long requests (--scheduler): in a mixed load at 32 clients short requests answer in 0.81 s instead
  of 1.38 s (p50); long requests take about 7% longer.
• FlashAttention 3 on Hopper by default for bf16 weights when installed (FP8 keeps FlashAttention 2, the checked
  path, unless --attn fa3: 21–26% faster, but 3–8% of decisions differ, mostly near ties, and accuracy moved by
  −0.8 to +1.4 points on a development sample;
  --attn auto | fa2 | fa3 | fa4 | cudnn), merged q/k/v and gate/up projections for bf16 weights
  (--merge), pass-2 CUDA graphs for branched requests.
• Opt-in, off by default: --lean, --tree flex, --readout-layer, --state-cache-gb, --prefix-cache,
  --long-lane.

API and tools

• tool question type: the next tool, or answer directly / ask the user / no suitable tool, from OpenAI, MCP or
  Anthropic tool definitions; rank, "multi": true for a set of tools, max_error deferral (experimental).
• MCP for agents (experimental, basal[mcp]): basal-serve --mcp on also serves the decisions as an MCP server at
  /mcp (streamable HTTP; tools decide for one question and decide_many for every question type over one
  state), and basal-mcp --url <server> connects a running server to agent hosts over stdio.
• multi: set objectives iou, f1 and exact; exclusive and fallback labels.
• basal-calibrate: fits temperatures and certified thresholds on labelled requests, with the engine you serve.
• --deterministic: answers independent of load and of other requests; basal-loadtest --determinism N also counts
  decision flips under load.
• --context-overflow truncate | refuse: a state over the context loses its middle (marked in usage.truncated).
• --think off | auto, --think-tokens, --think-python (experimental thinking mode, basal-2.0 4.5B).
• basal-bench --identity: every prefix-reuse path of the fast engine against an eager forward of its own prompt;
  synthetic long-state items for speed runs; basal-loadtest --questions-per-request.
• GET /v1/models reports the temperatures in use.

Upgrading from v1.5.0

• Long states are cut, not refused. With the default --context-overflow truncate, a state over the context loses
  its middle and the answer says so in usage.truncated; --context-overflow refuse keeps HTTP 422.
• max_error defers without a certified threshold. When the calibration file has no threshold certified for the
  question's type and target, every automatic act / tool decision of that type is deferred; a threshold pooled
  over all types is never applied. With the basal-2.0 files, max_error: 0.01 defers every decision
  (except that one choice certificate of basal-2.0-max-GGUF in --mode ollama).
• Calibration and option orders. A calibration file's thresholds apply only to the option orders it was fitted
  with; served with other orders, the server keeps the temperatures, drops the thresholds and says so at start-up. A
  missing --calibration file is refused at start-up.
• Answers are not bit-identical to v1.5.0 (fp32 option-code logits, other GEMM row counts); decisions can change
  near ties. --deterministic is reproducible on a given GPU.
• basal-1.5 and basal-1.0 models are served unchanged (prompt format 1, two option orders by default).

Known limitations

• The calibration is cautious on new tasks: no 1% threshold is certified in the shipped files (except
  basal-2.0-max-GGUF's CALIBRATION.ollama.json, for choice); use 0.05 or fit your own thresholds.
• basal-2.0-mini has an 8,192-token context (4.5B and max: 32,768).
• The basal-2.0 models ship no evidence head: "evidence": true is refused for them.
• Ollama returns at most 20 log-probabilities, so questions with more than 20 options need llama.cpp or another engine.
• The basal-2.0 numbers were measured with the basal engine; vLLM and SGLang were checked against it with basal-1.5.
• act, tool, multi over many labels, the MCP server and the thinking mode are experimental. The MCP endpoint,
  like the HTTP API, has no authentication: bind --host 127.0.0.1 or put it behind a gateway.
• basal-omni is experimental: trained on text decisions only, images and audio read zero-shot and answered with the
  text calibration; eager mode only, without the ports and the thinking mode of basal-2.0.
• Decisions with serious consequences for people should be reviewed by a person.