Skip to content

v0.3.0 - deterministic-first, local Laya backend

Latest

Choose a tag to compare

@JLegends JLegends released this 22 Sep 18:16

Replaces the model-judged decision core with a deterministic one, backed by a local open-weight model. No API key, no per-request cost.

Why

v0.1 asked a hosted model a judgement per tool call. Independent measurement showed why that fails: with a noul-style primitive, factual questions are reliable (0.996 on an explicit fact) and judgement questions are not (0.003–0.28). It also cost about $1/day — a 25k-token state resent ~1,000 times a day, against a saving of $0.0001 on a model whose cached input is $0.003/M.

What changed

  • Facts are computed, not guessed. superseded, error-resolved, referenced (exact string) and small-result involve no model.
  • Deletion requires deterministic evidence. A model answer can only ever cause a truncation, which keeps a bounded head plus a re-run note.
  • The residual question is a choice, not a noul. Measured against the same local model: a statement and its own negation both scored ~0.95, while the choice separated cleanly (0.75–0.99 on a real quote, 0.80–0.91 on unrelated text).
  • Payload per question: a few hundred tokens, down from a 25k-token wholesale state.
  • Defaults to local Laya via MLX (a few hundred MB of weights, single-digit-ms per decision), with scripts/laya-server.py as a thin Jev-protocol transport.
  • Degrades to deterministic-only when the backend is absent, and LAYA_COMPACTION=0 disables it.

Verification

All five policy branches exercised end to end, against a protocol stub and against real local Laya:

A referenced        -> kept
B superseded        -> dropped
C error-resolved    -> dropped
D small-result      -> kept
E large, unmentioned-> truncated (model answered "does-not")

Naming note

The package and repository still say jev-compaction; the plugin id is laya-compaction. The backend accepts any Jev-shape endpoint via LAYA_BASE_URL, so the old name is not wrong, just stale.