Skip to content

v0.2.0

Latest

Choose a tag to compare

@snakajima snakajima released this 17 Oct 00:18
· 2 commits to main since this release

Calibration fix for large choice questions, closer parity with upstream laya, and a new answer_confidence field.

Calibration

  • choice questions with 11+ options are no longer ~99% confident. The upstream checkpoint ships choice:11+ = 0.1006, which multiplied those logits by ~10 (#10). The published bundle (receptron/laya-onnx) now uses 0.945, the held-out fit for the English checkpoint (NandhaKishorM/laya#1245). The cached config refreshes on the next Laya.load() with network access.
  • Temperatures are clamped to [0.5, 5] at load and again at the point of use, so a malformed bundle or a later edit to laya.config cannot bypass it. Out-of-range or invalid values print a warning; an invalid per-option entry falls back to the question type's temperature. Bundles without temperature_by_options (fine-tuned exports) now load (#11, thanks @kuishou68).
  • export/export_onnx.py replaces the known-bad choice:11+ value, so a re-export reproduces the published config.

Confidences for 11+ option choice questions intentionally differ from the Python laya package, which clamps the same entry to 0.5.

New

  • answer_confidence on every answer (choice, score, noul): max(p), the probability of the reported answer. It is the quantity the temperatures are fitted for and means the same thing across question types, so use it for confidence gating. confidence on choice / score is still 1 − normalized entropy.

Fixes (parity with upstream laya)

  • A single-option choice no longer crashes in the ONNX graph's TopK node.
  • Object instructions are serialized with Python json.dumps separators, matching the reference tokenization.
  • A list (conversation) state that does not fit keeps its newest turns, as in training.

Behavior changes

  • Duplicate or null labels in a list-form choice (["a", "a"], ["1", 1]) now throw instead of being silently merged into fewer options.
  • Long array states are truncated from the start instead of the end.
  • Answers for object-valued instructions shift slightly (they now match Python).

Full changelog: v0.1.2...v0.2.0