Repository navigation
Amada 3.0.1
Recalibration of 3.0. Same weights, no retraining, identical accuracy. Only the temperatures change.
- Fixed: the 3.0 calibration was fitted on soft targets. Every temperature was above 1 and the model was
under-confident: choice ECE went up even on its own validation set (0.032 -> 0.076). - Added: temperatures that depend on text length L, plus list size K for choice:
log T = a + l·log(L/512) [+ k·log K + m·log K·log(L/512)]. - Results (bake-off, English): ECE choice 0.169 -> 0.022, score 0.129 -> 0.021, yes/no 0.057 -> 0.010.
11 languages: choice 0.180 -> 0.020. - Known issue: synthetic "many" lists with 6-20 options are calibrated worse than in 3.0 (ECE 0.050 -> 0.081).
Install: unzip into a Release/ folder next to play.py, then python play.py --version 3.0.1. Keep tokenizer/ next to amada_model.onnx.
License: CC BY-NC-ND 4.0 (non-commercial, no sharing of modified versions). Base model notices: NOTICE.md.