Skip to content

Amada 3.0

Choose a tag to compare

View all tags
@DevLiamDb DevLiamDb released this 08 Nov 04:59
· 14 commits to main since this release
v3.0
2fc6015

Long texts, many options, trap texts. Starts from the 2.0 weights.

  • Added: 8192-token context. Long examples that do not fit are skipped, not cut.
  • Added: 42,870 many-option examples (3 to 386 options, 384 label sets, 8 languages), 4,875 trap texts
    (12 languages), long labelled texts (QuALITY, ContractNLI, Qasper, 5,864 long texts written by GPT-6 Luna).
  • Changed: token-budget batches (16,384), gradient checkpointing, lower learning rates, 2 epochs (about 9 h on a T4 GPU).
  • Results: v3 validation 0.846 overall. Banking77 0.337 -> 0.605, AG News 0.714 -> 0.790.
  • Known issues: calibration bug (fixed in 3.0.1). QuALITY 0.44, Qasper 0.57. The second epoch only overfitted.

Install: unzip into a Release/ folder next to play.py, then python play.py --version 3.0. Keep tokenizer/ next to amada_model.onnx.

License: CC BY-NC-ND 4.0 (non-commercial, no sharing of modified versions). Base model notices: NOTICE.md.

Checksums (check the file with Get-FileHash <file> in PowerShell or sha256sum <file>):

SHA-256  259b5756a83ca040ff80f6de30b672a8faa4a39e2015c27a2af3f2a5c75c5a07  amada-3.0.zip
Assets 3
Loading