Skip to content

OptimaMT 1.0.0

Choose a tag to compare

@AmerAlreyahi AmerAlreyahi released this 10 Nov 04:02
· 9 commits to main since this release

First release: 49 OPUS-MT translation pairs as ONNX models that use about half the memory in ONNX Runtime's fast mode, with identical translations.

Models: Hugging Face collection · one repository per pair, a drop-in folder for transformers.js and ONNX Runtime.

What changed

In the original int8 exports, the decoder builds its output layer in both branches of its If node, so ONNX Runtime's fast mode (session.disable_quant_qdq = 1) folds it to float twice. OptimaMT moves that subgraph in front of the branches, so it is folded once. No weight is changed, retrained or re-quantized.

Measured

Linux Windows
Fast mode memory, original → OptimaMT (median, 49 pairs) 525 → 281 MiB 500 → 257 MiB
Defaults memory, original → OptimaMT 277 → 217 MiB 196 → 166 MiB
ms/step, defaults → fast mode (median, 5 pairs) 17.3 → 6.9 21.3 → 9.1
Speed change from the rebuild none none
  • Same translations: FLORES-200 devtest, 49 pairs × 2 settings × 1012 sentences: identical output and identical BLEU and chrF in 98 of 98 runs.
  • Browser (transformers.js): fast mode 1288 → 1050 MiB after load (−18 %); no saving with the defaults.
  • The 2.3–2.5× speed-up is ONNX Runtime's fast mode; OptimaMT removes its memory cost.

Full method and every number: REPORT.md · interactive results and a live demo: OptimaMT/results

Licences

Code: MIT. Models: CC BY 4.0, OPUS-MT by Helsinki-NLP, ONNX exports by Xenova, modified (see NOTICE.md).