Skip to content

v1.4.1 — ORT optimization + ONNX GPU benchmarks

Choose a tag to compare

View all tags
@anulum anulum released this 01 Mar 08:27
· 3036 commits to main since this release
v1.4.1
1902d7d

Changes

  • ORT_ENABLE_ALL graph optimization on ONNX inference sessions
  • Suppress Memcpy transformer warnings (log_severity_level=3)
  • MiniCheck AggreFact benchmark (benchmarks/aggrefact_minicheck.py)
  • ONNX GPU batch benchmarks: 14.6 ms/pair (DeBERTa-v3-Large, GTX 1060)
  • Zenodo DOI: 10.5281/zenodo.18822167
  • Version bump to 1.4.1

Performance (FactCG-DeBERTa-v3-Large)

Backend Latency (ms/pair)
PyTorch GPU batch 19.0
ONNX GPU batch 14.6
Production NLIScorer 13.2
Raw ORT (short inputs) 9.0
Heuristic (no model) 0.03

Accuracy

75.8% balanced accuracy on LLM-AggreFact (29K samples, 4th on leaderboard)

Assets 2
Loading